In Rokugan, there are two problems with physical contact with other samurai:
The specific taboo of touching another samurai
Even if it is the affectionate kind of contact, like holding hands with your spouse, a father patting his son's head, or a hug between brothers that have not seen each other in a long time, if you are being observed. This means that many samurai refrain for doing this even in the most private circumstances, simply because they can never be sure that nobody is watching at them. Note that any other type of contact, such as helping a fallen man to stand up, shoving another person out of the way, or grabbing someone to stop it from moving or falling, is always done only in case of need or duty and always avoiding skin to skin contact. Otherwise that could bring consequences, from a stain on both samurai honor to the one being touched perceiving it as dire insult.
Familial touching is often a show of affection
And, for many samurai, that alone is a sign of weakness of character. That's something no samurai would ever want to show, even to his own family. This prevents most samurai from hugging, kissing or doing many other things that would be considered normal to most players mindsets, even in the most intimate circumstances.
This mean that any kind of physical contact has negative connotations and, as with many other things in Rokugan, is restricted to the most private moments or avoided completely.