It is always a good practice to wash your hands. Most people do not know this, but hand washing is probably the best public health measure that has been enacted in the past couple of centuries.
Having said that, the entire world is covered by microorganisms, including in ocean water. Think about how we desire to populate our tanks with bacteria to process nitrogenous waste. Washing your hands with soap and water does not sterilize your hands unless you use special soaps and wash for at least 30 seconds. Even then your hands get re-populated with microorganisms very quickly.
While I am unaware of any studies looking at cross infection between human and ocean based micro-critters, I suspect that the risk of cross infection from casual contact is very low. That's because complex organisms have well developed defenses against infection and few microorganisms are actually pathogenic (disease causing).
So I recommend that you be careful and practice good hand washing techniques, but do not get paranoid about it.