Def remove the rocks theyre on.
many will say to boil and scrub the rocks,I HIGHLY suggest NOT doing this as zoas and palys release toxins, and when combined with steam from the boiling water those toxins mix with the air and can enter your respiratory system and easily cost you your life. I have seen a couple of threads here on this where people have been in the hospital clinging to life.
I'm not an expert on the subject but do recommend do a search or starting a separate thread regarding the issue.
HTH