It can happen from time to time. I had mini stars, bristle worms and the stomatella on various seahorses and I never notice any ill effect. However other people had had problems. Most time, if my seahorse didn't like it, they will try to shake or rub them off.
Kind Regards,
Tim