Could be sponge. I had some on a rock that was starting to take over some of my zoas. There was something else bugging them as well, I took it out, exposed the coral to air, dipped in coral rx and the sponge and the other thing that was bothering them went away and the zoa is happy now.
I think exposing sponge to air will cause it to die off. Just something to try and I believe in general its pretty harmless.