They are vermetids, and you did the right thing to take them out. Once they multiply, it gets pretty hard to get rid of them. And they can irritate a nice coral enough that it can be the difference between thriving and not. I left mine alone until they got out of control, and it was a ton more work and trouble, and even now I have them in hard-to-reach places.