Yeah, I had a tuft of that on a monti frag.
Nothing really touched it, that I saw, and I have tuxedo urchins, turbo snails, ceriths, nerites, trochus, and astreas. So I just chipped off the piece of rock it was on.
A combination of nutrient control, tweezers...and rolled up sleeves is what you need to do.
Do you have a fuge and skimmer? Both will help prevent the spread of algae in the main tank.