IME, no they're not "safe."
With SPS, as lilchris menionted they pester them. Not only do they perch on them, but my pair would pick at them non-stop, not necessarily at one species but they'd share the love :mad2:. It didn't appear they were eating the polyps themselves (no polyps ever went missing, just retracted), but the slime off the acros. I can only guess they eat the food out of it that the coral catches.
I ended up getting them out, and PE returned to full force on my acros. If you have a large reef with large colonies I dont think one or a pair would be as bad...they'd have much more to pester.