If they're in constant contact with the crown it might p!$$ the Toadstool off, but if they're just growing around the base it shouldn't be a problem.
FWIW, I've got a Toadstool that has an Acropora, a Montipora, some Zoanthids and a Chalice touching the base and it doesn't seem to mind at all.