All the polyps are kinda partially retracted...
Could be that the sponge and the worm have nothing to do with that.
I don't remove sponges and worms from my zoas unless it's in a great number and they are really suffocating them, which never happened before here.
Normally algae is a problem and some species of sponges. Yes they could irritate the polyps.
Do you have more zoas in the tank? How are they doing?
What are your parameters, light, etc... ?
If you want you can remove them with the tweezers and see what happens. We do hope that should do it, other wise the problem would be probably chemistry or light.
Are the zoas new in the tank?
How long do you have the worm and sponges there?
Were they retracted all that time?
Grandis.