I believe that a lot of sponges contain distasteful or toxic chemicals to help ward of predation -- remember, out in the ocean there are all those angelfish that feed on sponges. I do not think that most sponges are toxic in the sense of the "sea apples" (Pseudocolochirus spp.), which can release the toxin holothurin if stressed and potentially kill fish tankmates.