After reading several other threads on various sites, I believe this was a fungus of some sort. From my reading, sponges grow around the polyps but don't directly damage them. Whatever was on this rock was dissolving the polyps.
I took the rock out of the tank, broke a few pieces off the clean end of it and then trashed the rest of it. Whatever the white stuff was basically turned to a slimy mucus by the vibration of chiseling through the rock. Definitely not a sponge.