Peabody,
Is your clam showing any signs of stress? If your clam appears normal, I would leave sponge alone. If you are concerned and do not like the look of the sponge, carefully scrape it off, and disacrd it.
Is shrimp still inside of clam? Wish I could see.
Rob