A lot of people like the various species of hermit crabs and they do well in various aquariums, even some reefs.
But in answer to your question: YES.
Many hermits have proved to be Not reef safe. Some will pick at and eat corals.
It's hard to say really. they'll eat algae and detritus, scraps of food. Anything they can get to clean up easily. Sometimes they turn on some corals after getting a taste for them somehow.
Many reefers don't keep them now because of this experience.
I can't say for sure in your case exactly what they are doing and I'm not trying to alarm you needlessly. But I'd just watch their behavior though and if things seem to be taking a turn for the worse with a coral after they've been picking on or near it- for sure.. then get rid of the hermits.