Oh... could be so hard to figure that out!
Check your alkalinity and Mg!!! Make sure pH is between 7.9 to 8.4.
Are they getting good water flow?
Make sure there is nothing irritating them, and no predators. Check at night!!
Assuming all the parameters around normal and acceptable ranges, no predators and no poisoning of any sort...
Do you target feed them?
I would say it could be a nutrient deficiency, and I would target feed them to check that out. If so, they will probably come back slowly. You can start offering them a quality coral food like: Reef Roids, Coral Frenzy or Fauna Marin zoa food.
Normally the polyps grow good, but after a while they get weak and loose their ability to close on food or touch. After that they begin to shrink and loose their colors a bit. They need food particles!!!
If you wouldn't target feed them, you won't know how good that would be to them.
Go slow and offer only what they can ingest. Once a week is enough.
That's my $.02.
Grandis.