This is actually fairly normal behavior for many Xenia species. In the wild, they grow like crazy, almost exploding, it seems, out from the mother colony. This is usually in the spring. Then, towards the summer, they die off, as whole colonies crash. No one is entirely certain why this occurs. In the home aquaria, where conditions are more sustainable, colonies may grow and spread for 2-3 years before ever exhibiting anything at all like a crash or meltdown. Various techniques have been tried to mimic natural environments in an attempt to avoid these seemingly regular (although, rather irregular in the home aquarium), however, it still seems like colonies will melt down or crash at random. =/
What to do in the event of a colony crash? Maintain your water quality, noting that larger colonies can most certainly turn tanks, especially smaller tanks, putrid with rotting matter. Keep the remaining polyps and stalks as clear of decaying matter as possibly by gently blowing or wafting the dying polyps away. And, above all else, cross your fingers.
G'luck!