Caulerpa is loaded with fairly potent toxins (caulerpins) and is not eaten by many grazers. As you pull away the Caulerpa when hand-harvesting, its believed that you release a large amount of caulerpins into the water as the algae is torn. I don't know if this is true, but when I use to keep caulerpa, I removed it from the tank to harvest it. Caulerpa is reputed to be - in general - a "leaky" algae and if you have a lot of it in your tank you will have a lot of caulerpin in your tank, anyway.
I especially dislike Caulerpa's habit of rapidly spreading across rock by sending out runners. In a refugia this is acceptable but in the display tank you constantly have to fight it to keep it from dominating in competition for the rock surface. I would recommend that you remove it entirely from the rock, myself. If you'd like to keep growing it, shove some into the sandbed where it will send down holdfasts to anchor itself. Or grow it in a refugia.
I would recommend removing the effected rock from the tank and pulling off the caulerpa. Then i would wire brush the area where holdfasts attached the caulerpa to the rock. Rinse the rock with saltwater until it is clean of debri and return it to the tank. This should remove caulerpa completely, IME.
I have not used them, but have read that lettuce sea slugs are an effective control on caulerpa. I doubt that they would erradicate it. IME tuxedo urchins won't touch it. Diadema urchins might. I don't keep gazing fish but have read that most (none?) won't graze caulerpa. It is fairly easy to rid the tank of it by hand, IME, and that is how I would recommend doing it.
