Removing an anemone takes some patience. Basically, you need to gently prod the base of the anemone with your finger until it lets go. If its new, I would wait a while as it is likely stressed.
Personally, rather than remove the nem, I would consider dealing with the pests themselves one by one. If its aptasia, by using kalk powder, etc...