Yes the fish would have no problem with out the anemone.However,if the fish have been with the anemone for a bit,they wont go down with out a fight.When I remove some of my RBTA clones I get the snot bit out of me cause they dont want me touching their anemone.
Just a word of caution!