I kind of have the same experience. The only anemone I ever had that wandered on me is my ritteri, which, when it was in a different tank than it is in now; that tank had 2 lights over it, and the anemone wandered on me when I had a ballast problem and one of my lights (the one directly overhead the ritteri) didn't light up. In the half hour after the lights turned on, that it took me to notice that the light hadn't started, the anemone had wandered over to the side with the working light.
I have 2 gigantea's and only one of them ever wandered. The green one, when I bought it, was attached to a seashell. As the anemone grew, he eventually outgrew his seashell and sought a different attachment rock. So I think he wandered all of about 6" in one weekend before finding a rock that suited him.
Other than that they are almost exactly where I put them, in two years of keeping one and in three years of keeping the other.