I assume you're doing all the normal stuff, like wearing subdued clothing, and sticking to a closely timed routine?
You might try wearing the same shirt when you approach the tank (to help with recognition). Maybe try taming them to hand feed as time permits in your 5 minute sessions?
I think you are right to leave as soon as she gets agitated. I don't know if it works for fish, but with dogs and cats to desensitize them, the goal is to stop *just before* they get upset.
If that doesn't help, perhaps when they move out in the display tank, they will get a chance to get used to human activity around them from a safer distance.