No, I have never worried about it in the past. At least, it doesn't affect a matured screen, and anything that clings to the screen is pretty negligible.
If you wanted to have peace of mind about both not killing off algae and not introducing any chlorinated water (more important if your muni uses chloramines instead) you can draw a shallow pan of water out of the tank into a flat bottom container like a sterilite storage box and then place the screen in that and just give it a swish and after the palm swipe and you're good.
I read somewhere that marine algae exposed to FW can kill it (something similar to osmotic shock) so cleaning and rinsing in SW is what I recommend for screens that don't want to mature. But the vast majority of the time tap water is fine