Chlorine does not take two days to kill, but some other things may, or something that just kill enough animal that start a crash a day or two later.
If the only thing common is top off water, then top off water is what cause the problem, or something in the air.
Good luck.
BTW, I think tap water is find, but if I use tap water for top off, then I would change water much more often.