First off, vacuuming your sand is something that should only be done in freshwater tanks, not saltwater. By vacuuming it you are removing as many beneficial bacteria as you are particles of detritus. In the long run, it's not good for tank stability. The more flow you can get in your tank the better. Shoot for somewhere between 30-50 tank turnovers per hour ( so between 1650 - 2750 GPH) for a reef tank. Make sure your skimmer is designed for at least twice the system capacity you have, preferrably 3 times the capacity. Try feeding less often, as most people over feed their tanks. Finally, I found a refugium to be a big help with nitrates and phosphates. I'd go somewhere around a 20G refugium for your system. Hope this helps.