Best to rinse it with ro/di. If you don't, you can end up with brown water and a soupy mess. Plus algae is fueled by phosphate that comes in via sand and live rock, so you might as well get rid of all the fine particulate you can, which dissolves easily.