You could definitely leave the sand and the rock, I would just be concerned about the rock falling (assuming you have it stacked) and that resulting in scratching, or even worse, chipping the glass. Nothing is worse than a scratch on one of the viewing sides.