weights
weights
You will probably need between 5 and 7 lbs. to achieve neutral bouyancy, maybe slightly more ofr less. It sounds to me as though you are essentially snorkeling, and want to dive 10 or 20 feet down, and be able to stay there without effort. I do this kind of free diving very frequently, but wear lycra skins, which do not affect bouyancy at all, and protect against the sun, jellyfish, etc.
The % of body fat affects bouyancy. I weigh 200 lbs, and use 7 lbs of soft weights, which I wear in a compartmentalized belt, to keep the shot bags distributed properly. I can float at the surface effortlessly, very easily kick down to look under a coral head, and stay there, perfectly still, no effort, the only limit my ability to hold my breath. A couple of kicks upward, and I'm back at the surface.