My general assumptions for setting up vegetative filters that can effectively compete for nutrients are as follows:
1) Better growth conditions should be present in the filter than the tank. Ideally, light levels are higher, and flow rates (for nutrient delivery and gas exchange) are higher than in the tank itself.
2)Harvest of both plants and collected detritus is easy. The veg filter should be easy to siphon or drain. Surfaces should be easily scraped to harvest the plants. For these reasons, I don't put rock or sand in my veg filter, and prefer plastic or ceramic surfaces for the plants to attach to.
3) A community of plants is better than a mono culture. Plants don't have equal uptake requirements. When one plant is limited, another plant may be growing very well. As water conditions shift, a filter with a variety of harvestable plants can adapt better than a filter with just one specie.
4) Grazing of plants in the filter is as limited as possible. For nutrients locked in plants in the filter to stay locked in the filter until export, the plants must not be grazed until they are harvested.
Under the above assumptions, an Algae Turf Scrubber is an ideal veg filter. "Dynamic Aquaria", Walter Adey, describes the ATS. A general discussion of the ATS and other veg filter methods is in "Reef Aquaria Vol 3", Delbreek and Sprung.