I consider who is doing the study and then even with a trusted source look at whether it's truly a scientific method or just uniquely anecdotal or in some cases just marketing testimonial or perhaps just a need to publish and practice pedantry.
I read carefully and check similar studies and observations. I ask: How does the claimed outcome happen? What are alternative explanations? Are the extrapolations reasonable. What are the external variables?
When I noodle around with different techniques and approaches I don't think of them as studies or experiments; just experiences in my specific system(s). When I observe something, I try to be sure my rose colored glasses are off and then start to ask why it happens :sometimes using a search of scientific studies and the experiences of others I trust to report accurate observations. I'm reluctant to put stuff out there as a recommendation unless specifically asked to do so in a specific situation but very open to sharing what I see once I'm sure I actually see it with others also searching for ways to improve the hobby.
I know my euphyllia did better in terms of growth higher nutrient systems as did my rbta based on years of observation . I'm not sure ,however, that other variables aren't the cause even though virtually everything else is the same. (Tap water may bring in nutrients in some cases ;on the other hand it may bring in toxins like free metals . I didn't use it in any case.) In my case perhaps, the increased bacterial activity from organic carbon dosing is altering the bio available organic mix in a way that comes up short in a specific type of organic for these creatures and not others; like aspartate, glutomic ammin oacid ,or a specific nutreint like nitrogen, nitite, ammonia, etc.
There are many "studies" ,"experiments" of dubious value and worse out there ,some are quite rigorous but still have little broad application. and are often extrapolated to generalizations which don't hold up.
Some examples:
One study claims bacteria are lower in our tanks than on the reef and a shortfall is deleterious to the food chain and nutrien nt management. I'm inclined to believe that butthe study doesn't prove it. In one section of the study increases in organic carbon dosing are compared to bacterial growth rates finding lsome increase in bacterial mass but less than expected , concluding low growth or some diminishing return.. In fact the whole study focuses on planktonic bacterial counts with pages of tedious reading about the methodolgy used to count them ;failing to consider that the bacteria involved are mostly non planktonic; they are benthic.
An example of a poor extrapolation is one case touting ammanox bacteria (anaerobes that directly take ammonia to di nitrogen and nitrogen gas) as a significant removers of nitrogen in the world's oceans of up to 50% of the total removal . Good so far; but then the extrapolation to reef tanks from waste water applications makes no sense since these bacteria occur in highly anoxic areas deep in the ocean floor or in places like the Dead Sea or specialized vats pumped up with sludge and don't compete very well with ammonia oxidizing aerobic bacteria which are plentiful in our tanks. If we did get lot's of ammanox activity our tanks would be dead already or die from the hydrazine( highly toxic jet fuel ) these bacteria produce as a by product of ammonia reduction .
There are also many , many studies that are very helpful if the extrapolations are on point. Insights can be gained from reading them if you can get past the way they are written. Seems like deliberate efforts are made to obfuscate in may of them ,probably to avoid a challenge or maybe it's just really hard to apply a finding to the hobby.
