What we test in the water

[Habib]

Ron,
I have a couple of questions regarding the water samples and test methods which were used.

1] Were the samples in which nitrite, nitrate, total N, proteins and such were determined, were these samples preserved?
Also what was the time between sampling and measurement?

2] Were the samples in which trace elements were measured acidified just after sampling and were the containers tested for leaching and adsorbing ions?

3] Was the alkalinity measured by the hobbyists at the moment of sampling?

4] Were the calcium measurements conducted by the hobbyists just before sampling?

5] Is it possible that the samples, especially the samples in which calcium was measured by ICP, were exposed to low temperatures (say below 5 dgr C)?

6] Were the calcium measurements by ICP conducted on the undiluted sample or were the samples diluted. If so what was the dilution factor.

7] Did the lab use excisting calibaration line for the calcium measurement or did they prepare a new one and was the calibration done using a marine water matrix or was it done by standard addition?

8] What is the standard deviation of the calcium measurement and what were the statitical data of the calibration lines?

Sorry for asking so many questions but since they are more or less standard things with such measurement I assume that either you or the lab can answer them easily.

 

[rshimek]

Re: What we test in the water

Originally posted by Habib

Hi,

1] Were the samples in which nitrite, nitrate, total N, proteins and such were determined, were these samples preserved?

They were frozen, but after a couple of days in shipment to me. Once I got them they were frozen and remainded so until analysis. However, there will some errors introduced by this lag time.

Also what was the time between sampling and measurement?

A week to 10 days.

2] Were the samples in which trace elements were measured acidified just after sampling and were the containers tested for leaching and adsorbing ions?

The containers were provided by the analytical laboratory and I presume they were previously tested. The samples were not acidified. Three different containers were used, all filled at the same time. The laboratory felt that as the hobbyists were not "professionals," they did not want the liability for any potential problems developing from the potential misuse or accidents occurring with the acids. The laboratory personnel also indicated that they thought the errors introduced by the lack of acidification would be inconsequential.

3] Was the alkalinity measured by the hobbyists at the moment of sampling?

No.

4] Were the calcium measurements conducted by the hobbyists just before sampling?

I presume so, that was the instruction.

5] Is it possible that the samples, especially the samples in which calcium was measured by ICP, were exposed to low temperatures (say below 5 dgr C)?

Yes.

6] Were the calcium measurements by ICP conducted on the undiluted sample or were the samples diluted. If so what was the dilution factor.

The sampling methodology references are given. You can examine them.

7] Did the lab use excisting calibaration line for the calcium measurement or did they prepare a new one and was the calibration done using a marine water matrix or was it done by standard addition?

The lab did these analyses as per the protocols listed for marine samples. They are an analytical lab specializing in the analyses of sea water samples for environmental sampling. However, I don't the specific answers for this. See the listed protocol.

8] What is the standard deviation of the calcium measurement and what were the statitical data of the calibration lines?

This was not reported for the individual samples, and are not part of the standard reports. For the total test across all individuals, the mean and one sample standard deviation are given in the report.

 

[Habib]

Thanks Ron.

In the April Issue of Reefkeeping Online Magazine you write:

"The major tool that aquarists have for ascertaining the calcium concentrations of the water in their systems is a calcium concentration test kit, and there are several of these on the market. To test the validity of using these test kits, I asked the participants in the survey to indicate if they used a test kit for calcium and, if so, what was the tested value for the calcium concentration in the water sample. I also asked the participants which test kit they used. These tested values were compared to the actual values (Table 5). "

In this table you are presenting the ICP value as the actual concentration. I would however like to make some remarks.

According to the Lab where the ICP measurements were carried out the error in the Calcium ICP measurement is -8% - +8%.

So as an example a value of 400 ppm in your table could be any value between 368 - 432 ppm!

Furthermore alkalinity of the samples were not measured and it is not unlikely that at least some samples contained calcium and alkalinity above or against saturation values.

This could have resulted in precipitation of calciumcarbonate (that is a lowering in calcium concentration AFTER the samples were taken and measured with a test kit.
This especially because the samples were not acidified immediately after sampling, a long time between sampling and measurement and possible temperature effects.

Please correct me if I am wrong but I think that first the ICP measuremnt for Calcium in your study as presented in the April number has a very significant "error-range" (see mine comments above). This should be shown clearly in the table.

And secondly there might have been calciumcarbonate precipitated in some samples. This would imply that the dissolved calcium concentration in the sample was higher at the moment of sampling and would have become lower at the time of ICP analyses.

Since samples were not acidified and alkalinity was not measured at all, it is impossible to say in which samples calciumcarbonate might have been precipitated.

Theoretically (according to me) the possibility excists that the results obtained by test kits were (virtually) all correct (say +/-2%) and the ICP measurements were incorrect.

Do you share the above views or do you have a different interpretation of the results?

Thanks

 

[rshimek]

Originally posted by Habib

Hi,

So as an example a value of 400 ppm in your table could be any value between 368 - 432 ppm!....This could have resulted in precipitation of calciumcarbonate (that is a lowering in calcium concentration AFTER the samples were taken and measured with a test kit.

Yes, there is always a sampling and measuring error. For the purposes of this particular question it was irrelevent. I assumed sampling errors were random and cancelling.

The total sample calcium was measured. There were no precipitates in any of the samples.

Theoretically (according to me) the possibility excists that the results obtained by test kits were (virtually) all correct (say +/-2%) and the ICP measurements were incorrect.

I think the probability of this interpretation being correct is vanishingy small, and than such an occurance is very unlikely.

 

[Habib]

Ron,

Yes, there is always a sampling and measuring error. For the purposes of this particular question it was irrelevent.

The question was (from your article in April Issue):

To test the validity of using these test kits, I asked the participants in the survey to indicate if they used a test kit for calcium and, if so, what was the tested value for the calcium concentration in the water sample. I also asked the participants which test kit they used. These tested values were compared to the actual values (Table 5).

So you are saying that you are ignoring errors in the ICP measurement results which for your samples (according to the lab where you had them analyzed) have an error of +/- 8%!
I am, for the sake of simplicity, ignoring calcium carbonate precipitation (that would have made the ICP results even worser).

When people would read your article and would view the results in table 5 (April Issue Reefkeeping...) they would assume that the ICP value are very very accurate and would come to the conclusion that the test kits are totally wrong.

In fact you even endorse this (in my opinion) totally incorrect conclusion by saying (just below that table 5.) :

Unfortunately, if these aquarists had to depend on the test kit values as being anything other than approximations they would be out-of-luck.

If you would not ignore the measuring error for your ICP analyses for calcium then it would be clear that MOST of the ICP results in that table CORRESPOND to the test kit results or v.v.!!!

You consider it irrelevant but it is according to me not irrelevant but highly relevant because it leads to a totally different conclusion.

And now to the point of calcium carbonate precipitation.
It is not unlikely that calcium carbonate was precipitated. This has been described in an earlier post (this same thread).

The total sample calcium was measured. There were no precipitates in any of the samples.

Probably no one can and will analyse the whole sample by ICP.
A small amount of sample was taken and diluted (100x) and acidified (after taking the sample).

There were no precipitates in any of the samples.
Even if the samples were visually inspected for precipitates it still does not mean that no precipitates were present (conatiner wall, microcrystalline,..).

To conclude this post:

From your article

This study showed that "the hobbyist-calcium test kit combination" is not one that leads reliably to any meaningful answer.

If the errors in the ICP analyses would not have been ignored then it is very likely that above given conclusion could not have been drawn in THAT article.

 

[rshimek]

Originally posted by Habib

Habib,

So you are saying that you are ignoring errors in the ICP measurement results which for your samples (according to the lab where you had them analyzed) have an error of +/- 8%!

I am saying that the ICP error is small and random and would be neglible. Contrary to your supposition, there is no evidence that it is not. The hobbyist/test kit error is much greater.

When people would read your article and would view the results in table 5 (April Issue Reefkeeping...) they would assume that the ICP value are very very accurate and would come to the conclusion that the test kits are totally wrong.

The conclusion as I stated is that the test kit/hobbyist combination gives variable results that are statistically indistinguishable from using no kit at all.

Unfortunately, if these aquarists had to depend on the test kit values as being anything other than approximations they would be out-of-luck.

Yup. That's what I said.

If you would not ignore the measuring error for your ICP analyses for calcium then it would be clear that MOST of the ICP results in that table CORRESPOND to the test kit results

You presume there is this error, and that the test kit readings were within +/- 8% of reality. This is not the case. Most of the readings were outside of that error window, if indeed it even was that large.

You consider it irrelevant but it is according to me not irrelevant but highly relevant because it leads to a totally different conclusion.

You are entitled to your opinion. I think it is the wrong one.

And now to the point of calcium carbonate precipitation.
It is not unlikely that calcium carbonate was precipitated. This has been described in an earlier post (this same thread).

There was no precipitate in any of the samples.

This study showed that "the hobbyist-calcium test kit combination" is not one that leads reliably to any meaningful answer.

If the errors in the ICP analyses would not have been ignored then it is very likely that above given conclusion could not have been drawn in THAT article. [/B]

That is your opinion. I think it is wrong.

I suggest you run similar tests with 20 hobbyists scattered the breadth of Europe and see what your results are.

 

[rshimek]

Habib,

Some further points.

All the samples were acidified at the lab eliminating any precipitates. Each sample is run 3 times, and indeed the variance is 7-8%.

On the odd chance that this would change the results of the analyses, I reanalyzed the results skewing the data by 8% toward the test kit value (basically this assumes the test kit gave the correct value). I then subtracted the test kit reading from the ICP reading. If the value was within +/- 10%, I called the difference zero. This significantly biases the data toward the values given by the test kits (in effect it says the kits are always right.

That gives the following resulting values:
The numbers in each column are the difference between the ICP values and the test kit values for column 1, and between the ICP values and 410 for column 2 (participants not using test kits).
Test kits No test kits
-132.800 .......... 40
-89.600.............-110
-82.000............. -60
-66.400.............-100
0.000............. -30
0.000............. 40
0.000.............150
0.000.............110
0.000............. 90
0.000............. -50
0.000.............-200
0.000............
0.000.............

I ran a 2 sample t-test on those data, testing as before.

t-Test: Two-Sample Assuming Unequal Variances
Variable 1 Variable 2
Mean -28.5 -10.9
Variance 2185.05 11369.09
Observations 13 11
Pearson Correlation NA
Pooled Variance 6359.61
df 13
t -0.508
P(T<=t) one-tail 0.309
t Critical one-tail 1.770
P(T<=t) two-tail 0.619
t Critical two-tail 2.160

The final results are effectively the same. There is no statistical difference between the two groups.

So... for whatever reason, the hobbyist/test kit combination results in no better realization of getting to NSW values of calcium than does using no test kit at all.

Notice, I didn't say the kit was inaccurate. In point of fact, they may be quite accurate - but with a large minority of users, the test kit/hobbyist combination results in inaccurate data.

 

[Habib]

Dear Ron,
and indeed the variance is 7-8%.

Thank you.
It is important that you are confirming this.

Ron one of your interests is:
Aqueous ethanolic solutions of a barley malt origin....
This is also one of my interests. But could you please leave that aside for a moment.....you will then see that I am constantly referring to Table 5 and NOT table 6 as you are doing

Table 5 is : Differences between the values determined by ICP and test kits for Calcium concentrations in ppm.

If you would insert in Table 5 (not 6) the range of the ICP values +/- 8% you will note that most test kit values are within that range (if you would allow +/-3% or so range for the test kit).

A few samples are totally out of range.
This still could be precipitation.
Yes, I know that you say:

All the samples were acidified at the lab eliminating any precipitates.

But my understanding is (after talking with the lab which did the measurements) that a small sample was taken from the UNACIDIFIED 500 ml sample bottles. That small sample was diluted and that one was acidified.
So precipitates is still a possibility for the few remaining totally out of range samples. But it could also be a bad test kit or a bad test kit / hobbyist interaction.

Notice, I didn't say the kit was inaccurate. In point of fact, they may be quite accurate - but with a large minority of users, the test kit/hobbyist combination results in inaccurate data.

I think that modifying table 5 (again not 6 ) will show that the test kit/hobbyist combination is in most cases at least as good as the ICP measurement.

Perhaps you know or maybe even not but I am the manufacturer of the Salifert Test Kits. Since I know these test kits very well I was surprised by the results presented in table 5.

If the ICP results would not had a variance of +/-8% but say less than 1 or even 2% then either the test kits would not be what they should be and/or the combination with the hobbyist would have been bad.

In either case I know my responsibility.
This is the reason why I want to have it very clear not only for myself but also for the hobbyists which use them and pay for such test kits. Especially those who participated in your test program.
I hope you will understand this.

Now lets have some :beer:

 

[rshimek]

Originally posted by Habib

Habib,

But my understanding is (after talking with the lab which did the measurements) that a small sample was taken from the UNACIDIFIED 500 ml sample bottles. That small sample was diluted and that one was acidified.

The laboratory manager indicated to me that the entire sample was acidified.

I think that modifying table 5 (again not 6 ) will show that the test kit/hobbyist combination is in most cases at least as good as the ICP measurement.

No, it doesn't. As long as the mean is a good indicator of central tendency, the values indicated would still stand, and my conclusions would not change.

Additionally, as indicated in the post above, the magnitude of the discrepency for several of the tests is still great enough, even when all the values near the range are considered have a zero difference that statistically there is no difference between the accuracy of the aquarists using a test kit and those who did not.

I have requested the editor change Table 5 and the following paragraph to read:

Table 5. Differences between the values determined by ICP and test kits for Calcium concentrations in ppm. The potential experimental error for the mean ICP reading is +/- eight percent.
Mean ICP Value.......Test Kit......Difference
+/- Potential ...........Value
440+/-35................ 608........ -168
380+/-30................ 500........ -120
350+/-28 ................460........ -110
320+/-26................. 412......... -92
490+/-39................. 540........ -50
440+/-35................. 450........ -10
320+/-26................. 325........ -5
430+/-34................. 415......... 15
400+/-32................. 380........ 20
350+/-28................. 325........ 25
460+/-37................. 430......... 30
320+/-26................. 285........ 35
440+/-35................. 390......... 50

Amongst the users of test kits the average calcium concentration was 395.4 ppm, so the overall average was pretty good. Unfortunately, if these aquarists had to depend on the test kit values as being anything other approximations they would be out-of luck. The average difference between the test kit value and the tested mean value was 56.2 ppm. It would appear that the combination of aquarist and test kit gives the correct value Ã"šÃ‚± about 56 ppm. The potential ICP test variance of about eight percent is indicated and the ICP values should be considered to be within that range. Unfortunately, there is no way of predicting the direction or actual magnitude of the error. Consider the last value in Table 5, for example, the ICP reading was 440+/-35, so the true value is likely to be anywhere from 475 to 405. Given the test kit reading of 390, the aquarist could be pretty close, if the actual value was 405, or way off base, if the reading was 475. The experimental error for the test kits is unknown and will also confound the accuracy. Aquarists using test kits should aim to keep their tanks calcium levels in the middle of the normal calcium range.

 

[Randy Holmes-Farley]

If I look at that table, and allow for an 8% error in the ICP result and a 10% error in the kits, 4 of the samples did not have overlapping ranges, and 9 did.

To me, that seems pretty darn good for tests done far apart in time, maybe not on exactly the same water, by 14 different operators, using different test methods.

 

[rshimek]

Hi Randy,

Mebbe so, but the ones that were off --- were so far off that it made the two groups indistinguishable statistically.

I think the test kits are fine, as far as they go.

I think that probably most aquarists can read the tests correctly, however, I also think that there is a sizeable minority of aquarists that cannot - for whatever reason - do this.

From posts over the years, I don't think most aquarists find many of these tests easy to read or work with, but they learn to do so, or simply give up.

If the test kits are important - and for calcium they may well be - I would like to see tests less prone to lead aquarists astray.

This having been said, I don't think the kits can be made to be fool proof. As the saying goes, fools can be most innovative at finding ways to defeat foolproof solutions.