Test Kit Comparisons: Calcium and Magnesium

[Whitebeam]

I have been a professional Test and Evaluation Engineer for the last 24 years. For the last year and a bit I've been looking at recommendations for test kits on here and I can't help wondering how much evidence there is for some of these - I just had to test the claims for my self!

With some money received for Christmas I went out and bought (or in a couple of cases, begged) the following test kits and reference solutions:

Calcium tests: Red Sea & Salifert (Coral shop just acquired - will report shortly)
Magnesium tests: Coral Shop, Red Sea & Salifert,
Reference solutions: Coral Shop, Fauna Marin & Salifert

All tests were conducted using the same methods.

For the Red Sea & Coral Shop tests I used the Red Sea glassware and titrator; for the Salifert tests I used the plastic vials that come with the tests.
All syringes used were those supplied with the individual tests.
All syringes and vials were washed in 0TDS RO/DI water and thoroughly dried before use.
When drawing sample water, all air was expelled from the sample syringe, but great care was taken not to expel any of the water remaining in the syringe tip when placing the sample into the vial.
Titration drops were initially added in groups of five and then three with constant mixing.When getting nearer to the expected end point, drops were added singly and the vial agitated for at least 10 seconds to give the reaction time to stabilise.
For the Ca tests, the end point was taken as the first blue colour which did not show any trace of pink 10 seconds after the agitation was completed.
For the Mg tests, the end point was taken as the first pure blue colour.

Units reported are 'ppm' in the precise units identified in the calibration charts for the individual tests or clarified by the kit manufacturer after the fact. Errors reported are KIT READING - TRUTH, i.e. a error of -30mg/l indicates that the test under-read by 30mg/l.

The Calcium results were as follows:

Kit Brand   Error vs CS Ref   Error vs FM Ref   Error vs Salifert Ref
Coral Shop        tbd               tbd                 tbd
Red Sea         -30mg/l           -32mg/l             -22mg/l
Salifert          0mg/kg          -12mg/kg             -7mg/kg

The Magnesium results were as follows:

Kit Brand   Error vs CS Ref   Error vs FM Ref   Error vs Salifert Ref
Coral Shop       40mg/l            51mg/l              50mg/l
Red Sea         -30mg/l           -14mg/l               5mg/l
Salifert         -5mg/kg            8mg/kg              5mg/kg

I have not been able to do a sufficient number of repeat tests to give a rigorous statistical evaluation of the spread of errors from these tests, however any 'odd' looking results were repeated to ensure no fundamental errors had been made. For this reason I suggest that all results above should be read with an error margin for 'experimental noise' of +/- 1 drop of titrant - translating this to the equivalent values for each test gives a suggested error margin of:

Kit Brand           Ca                  Mg
Coral Shop       +/-5mg/l            +/-15mg/l
Red Sea          +/-5mg/l            +/-20mg/l
Salifert         +/-5mg/kg           +/-15mg/kg

I intend to expand the range of test kit manufacturers included in this comparison as money to buy new kits allows me.

Peter

 

[Fryman]

Wow, that's surprising! There's a BIG difference between kit prices, but I'm not aware of any independent analysis of actual test accuracy & variation.

Salifert kits are cheaper, right? These aren't the expensive Red Sea "pro" kits are they?

I'm interested in these results, & any stats if avail. I'm no expert but I have Minitab, with enough training/expertise to be dangerous. If I could make a request, it would be for the Hanna Ca & phosphate testers & the Red Sea "Pro" kits. I like the hanna digital meters for the numerical results rather than color matching but I'm not sure about the accuracy.

Also, I have a hard time believing the price premium of the Red Sea pro kits is worthwhile.

I would even be willing to "subsidize" some testing! Send me a PM if interested.

 

[Whitebeam]

These aren't the expensive Red Sea "pro" kits are they?

Yep - the Pro kits.

Here's a price comparison:

Looking at (for example) Mg tests from one UK mail-order supplier:

Coral Shop =£19.99
Red Sea = £19.95
Salifert = £10.99

Peter

 

[Ron Reefman]

Peter,

I'm really glad to see your test results and want to encourage you to both continue testing and showing your results. I don't think I've ever seen anybody do this kind of a side by side comparison. And for that I say "Thank you!"

I'm of the same mind as fryman, in that if there is anything I can do to help you with this trial, just send me a PM. I'm not rich, but this is testing I think we can all appreciate. In fact, given the number of questions reefers have posted on RC about which test kit is best or I tried 2 different kits and got these wild different results, I'm surprised there aren't more posts. I'm definetly subscribing to the thread!

Good luck and keep up the good work.

 

[reefnewbie83177]

I think what is really tough with all of these test kits is that we are relying on the individuals eyes to determine what the color is. As well, measurements are not exactly the same every time. I use the red sea pro for my mg/ca/alk, and with the Ca, I notice that as I add the third titrating reagent, I get purple and then blue, but after a couple of seconds the blue will fade to purple, and then I add a couple more drops and it goes to blue and stays blue. The measurement of the dry powder is not exact either. My thoughts on all of this is to use your test kit of choice and stick with it. Try your best to measure and read the same way every time. This way, even if you are off my 15-30 at least it will be that way every time. Obviously we want to be in the right range, but for all three of these it seems there is quite a wide range of accepted values. It seems that what is more important is that we maintain those numbers consistently. I may be measuring 430ppm for my Ca, which may actually be 460 or 400, but as long as I am measuring 430 every time and my corals are growing then everything is ok.

I am not in anyway discounting the work you guys are doing on this, but I think that test kits are not going to be comparable b/c each has a different process. Hopefully one can find a test kit with less error in the experimental steps as well as more distinct color changes to reduce the error in the human eye.

 

[Ron Reefman]

reefnewbie83177, I don't disagree with you about the variation in results. But on the other hand, if the same person is doing the same tests, using the same hands, eyes and brain, the results should be fairly easy to duplicate (for that person). I've done tests with my Salifert kit where I was sure the calcium should have been a higher number. So I do the test again and end up with the same result. And then do it with an API kit and although it's a wider range of numbers, the Salifert results lie inside the range if the API kit. So it's not like what Peter is doing is impossible or completely unreliable either.

 

[ek9vboi]

I think what is really tough with all of these test kits is that we are relying on the individuals eyes to determine what the color is. As well, measurements are not exactly the same every time. I use the red sea pro for my mg/ca/alk, and with the Ca, I notice that as I add the third titrating reagent, I get purple and then blue, but after a couple of seconds the blue will fade to purple, and then I add a couple more drops and it goes to blue and stays blue. The measurement of the dry powder is not exact either. My thoughts on all of this is to use your test kit of choice and stick with it. Try your best to measure and read the same way every time. This way, even if you are off my 15-30 at least it will be that way every time. Obviously we want to be in the right range, but for all three of these it seems there is quite a wide range of accepted values. It seems that what is more important is that we maintain those numbers consistently. I may be measuring 430ppm for my Ca, which may actually be 460 or 400, but as long as I am measuring 430 every time and my corals are growing then everything is ok.

I am not in anyway discounting the work you guys are doing on this, but I think that test kits are not going to be comparable b/c each has a different process. Hopefully one can find a test kit with less error in the experimental steps as well as more distinct color changes to reduce the error in the human eye.

Very true, although I have yet to use a Hanna tester, looks to me that it might be better for testing as a majority of the guess work with color changing is not there. But then user error is always still there. The more kits I use, the more confused I get. Not sure what to believe but what you said is correct, as long as it is consistent results and at optimal range then it should be fine. After all, stability is more important then chasing numbers.

 

[reefnewbie83177]

reefnewbie83177, I don't disagree with you about the variation in results. But on the other hand, if the same person is doing the same tests, using the same hands, eyes and brain, the results should be fairly easy to duplicate (for that person). I've done tests with my Salifert kit where I was sure the calcium should have been a higher number. So I do the test again and end up with the same result. And then do it with an API kit and although it's a wider range of numbers, the Salifert results lie inside the range if the API kit. So it's not like what Peter is doing is impossible or completely unreliable either.

Hey Ron,
I was never saying that what Peter was doing was impossible or unreliable. I do like that we are seeing which test results give a wider range and looking at which test kit fits most accurately into the range that we would like. All I was saying was that if one finds a test kit that is easy to use, less prone to error, and easily read by the naked eye, then one should stick with that assay.

 

[Ron Reefman]

Hey Ron,
I was never saying that what Peter was doing was impossible or unreliable. I do like that we are seeing which test results give a wider range and looking at which test kit fits most accurately into the range that we would like. All I was saying was that if one finds a test kit that is easy to use, less prone to error, and easily read by the naked eye, then one should stick with that assay.

I didn't mean to read too much into your first post, just that what Peter is trying is worth the effort. I agree with you about easy of use and less prone to error. In fact I use API kits almost as often as I use salifert kits because they are easy (my eyes see the colors the same way every time) and reliable (I find getting a specific number less important whan a basic ballpark). So we are pretty much on the same page. :beer:

 

[jimrawr]

Would love to see how ELOS stacks up, and Hanna

 

[disc1]

I was trying to put together something similar some time ago but ran into the trouble of not being able to get enough kits together to really do anything. I had hoped that lots of local reefers used different kits. Turns out all anybody had was Salifert or API. The one in particular that I wanted to try and didn't get to was the RedSea Pro.

What were the values of the standards used? I'm just curious what this works out to percentage wise? Were the three brands run against only their own standards or were all three kits run against the same standard?

 

[Whitebeam]

What were the values of the standards used?

Values printed on the bottles are:

CS: Ca = 420mg/l, Mg = 1250mg/l
FM: Ca = 422mg/l, Mg = 1314mg/l
Sal: Ca = 412mg/kg / 422mg/l, Mg = 1285mg/kg / 1315mg/l

(The last two look very similar - I wonder if Habib gets his standard from the same place as Claude? ;-)

To convert mg/l to mg/kg, divide by an assumed density of 1.025kg/l (hence 422mg/l = 422/1.025 = 412mg/kg.

Were the three brands run against only their own standards or were all three kits run against the same standard?

If you look at my tables again you will see that all tests were run against all standards.

Peter

 

[Whitebeam]

Would love to see how ELOS stacks up, and Hanna

Elos are on my list to get one day. I'll have to come into some money before I can test the Hanna.

Now, if the UK suppliers of these tests would like to send me one, I'll surely test 'em and report my findings here ))

Peter