The linear overflow calculator has nothing to do with flow. It is designed to determine the minimum amount of linear overflow length to achieve good surface skimming. Teeth reduce the amount of "linear overflow" at least the sum of the width all the solid teeth in your overflow. In other words if you have an overflow that has a perimiter of 10 inches, and you cut 1/4" teeth 1/4" apart, then your total "linear overflow" is 5 inches. Or look at it this way. If the water over a FLAT 10" overflow was 1/4" deep, then the overlow with 1/4" teeth 1/4" apart would have a flow depth of 1/2".
In other words, if you can get away without using teeth, then by all means do so.
Hope this makes sense.