I only read the last two pages so sorry if I ask a question or two that may already be addressed.
First of all, it looks like you had a good run for 1.5 years so that rules out any obvious oversight. It looks like it was just your time with a buildup of organics and older substrate (nutrient sink).
Is the "hair algae" bryopsis or derbasia? Bryopsis is the feather-like one that grows in clumps with a single stem-like base. Derbesia is the more common one with finer filaments that attach directly to surfaces rather than clumps, looking more like hair and less feathery. You treat them both the same, with the exception that bryopsis is easier to physically remove. Derbesia is a little more tenacious as it has a strong holdfast.
Raising your magnesium (slowly) to 1800 ppm with Kent Tech M (magnesium sulphate & magnesium chloride) can be a miracle cure. Other magnesium supplements with magnesium sulphate alone aren't as effective. The idea is it gives the algae too much of a good thing, interfering with chlorophyll production. Apparently a woman from SeaChem gave an eloquent talk at MACNA about how this does not work, but I've seen it work several times and you can read more about it here.
http://reefcentral.com/forums/showthread.php?s=&threadid=1113109&perpage=25&pagenumber=1
Raising your alk level to 11 is another good practice, but it looks like you are there with the new test kits. Just keep buying new test kits until you get the answers you want
The added benefit of a higher alk/Kh is it fosters the growth of coraline algae which "should" outcompete nuisance algae in a "healthy tank".
Removing the substrate and manually removing as much algae as possible gives you a level playing field. Nuisance algae can literally feed itself and live independent of nutrients in the water. Your test kit measures free phosphates (inorganic phosphate) only. There is a substantial amount of organic phosphate bound in the algae and calcarious (calcium-based) surfaces in the tank. Simply "starving the algae to death" will not happen. Shutting the lights off for a few weeks will, but it doesn't work when you have corals. Some recommend shutting the lights off for an hour in the middle of the day to interfere with the photosynthetic compensation point, but I don't think it will help much. Lowering the photo period to 5 or 6 hours is a good compromise.
A diatom filter is a good tool for removing algae spores and free-floating filaments while you are manually cleaning the tank. Algae reproduces and releases nutrients when it is trimmed/torn. I use Ket Pro-Clear (coagulant) to bind the free-floating algae and make it easier to collect or simply choke out. Don't overdose it as it can deplete the dissolved oxygen rate. Many aquarists overlook the value of a mechanical filter. Protein skimmers don't pick up the slack, so your substrate becomes the default mechanical "sediment" filter. Filter socks/bags are also ineffective with exception to short usage while cleaning the tank. Mechanical filtration can be as simple as a sponge in the overflow box or as elaborate as a 25 micron canister filter. OceanClear has a unit with an Aqua UV brand 18 watt UV sterilizer built into it.
Have you checked your RO/DI source water? Sometimes a faulty cartridge or membrane can cause you grief. What is the effluent TDS reading?
Another common cause is residual Co2 from the calcium reactor. Switch to chemical dosing for a few months if possible. This is easier than troubleshooting the reactor in some cases. Co2 is the ultimate algae catalyst and you may be adding it without knowing. Make sure the reactor has a large second pass media chamber. You can even use a 5 gallon bucket of aragonite to be sure you have neutralized all of the carbonic acid (Co2). Dosing calcium is always a good thing because calcium bonds with free phosphate to form calcium phosphate as it falls out of solution.
Carbon dosing may be a good long term move, but I think it will only feed the algae at this point in time. Photosynthesis is carried out through carbon fixing. That carbon can come from your calcium reactor or a vodka bottle. Ascorbic acid may be a safe way of introducing a carbon source
http://reefcentral.com/forums/showthread.php?t=1252294. Alternatively, SeaChem's calcium gluconate provides calcium and a carbon source (sugar). Once again, this may feed the fire but it kills two birds with one bottle.
Some people swear by sea hares, but I haven't had success with them eating derbasia or bryopsis. The algae eating fish and inverts like tangs, blennies and urchins and snails will not keep up with the demand... or is it supply
They certainly don't hurt though. Bryopsis is more toxic than derbasia and subsequently less palatable.
Did you look at your bulbs as a source of the problem? Older bulbs can have a colour shift that encourages nuisance algae. Other than changing the bulbs, raising the lights 6-12" may ease your pain.
It's great that you are using carbon frequently. Just make sure that that isn't a source of phosphate or nitrate for that matter. Some carbon is activated with phosphoric acid, while other animal bone sourced carbon naturally contains phosphate and nitrate. You can test RO/DI water for Po4 & No3 then add a few granules of carbon to each vial. This isn't the most accurate testing method, but it may show higher readings. The easy way of assurance is to buy premium brands. It's just like buying a $2700 designer hand bag, you know the zipper won't stick
You should fine tune your refugium to outcompete with the nuisance algae. Install a 6500 K MHL bulb over chaetomorpha and gracileria. Caulerpa may even be a good idea as it grows the fastest. As long as you run a steady reverse photoperiod of 16 hrs per day, it won't reproduce sexually and crash. A prefilter keeps it from reaching the display tank. Make sure your refugium algae bed is shallow so there are no dark/shaded spots below the algae mass where it can slowly die-off and release its nutrient catch.
Wet skimming and adding ozone to your skimmer is another approach. You can think of it as a passive water change. This method is particularly useful during and after your daily nuisance algae harvest.
If the algae is concentrated on a few rocks in particular, try to ascertain why. Is this an area where detritus or food settles? Is there more or less flow there? As an aside, I agree with the more flow the better idea, but technically adding flow to an algae problem just delivers more nutrients and Co2 to the algae. Have you blasted your rock out with a strong powerhead? You may have billows of detritus trapped in there. The more algae that grows and subsequently dies, the more detritus will accumulate, and so goes the cycle. Flip the rocks upside-down if you cannot remove them for any reason. Taking them out and scrubbing & rinsing in freshwater helps a lot. I use a big denture brush or a vegetable scrubber. Power washing also works.
Are you thoroughly rinsing frozen foods with RO/DI water? Are you adding garlic (high in phosphates BTW)?
If the tank is FUBAR you should move the corals to a quarantine/hospital tank and completely overhaul the display. Bleaching the rock to sterilize it then spraying it with a weak acid (even vinegar) to remove bound phosphate will give you a new start.
Maybe I don't need to make the Dr. House call
