Great thread! It's like a mystery, that the author and readers are all trying to solve. RC at it's best! It's great to see so many folks trying to help.
I just read through the whole thread, and I have a few thoughts to add to the discussion.
In my experience, it's best to focus on the basics, when fighting algae. Food availability, predation, and competition for food are the biggees. Why haven't I included light? Because light is a constant. You can't turn it off. Though a strategic multi-day blackout can be used as a knock-out blow, once you have the algae 'on the ropes'.
Food availability is a key factor. I'm assuming you are not overfeeding your tank, but you may need to take a look at that. Since we can all agree that algae can feed itself, we need look no further for it's food source. It's the algae itself! Where do think is the most concentrated food source for algae? The water column? The rock? It's the algae itself! This saves us the hassle and mystery of figuring out the food source. You are what you eat-especially if you're algae. So, how do we address this? Manual removal! I'm not talking half measures either. I mean serious, relentless, obsessive-compulsive level removal. Write down a manual removal schedule and stick to it. It sounds like you are already having some success with manual removal. That's good!
Next up is predation. I've seen no mention of a clean up crew on this thread. I think they are vastly underused in the hobby, mainly because a lot of folks just don't know enough about them to use them effectively. Tossing a few snails and crabs in doesn't cut it. In fact, don't bother with hermit crabs, unless you plan to isolate them with no food inputs at all. Once they get a taste of fish food, they become pathetic algae consumers. Plus they kill snails, which are vastly superior algae eaters. So what you need are several, different, reproducing snails, like Ceriths, and others. There are several snails available that reproduce in our tanks. Another predator that gets little attention is PODS. These little guys multiply and eat a lot of algae. Get thousands of them. These were actually instrumental in my winning the war with dinoflagellates. Finally, there are several fish that would love to help out. Scopas Tangs, algae blennies, and even mollies are just a few possibilities.
Competition for food is another effective tool. I think the OP mentioned he had an algae scrubber, but hadn't added it yet. Add it now. Whether it's an algae scrubber, a planted refugium, a chaeto reactor, or just planting some macro algae right in the tank, any of these will provide competition for food. Some algae is gonna happen. By picking an algae export method, YOU control how much and where it is.
Sorry I rambled on. I hope some of this is helpful. Good luck! You're getting there!