I have a D50, which is the little bro of the D80. For what it is worth, I think you have made a GREAT choice with the D80. And that 18-200mm zoom lens will be a GREAT general purpose lens.
I mainly use an 18-70 lens and a 70-300 lens. I am constantly switching them on and off the camera and would love a nice lens someday that covers wide to telephoto. I have also discovered recently that I need a super-wide lens for good landscape and architectural shots, but that's another story.
As for picture format, do NOT use RAW unless you actually know how to post-process them. In short, don't use raw pics. The pics are harder to deal with after the fact and take up MUCH more space on both your memory card and your hard drive.
I would recommend for "picture quality" setting, get to know how to change easily. For my D50, there is a "Qual" button on that back which, combined with the wheel, allows me to scroll between size (Small, Medium, Large resolution) and compression level (Fine, Medium, Low).
As I am taking pictures, I generally try to stick with medium size & quality for general shots. If I think that I am likely to want to print something BIG that I take, then I will hop it up to large size and either medium or fine quality. If I KNOW that I will only EVER view a picture on my laptop or print it small, then I will jump down to small size and medium quality.
I take enough pictures that if I do not manage my "picture size" like this, then I will end up with WAY to full of a hard drive. It also allows me to take more pics on a memory card.
As for MODE, stick with full auto for now. It will work surprisingly well for your general purpose needs. The ONLY thing that will specifically improve your reef shots is to learn how to set custom white balancing. It basically involves putting the camera in a white balance configuration mode and then taking a picture of a neutral gray card in the lighting environment that you will be photographing. That can very much help with resulting colors in reef pics.
Lastly, buy the close-up filter set. It will be FAR worth it for the inexpensive price (especially compared to what you have already spent on the camera and lens) and will allow for a MUCH closer minimum focal distance.
Just my two cents (or 5 cents) or so....