Doc's Brewing supplies in Binghamton is a great source for beer, wine and cider supplies. Doc is a great guy, very friendly and knowledgable, and he can set you up with everything you need to get started. Brewing itself is really pretty easy... but I guess it takes experience and some luck to get the end product to be really good.
Beer: As far as beer goes, I got started with a Beer Machine Kit I got for my B-day. That said, I wouldn't suggest commercial kits. The best way is to use what is basicly a 5 gal bucket, or glass container, with an airlock mechanism. (lets gas out, but keeps bad bacterial from getting in.).
As far as beer mix, I don't have a ton of experience, but you can:
a. go with a commercial premix (beer machine or Mr. beer online)- you just mix w/ yeast & water, ferment, bottle. These may be good for a first attempt, but they have been hit or miss (whether it was the flavor I chose, or my lack of brewing skills)
b. you can also get all the individaul ingredients and mix your own recipes.
As for bottling, I bought a case of 1L grolsch style bottles, which have the reusable corking mechanisms. (I didnt want to bother with caps, and the capper). You can even use plastic bottles with screw on caps, but I though that was just cheesy.
Cider: It may be the easiest, or at least turns out the best for me. Pretty much the same general steps as beer, but there are a million ways to do it. Sweet/dry/carbonated/flat... you can even put a jug on the back porch and let it ferment from natural yeast.
I myself make a dry, champgne-style cider. I get fresh juice from the cider mill (pastuerized or raw), add a small amount of sulfite to kill wild yeast, add pure sugar, apple pectin, and some enzyme stuff along with champagne yeast. Let it ferment until the bubbling stops. Bottle, add small amount of sugar to charge(carbonate) while inside the bottle... and wait. Usually the whole process is done and ready to drink in about a month.
This is an overview, missing some details, but there are many methods and tons of info out there online. If you're serious, I really suggest you stop talk in and talk to Doc. He knows his stuff, and seems very reasonably priced.