I've used Visible dust swabs, (both orange and green), but now I generally use 'Sensor Swabs' from Eclipse. It has nothing to do with whether one is better than the other, it's just that my local camera store started carrying the Eclipse swap so I don't have to pay shipping anymore.
The good news is that what you're cleaning isn't actually the sensor. It's a glass filter (the hot mirror) that sits on top of the sensor. It's pretty tough but it is possible to scratch it.
What likely happened is that your brush might have picked up some grease, still left over from assembly. You'll want to clean the brush thoroughly before you use it again. I'd use something like Dawn and hot water to break up the grease and then give it a final rinse in alcohol.
If your smear is grease it may take a few swipes to get rid of it. Visible dust has a solution called "smear away" that is especially for grease. It will, however, leave residue of it's own so you have to follow it with their normal cleaner.