I know this will probably seem naive, but where is the bacteria coming from? Chlorinated water is relatively sterile going in, why would it carry bacteria with it?
The water going in is generally pretty clean, but it isn't sterile. You can find info about bacteria in your water from your utility's water report, though most only provide info on the percentage of samples that tested positive for coliforms (which is all they're required to report).
Neither the filters themselves or the housings are sterile though, and they, rather than the water, are one of the main sources of bacteria. When you change your filters next time, you'll notice that the insides of the housings are a bit slimy. That's a bacterial film.
If that were true then wouldn't we would all be getting sick from our refrigerator filters and any other tap water filters we use around the house?
No. First, most bacteria aren't going to make you sick, so even if you were drinking bacteria-laden water, you would be fine in most cases, and probably wouldn't even notice.
Second, an RO unit isn't really comparable to a faucet-mount or fridge filter. Faucet mounted and fridge filters remove the chlorine from the water right before it goes into your glass and you drink it. There's no time for any bacteria present to multiply in the water after it's been dechlorinated. On the other hand, everything between the carbon block and RO membrane in your RO unit is dechlorinated, but not sterile. When you aren't running the unit, the bacteria are sitting in a stagnant, nutrient-rich brine with nothing to keep them from multiplying. The Brita pitchers offer the best comparison since you have dechlorinated water sitting around for a while, but they're usually washed semi-regularly and kept near freezing in the fridge, which inhibits bacterial growth. As I learned in college, if you don't wash them for a while, they will indeed grow nasty things, even in the fridge. Even then though, you aren't passing the water through a filter that concentrates any waterborne bacteria right before you drink it as is the case when you're drinking RO wastewater.
While the risk of getting sick from drinking the brine is pretty small, a lot of people
do get sick from drinking from poorly maintained RO and carbon filters around the house. A former member here, AZDesertrat, who worked in water treatment, has lots of horror stories about being called to people's houses to do bacterial testing because people were getting sick, only to find out that their tap water was just fine, but the water from their tap filter or RO unit that they were drinking or making ice with was off the charts because the filter wasn't properly maintained and sterilized.
If you read the literature that comes with RO units and replacement filters, almost every supplier recommends regular sterilization of the unit and provides instructions on how to do it, for precisely that reason.
Use the brine to rinse the dishes, water the plants, flush the toilets, do the laundry, mop the floor, etc. but I would avoid drinking it, especially if you haven't recently sterilized your RO unit. Either drink the chlorinated water or the output from the RO side.