Agreed. The only fellow reefers I know personally who have tried multiple damsels in the same tank (and of the same species) have ended up with the fish thinning the herd through fighting. This may not be an issue in your case, as both of these people had 7 or so in the tank.
Also, keep in mind that if you want to add a fish later (let's say a damsel or two dies, and you decide to fill the vacancy with something else) you will be limited in your choice by how much the damsel(s) are likely to bully them.