It is most certainly difficult to tell until they have become adults, and especially to be sure of one being female as sometimes one that is thought to be a female ends up being a very late developing male.
Some are able to tell by using a strong magnifying glass to determine placement of the anal fin.
For the females, the anus and genital opening will be above the anal fin.
For the males, the anus will be above the anal fin and the pouch slit will be below.