It could be an epitoke, a specialized reproductive structure of some polychaete worms. When they're ready to reproduce, these worms basically split in half, and the epitokes, loaded with eggs and sperm, swim toward the surface. Near the surface, the epitokes release their loads of eggs and sperm, then die. The atokes (the asexual halves of the original worms) continue living in the substrate. Weird, huh?
Unless you know that this particular species of worm has been doing something bad, like eating zoas or attacking fishes, I wouldn't worry about the swimmers.