I think everyone pretty much does this, and yes it may cause a little trauma, but I doubt it would be enought to hurt anything.
Main reason that our lights usually end up on timers is because you are simulating day and night for them, but how often in real life is day and night exact.