What skimmer are you running and whats your turnover rate? Getting to the barebottom stage may take awhile due to disturbing the bed. however, after you are running BB, you need high flow to keep all the "poop" in suspension and a good skimmer to pull it all out. However, I went through the same things, removed a 4" bed in my 120 to go BB, and I haven't regretted it yet.
tom