It probably is absorbing it.
If you are feeding your fish anything or coral feed of any kind you are adding phosphate back in.
Also your PO4 test results will vary depending on when you last fed your tank.
Feed your tank and then do your PO4 test, it will be higher.