Measuring man's impact on the Earth is what makes us think we have an impact.
That means a whole lot to the world's oceans as nearly as we can tell. That 3% is on top of a system that was already roughly in equilibrium with the other 97%- i.e. if it weren't for the oceans capacity to take up about half of our 3% addition we would be out of balance by 3%. That means about 1.5% of the CO2 produced every year now accumulates in the atmosphere while about half of what we produce every year goes into the oceans, shifting the carbonate buffer system towards bicarbonate. Last time similar chemical changes happened, it meant the end of all reefbuilding for several million years, the extinction of one of the main groups of reef-builders, and the near-extinction of corals (they essentially disappear from the fossil record at that time). That meant a whole lot to the ocean's ecosystems.
We also regulate alkalinity in our tanks. The ocean doesn't except on geologic timescales.
I think you seriously misunderstand how we know about the impacts we're having. For one, we're not just spotting trends over a few years and assuming we're the cause.
We'll stick to CO2 for example. How do we know it's causing a problem and that we're to blame? For one, we know that the ocean is acidifying from the top down, which immediately rules out undersea sources like volcanoes and tells us the input is from the top of the ocean. Then we can measure the isotopic signature of that carbon in the upper layers of the ocean. The carbon from burning fossil fuels is chemically different from carbon sources like volcanoes and decaying plants. When you measure that isotopic ratio in the oceans, it shows that the main contributor of additional carbon is the burning of fossil fuels. It's not coming from the decay of plants or from volcanoes, and since we're the only major source of fossil fuel combustion, we can trace that carbon in the oceans back to humans based on its fingerprint.
We've also worked backwards and measured the increase in CO2 in the air and calculated how much CO2 will dissolve in seawater and the effects that will have on the carbonate chemistry based on the concentration in the air. These are a very straightforward set of equations that you can solve without even using a calculator if you have to (I had to do it when I took marine chem). These same equations have served oceanographers well for close to a century now, so we know they work. The answers from those equations tells us how things should change in the ocean if increasing atmospheric CO2 is the source, and those predictions match the real-world measurements.
We know that those changes are a problem because we can measure their effects in microcosm- highly controlled closed systems. We can for example, pump carefully controlled amounts of CO2 into the air above a sealed aquarium full of coral and see what happens to them. Except in a few experiments where alkalinity is artificially kept constant, their calcification slows and eventually stops. To rule out the possibility that there's some mechanism that we can't reproduce in an aquarium that would counteract that effect we also look at prehistoric reefs to see what happened to them when the ocean's chemistry changed. We can measure things like the ratio of magnesium and calcium in their skeletons and the form of CaCO3 (calcite or aragonite) they produced and determine the chemistry of the water they grew in. What we see are several periods where there is high pH and aragonite is favored. When atmospheric CO2 increases and the pH drops, we see species that produce aragonite disappear, sometimes being replaced by calcitic species. Other times, things get bad enough that aragonitic and calcitic species both largely disappear, which tells us that the abundance and makeup of reefs is closely tied to seawater chemistry just like in our microcosms.