It seems to me like you're using the nozzle / venturi (same thing) to adjust the skimmer, not the wedge pipe. Your wedge pipe being almost closed is what is causing the skimmer to overflow. Assuming the skimmer is not sitting too deep in the sump you should be able to run that nozzle 4 turns open without it overflowing, and then adjusting the internal operating height of the skimmer with the wedge pipe.
My suggestion would be as follows:
1. Open the nozzle to 3 turns open and leave it, do not touch it again.
2. Position the skimmer in 7" of sump depth.
3. Adjust the internal operating height of the skimmer by the wedge pipe, dialing it in that way.
This should get you to where you need to be. The vibrating you're noticing is from the nozzle / venturi being too far turned in. Anything that is from closed to 1 turn out will most likely cause the pump to vibrate, as it's just too restrictive at that point.