The generally accepted limit for SSB intermodulation products is a minimum of 24 dB between the lowest of the twin tones and the highest of either the third order (products immediately adjacent to the twin tones) or fifth order (products next over above and below the third order) products. My uBitx shows -12 dB at 3.6 MHz. Conditions are: 30 mVrms audio input and RV1 set for 5 watts RF output through a 4 MHz LPF. My board has the onboard PA filters removed and strapped and has the additional 45 MHz filter with 12:1 output impedance transformer in place of R27.
In order to better understand where the IMD is being introduced in the radio I started all the way back at the balanced modulator output and measured IMD at thoughtfully provided test points up through the driver output. In general, once IMD products are introduced at early stages they tend to only get worse as the signal progresses down the chain. The key to fixing it is finding the root cause as early in the process as possible.
TP17 is the output of the balanced modulator and the 12 MHz SSB filter. IMD products here were below the noise floor of my measurement configuration and I did not chase them down into the mud because they are at least 40 dB down and do not pop up through the floor.
TP16 is the output of the first bi-directional amplifier and IMD at this point measured -35 dB, already much too high and and indication of non-linearity in the amplifier that must be addressed.
TP14 is the output of the onboard 45 MHz filter following the 2nd mixer. This actually shows a slight improvement but probably within measurement error at -37 dB. This measurement pretty much exonerates the 2nd mixer as a significant contributor to the IMD issue.
TP16 is the output of the second bi-directional amp and again there is a serious deterioration in IMD with the amp adding 11 dB to the problem.
TP1 is the output of the 2nd transmit mixer (labelled 1st mixer in the text) and is the first time we see a signal at air frequency of 3.6 MHz. The mixer added 5 dB of IMD to the total.... much too much and probably indicative of low injection levels as has been stated in the past. On the other hand it is not the primary culprit by far.
TP3 is the output of the first pre-driver and of RV1 and it adds a little over 1 dB of IMD. The total IMD at this point is 5 dB less than the acceptable amount and it is only beyond this point that we are able to control power levels with RV1 which would normally be the adjustment point for controlling PA IMD. In other words we are starting out with an unacceptable signal from the low level stages and only now getting to where IMD is normally introduced. A contemporary radio would show normally show IMD levels at -45 dB or better at this point.
From this point forward, at the five watt level, the combination of pre driver, driver, and PA added 5 dB of IMD. This amount would be perfectly acceptable in most radios starting out with clean drive and would allow the total power to be increased by RV1 adjustment to significantly higher levels.
The IMD problem is rooted first in non-linearities in both bi-directional amps and then in both mixers.