There are errors in the diagram and the levels are really different than shown.
Looked at your diagram and its assumptions are wrong. If the final mixer can accept -1dbm without
producing spurs then why start with -16dbm at the very first (balanced modulator)?
The input to the balanced modulator would be around -10dbm(audio) an that bumps up the levels 6db.
The problem with the pad at the output is that and the filter (about 8db total) reduce the carrier to signal
ratio by that amount (not allowing for bad layout causing blow by). Excess gain makes it worse.
Having input to the final mixer of more than -10dbm causes great amounts of distortion and spurs.
So at that point you are showing about 9db of excess gain. Also the assumption is the mixers have
enough drive. When I tested the 5351 was set to 1,1,1 or 4ma (factory code) and at 3,3,3 or 8ma I
did get the stated 13dbm but exceeded to the total current for three outputs if I understand the spec.
That is only part of that story, for the moment.
So we are both under driving one and over driving mixers and hoping for a good result.
The output of the last mixer should be around -17dbm (+-2db) for a 7dbm input, about
7db loss and a -10 dbm input.
the two TIA amps the 12mhz nearly hits the 20db mark (actual on 2 were 19) and at
45mhz not even close maybe 14db mine measured 13db. Good thing too or the
over drive would be worse.
I The filter is before Q90! that means Q90 is part of the power amp chain. Than means
there is a gain block before the pre-driver and driver missing in the drawing.
Also the design gain of the amp is 3 stages (Q90 through q97) of 16++ db per stage and
1 stage of about 13db my match says that is 61db and most seem to do that at 3.5 and
maybe 7mhz. After that the gain falls by easily 11db to 10M. So the diagram does not
match the actual.
FYI the second bidirectional amp is not shown (post 45mhz fitler and before last mixer)
The line up should be 45mhz fitler, 45mhz bidirectional amp, mixer, 33mhz low pass fitler, then Q90 tx preamp.
Being PDF I cannot edit the drawing it does have errors.