Keep in mind a RTL-SDR is not an insensitive receiver. Its good enough on its own.
so putting the gain and mixers of the ubitx ahead of that means the noise floor is
increased by the effective gain or about 10-14db or so. None of which helps hear
signals better as the IF amp it self has a noise figure of maybe 4-6DB.
So yes, much cruft.
To use it effectively as a panadaptor the SDR gain has to be turned down so the noise is
the baseline at or near the bottom of the spectrum display or a light background on the
waterfall. Otherwise all you see is the noise and the signals will not stand out.
You likely need a buffer (or reduce loading) as well as tapping at that point is like putting
two load (resistors) in parallel and the receiver has less signal or effective gain. The buffer
could be as simple as a series resistor from that point to the coax for the SDR. Since the
SDR already sees too much cruft so lowering the signal (series resistor) will improve the
view and also lower the load on the IF to regain signal. Values likely are in the range
of 50 to 500 ohms and try by substitution may be easiest way. Its a useful experiment
and so try a 470 ohm for a starting point (may be too high).