The hart of this design is the hetrodyning up and down to provide the flexibility and selectivity. That is why calibration is sooooo important. There are two fixed points that need to be "hit" to make it work at all:
1 - the 45 MHz roofing filter
2 - the 11 or 12 Mhz (depending on version) "SSB" filter.
This can only be done with calibration of the 3 clock signals. The analogy that I think of is that 2 points define a line, and all three of the clocks need to be on that line. Of course it is more like a snaking curve with the up and down signal manipulation.
I am glad you found your issue. I would still recommend that you look at the SDR spectrum analyzer software for future use. It is like using a finishing file where a rasp is needed for some applications but better than no tool at all. It is also free with what you already have. The best bet may be to ask Santa for 100 MHz, or higher, oscilloscope for Christmas.