Thank you for your excellent explanation. I also assume that performance wise the system is almost in spec but just slightly below the margin. Therefore I wanted to start from changing the OPAMPs since their leakage will contribute a lot to the final precision of the voltage generation at the output. I've looked at the possible replacement of MC14051, but apart from TI CD4051 in the plastic package I don't see anything else. So, for the moment, I'll try to check the passives for the leakage and then replace TL084. Interestingly enough the service manual states that the JFET input stage opamp must be LF374, however in the real o-scope I have TI TL084.... So I've ordered both.
For the moment I'm very reluctant to change the DAC (which is PMI marked) to the Analog Devices part. But IMHO if the calibation signal is really 0V to whatever requested V, and everything else is in spec I can only assume that the improvised mutlichannel DAC and the ADC are out of spec.
Thank you again,
P..S. I can potentially completely reverse engineer the FW, but I have no time and motivation for this yet since skipping the LIMIT error woudn't make the instrument more precise.
For someone, who is interested in reverse engineering, now on e-bay it's possible to buy 16500A/B/C logic analyzer with 16550a card (or even 16510a/b) for next to nothing. 6802 inverse assembler can be downloaded directly from YAHOO groups HP/Agilent forum. IDA disassebler has the 6800 instruciton set extension so the routine in question can be easily identified by LA and disassembled. But this requires a significant amount of time and knowledge about the TEK hybrids to understand the routine in the detail. IMHO one of the possibilities would be to try finding/discovering some detailed diagnostics to be displayed on the screen if the LIMIT error is encountered (likely this is hidden somewhere by Tek but not documented). Same goes for some additional features as the calibration variables mark-up and manipulation. Also easy backup via the interface. For example, one could develop an I2C EEPROM manipulation routines to store and recall the calibration data while the SRAM is under repairs.