I understood Fabian is already measuring output power according to your description above with his homemade Elecraft DL1 dummy load clone. I googled the schematics:
Together with the Elecraft supplied Voltage vs Power chart this looks fine for me.
My first thought also was that it could be a power supply problem. Maybe a test with separated supplies for PA and the rest could be performed using the bench supply and the motorcycle battery. Or making use of the oscilloscope, monitoring power supply when keying in CW mode.
73 de Armin, DJ2AG