Notice that these peaks are attenuated as you approach 30 MHz. That is indicative of the loss in the system at those frequencies. The lower the loss of the transformer the higher and more uniform would be these peaks.
While the transformer may contribute along with coax losses bare wire testing also shows that. By that I mean the wire
itself shows a progressing reduction in the peaks. This is due to radiation, even though the mismatch may be there some
amount of RF is always radiated and the returned energy is not there as a result. An infinitely (or very long) long wire at
some high frequency exhibits a very low return loss with increasing frequency. However in the real world this can be
seen with a 20M wire at VHF.
I can detail why that happens but resistance of the wire is not constant due to skin effect and very long wires at very
high frequencies are very low Q as a result.