If it's any consolation, I have the same issue. Data burst audible (and visible on the waterfall) on VHF air band down to 20m at least. Not evident on 2m but I have a high noise level there. Usually I'm on 40 or 30 so I only noticed it during the recent Es on 10 and 6. Disconnecting yellow and green LEDs made no difference for me either.
My QLG-1 (internal patch aerial in a plastic box) plugs into a hub (in an ali case), which provides 9v and 5v from an external 12v supply, and connects 4 5-pin DIN sockets in parallel (Gnd, 5v, 1PPS, Data, 9v), so GPS into one, Progrock, Clock and one spare for something else.
I also have a separate clock and Progrock in a portable case with 6v SLAB, into which I can also plug the QLG-1, so I've been able to move this unit around the shack. No external power leads, just the lead between the GPS and the main box. The interference is reduced as I move away from the operating position, but is still audible when 6 to 8 feet away.
For now, I unplug the GPS from the hub if I want to use an affected band, which isn't that often. The clock time is ok, but if I forget and leave the GPS unplugged overnight, the date doesn't change and I end up putting the wrong date in my log!
Maybe I'll do a bit more investigation when I have some time and I'll be interested in what others may have to suggest.