Not claiming to be an expert but I would say that pattern is due to response in the far field.  As long as the 'matching' or other radiating mechanisms don't  have significant radiation compared to the 'antenna' then that response won't change.

Having said that, the *apparent* pattern can change. If mismatch gets high enough that other mechanisms, not intended to be included in what one calls the 'antenna', become significant compared to the desired/intended one then the pattern and performance can indeed change be different. 

This second situation is essentially what happens with common mode ingress. That which one *thinks* is the antenna is not really the total of what is providing response at the receiver.  Once intended antenna (mismatched) signals get down to the level of other signals, such as CM then the effective pattern is no longer entirely due to the intended antenna.

This problem can be aggravated by electrically small antennas which though they intercept the same aperture and, were they matched, would deliver the same signal power, deliver very small voltage or current levels due to their preamplifiers because of the low radiation resistance and high reactance and imperfect matching.

Perhaps surprisingly, were we able to perfectly match a [1 cm] dipole at 160m it would have just about the same performance as a full half wave. Of course we can't do that with materials we have, probably not even with super conductors so in practice this doesn't occur. 

But the general idea of using "something" as an antenna, no matter whether the feedpoint impedance is near something easy to access with reasonable matching materials, is perfectly fine as long as one compares the resulting mismatch to the target at a particular frequency of interest.  That target starts to get difficult at the high end of HF where we want to get increasingly close (or below) KTB. It's for this reason that either a loop, such as described by LZ1AQ, or my dipoles both fail to achieve the ITU 'quiet rural' noise floor at 10m.  More or less, it takes larger structures to get something that has a chance of 'not too bad' match over broader frequency range with a finite number of real components.