Gutbucket, please don't overestimate how much I understand about the physics of microphones--I'm just treading water like many of the rest of us here. But I do look at things much the same way as you seem to, including what you say about cardioids. There's nothing special about that pattern other than that it's midway between (and a superposition of) omni and figure-8. In some ways it offers the worst of both, and I never think of it as deserving to be the default choice by any means. Often just a small step to the left or right is preferable (I seem to think of omni on the left and figure-8 on the right; that's arbitrary, but it also seems to be the way that pattern switches or knobs are generally arranged).
Also, we should keep in mind that the "compromise" aspects of conventional microphones are deeply embedded in our recording practice. This includes their pattern deviations as well as non-linear distortions, which are quite considerable in many popular microphone types. The most highly revered "vintage" studio mikes, for example, generally have both types of problem--and the ways in which they're used have grown up around their characteristics as a whole, including those problems. Of course the particulars of those characteristics are important, too--but in general they tend to make the microphones suitable only for certain specific applications and usage patterns. For example most of us, I think, are way better off due to the fact that an omni of normal size that's relatively flat on axis (at working distance) will have rolled-off treble response off-axis, because of the absorption characteristics typical of building materials in most places where music is performed.
I'm all for making improvements, including ones that may seem small; they add up over time as more and more such improvements are made. But if you know your existing microphones well, and have struggled for years and found ways of working with and/or against their characteristics, then if Athena descended from Mt. Olympus and handed you a pair of perfect microphones from Her own private locker, your next few recordings would almost certainly be worse rather than better.
Also, sometimes the truth hurts. Most rooms suck! When you travel, sometimes you set foot in a place and say to yourself (or, obnoxiously, to your travel companions who aren't recording engineers), "Damn, I wish I could record in here." But that doesn't happen very often. Collectively what this hobby is about, and its professional counterpart to an alarming extent as well, is making the best we can out of very non-ideal recording situations--sometimes very VERY non-ideal--and fooling the ears of our listeners without getting caught at it too often.
--best regards