As mentioned many times upthread, the idea of the tank or AFV has been around for centuries before the technology caught up to make one. The reason is the need to cross fire swept ground and deliver shock or firepower is a long standing military problem, and in the past was "solved" with such innovations as chariots, armoured knights and even highly trained Infantry soldiers trained to run forward at all costs (Chasseurs à pied). The mechanized solution simply did the job better than anything else up to that time, and, so far, no other technical soution has been able to carry out the essential military task in an equally effective or economical manner. (uneconomical ideas like the 100 ton Maus have been tried, but the T-34 is probably the apex of tank evolution, being both effective and very economical).
So perhaps the best way to look at this is to stand the question on its head, and ask "what essential military tasks need to be performed?", and then see how they can be done.
WRT the massive costs of "improved" vehicles and the institutional inertia, part of this is systematic, since bureaucracies essentially exist to preserve themselves. The fierce resistence to adopting tanks in many armies was more a case of protecting existing rice bowls, and now the Armoured (or Armored for our American cousins) Corps is a big rice bowl all of its own, with a shiny cult object in the center. Part of it is the actual military problem it was ment to solve still exists, so we still need tanks, but now have to figure out how to make them survive a very much more hostile environment. The last part of the problem is that most people really cannot think outside the box, so gravitate towards refining existing solutions rather than look for other ways to attack the problem. (There is no shame to this, the vast majority of out of the box solutions are not going to be effective or economical or both; an "in the box" improvement is much more likely to be accepted and the developer showered with praise and rewards. Consider the people who came up with the idea of the MMEV and MGS as tank replacements in the CF; are they looked at today as visionaries or idiots?).
As a brief aside, hovercraft can be used for vehicular purposes. There have been various ideas combining a hovercraft with a trailer or self propelled vehicle, which allows for wheeled traction and control while lowering ground pressure. You can tow or drive heavy loads over soft ground like muskeg using this system. As a fighting machine, it would suck because the instant the blowers stopped working the thing would sink into the muck, but this might be acceptable as some sort of specialist logistics or engineering platform to operate in that sort of condition. Or you could use a BV-206...