Any standard video card is going to need at least some upper memory space. (IE, the B-page.) But EGA/VGA is, yes, very expensive if upper memory is what you're after; a whopping 160K from A0000-C7FFF, non-negotiable. Your best strategy with the Lo-tech card and VGA is give it the D and E pages, assuming you can keep your other BIOS ROMs (hard disk, etc) to what's left over in C8000-CFFFF.
In theory at least the most RAM you could possibly have in 5150 with standard (not-breaking compatibility) parts would be to install a CGA card (only needs 32K from B8000-BFFFF) and hack the lo-tech card/build your own to give you 96K from A0000-B7FFF, fill the C-E pages with 192K more, and I *believe* it may also be possible to place a (maximum 16K) option ROM at F0000 in the 5150/60 so you can even have XTIDE in this build. (I *think* the onboard ROM sockets only decode the top 48K, not the entire F page, and that the option ROM scanner is willing to look for signatures up to F4000, but I'm not entirely 100% solid on that claim. I run XTIDE at F0000 in my haxorred Tandy 1000HX so I know its scanner counts that high.)
If you do have memory in the A0000 (contiguous with the lower 640K) page it's actually possible with a little trickery to use that directly as conventional RAM in old versions of DOS that don't understand the whole upper memory thing. But, again, this is off the table with VGA.