OS/2 had a 387 emulator built in. If no FPU was available, the invalid instruction would be caught, and emulated. I had a customer that wrote some code using the FPU (time sensitive). It ran perfectly on a 486. On a 486SX, it took way too long. It had to be re-written to not use the FPU.
I've recently been experimenting with a couple 386DX's (were DX's first and SX's later as a cost savings?? I can't really recall).
I found that StarTrek TNG Final Unity, NFS 1 and MechWarrior II won't run on it - they all really do use 486+ instructions, and the system barfs a big opcode-not-found exception. But I wonder if a TSR could be written to trap that and emulate the missing instructions? Maybe the FPU was more forgiving on that kind of capability, since it was understood many end users wouldn't have that math coprocessor. But years later when they added the MMX instruction set, people didn't include MMX-enabled and non-MMX EXE's (was that due to compiler-support magic? i.e. any "use MMX" option proxied in appropriate jump vectors for MMX and non-MMX hosts?) Anyway, just feels that if MS-DOS can trap opcode-not-found, then emulating the instruction and "future-proofing" the old 386 seems possible?
I did get Quake to run on the 386DX (after adding a clock matched 387), but it is indeed 2-3 fps. I actually didn't try Doom, it might not be all that bad if it could get 8-10fps

But there is the 8088DOOM that runs just on 8086 and text-mode (plus an MDA version). There is a text-color version also now, which of course runs just fine on the 386. // I'm not sure yet if I'd call Falcon3 playable on the 386 - it sort of is, but then I got into some glitchy camera angles (but I've read the "realistic" flight model that uses the 387 was never really that great). Tyrian is playable (but slows when enabling audio), and OG Warcraft 1 (but both require 4MB RAM). I still find MOD MASTER as the best audio player IMO (and does support more than just .mod), I tried about 8 others that used graphics modes and just didn't work out even on the 386DX.
I haven't branched out too much on "exotic operating systems" for the 386, other than I did run DONKEY.BAS from MS-BASIC 1.0 to show it does still work on a 386 (and is throttled appropriately to be playable!). As 386's don't have built in BASIC ROM anymore, I used BASIC.EXE maybe from DOS 3.2 or 3.3 (or BASICA? something like that).
I was hoping to see how OS/2 WARP goes, on a 386 (I used it heavily on 486DX/100 and Pentiums for years, but never verified if a 386 could actually handle WARP in any satisfactory way) - but I'm really not into stacks of 3.5 floppies, and I can't find a 386 mainboard with firmware that supports CD-ROM booting (took awhile to find an IBM release of WARP on a CD-ROM, but I do have that now and it installs great on later model ThinkPad Pentiums).. I did finally get a CD-ROM working with the 386 (one 386 does so via the SoundCard IDE, the other 386 just uses a CD-backpack across LPT). Don't suppose anyone already has an OS/2 IMG file that we could DiskWrite to a CF card and boot?
NOTE: Speaking of that 8088DOOM that's text mode... I had hoped I could run MOD MASTER under DesqView, and then 8088DOOM in another window on a 386. But so far I never got that to really work out. But that'd be interesting to see IMO (I forget which DesqView version I was using - but maybe MS DOS 4 Concurrent could handle it?). It would be wild if OS/2 2 had enough DOS compatibility to pull it off. Note that MOD MASTER also supports dropping to a DOS shell, I can't recall if I had enough memory left to run 8088DOOM (been a few months since I tried it).