the example of text editors was chucked out there, and, well, it does kind of blow my mind when I download what *seems* on the surface should be a pretty simple piece of software but it's a 100+ MB download, but, well, there are reasons for it and some of them are kind of unavoidable these days. (Just as a for-instance, UNIX was *hella* smaller back in the 1970's, sure, but by today's standards it was also massively insecure and it was trivial to break it with bad inputs/buffer overflows/etc. Just adding the layers of security we need for anything that might even remotely touch the internet is a massive cost multiplier.)
Visual Studio Code and Atom are worst offenders for me.
Programmer's text editors are in range of couple of megabytes, IDE's are in range of tens, RADs in hundreds.
Here come Electron "apps" realizing something in between programmer's text editor and IDE, with a size and memory usage of RAD.
But there are always options. Many alternatives to these monstrous waste of resources.
Drifting a bit off topic, but I'm running Debian Linux. On an older generation system with CentOS and an earlier version of Mixbus using JACK I was under 20ms but that was using an Maudio Delta 1010LT. The newer machine, even with it being close to ten times faster than the old Precision M4300, isn't low enough latency for good overdubbing, so I pull out the M4300 (full PCI slot in the dock) with the 1010LT when doing punch in/out. Using a Focusrite Scarlett 2+2 on the other one. I have several plugins, including an excellent Fairchild 670 emulation.
What sample rate and frame count?
I could run 44.1 with 512 frames @ 16ms in mid 2000s with Pentium 4 and EMu Systems PCI card on Windows 2000 with ASIO.
The current computer runs Scarlett Solo 3rd gen at 48kHz with 696 frames @ 14.5ms with ASIO.
There's a chance the chipset/USB is slowing things down in your case. Have you tried a pci express USB card?