Yes if you have a system with more memory, it makes everything so much easier. As of Slackware 12 or 13, the busybox system will have to uncompress to ram, and it sounds like just to boot up, you'll need at least 32mb. That's why the install instructions say you need to activate and turn on the swap file to get anything done, which as chuck says, maybe you need to manually do this before launching setup, using the swapon command. As far as why the oom happens during install? Well it probably has to do with either the size of the package that is being installed, or a package post-install script calling something that requires more memory you have.
A couple weeks back I was running a vm using maybe 12gb out of 16gb and then kicked off many multiple parallel simulations. Then I had this thought, oh, let me close out my vm since I am running something else now. Well at that moment I felt a sudden sluggishness, and then I couldn't get my mouse to move to and then click the X. Then I realized I never activated swap, and this is a condition where you are close to out of memory on linux, that makes everything slow. The oom killer in linux is very conservative, meaning it tries not to kill your processes until the last possible moment. I knew my system was not completely locked, since I could ping it from the outside, but the normal keyboard and mouse actions did not do anything. Then finally I forced an oom kill using sys req, which killed my vm of course... I decided to not do a swap partition on this system, opting to enabling a swap file only when needed. It's not the best choice for linux, as it really wants a swap file, even if most user stuff do not need that much memory. But if you do ever run programs that push towards the limit, you're gonna regret how slow linux can be, and you'd be better off with swap as a failsafe. Maybe someday I'll correct this.
So basically, if during installing packages, the process seems excruciatingly slow, I suspect your swap is not active. It may also help to disable some package sets if you are only using this at the command line. I'm not sure what the minimum ram+swap size really intended to be by this version.