• Please review our updated Terms and Rules here

The joys of filesharing ..

M

mbbrutman

Associate Cat Herder
Staff member
Joined
May 3, 2003
Messages
6,432
Before the RIAA slaps a lawsuit on me, let me explain. :)

Getting TCP/IP to run on an older machine has some serious challenges:
  • Existing DOS TCP/IP software is out of date and not supported
  • Memory is tight on a 5150 PC. Some features just can't be supported.
  • Some source code exists, but then you have to be a master and porting/debugging.

My predicament is worse - I'm on a PCjr, which doesn't take ISA cards.

So a few years ago a friend of mine (and fellow PCjr nutcase) found that Xircom parallel-port to Ethernet adapters work fine on the Jr. You need to do a little tweaking, but with that you can run Telnet and FTP from Trumpet, or NCSA Telnet. Not a bad start. Trumpet isn't supported anymore, and there is no source code available. NCSA Telnet has source code available, but it is a giant monster.

Instead of living with what was out there, I decided to start writing my own .. Using Turbo C++ 3.0 on a 386-40 machine, creating executables that still run on the PCjr. The 386-40 has a real Ethernet card, while the PCjr has a Xircom adapter.

So far I've got ARP and UDP working. No TCP yet .. that's a big mess of code. I'm not stealing code from anywhere - this is all 'cleanroom' done by reading textbooks and googling.

To prove the UDP piece was working correctly, I wrote a 'fileserver' type program in Java that uses a simple protocol over UDP. That program can run on Windows or Linux. The client side runs on the old DOS machine. It basically does Xmodem over UDP.

So after a marathon hacking session, I've got it working. It only sends data to the server, and there are probably some bugs in it, but it does seem to send the data correctly.

On the 386-40 with a real Ethernet card it is pushing 170KB per second out to the file server. (The file is in a ram disk to eliminate any overhead that the file read might be causing.) I used a fairly good sized file for the test - 125KB.

The PCjr suprised me. It clocked in at about 16KB per second, which is less than one tenth of what the 386-40 can do. But when you consider that the PCjr is running at 4.77Mhz, doesn't have an L2 cache like the 386-40 does, and has to use the cruddy parallel-port adapter instead of a real bus attached Ethernet adapter, it's amazing that it did that well.

On the ToDo list:

  • Cleanup this nasty pile of code I threw together.
  • Add the ability to receive files as well
  • Enhance the error checking
  • Move to a 'sliding window' type system to improve throughput.

In a few months this will be to the point where if you have a suitable machine that runs Java to use as a server, you can put a Xircom or a real Ethernet card on your machine and have a fairly lightweight and fast file transfer program. Beats shuffling floppies ..

Eventually when I get the TCP part done this will turn into a Telnet BBS. Running on the Jr of course. :)

Guys/Gals - don't just collect the machines. Use them!
 
Hey, Mike,

You're already waaayy over my head with what you're doing. All I can say iz. Keep On Coding, and lemme know when you've perfected it, BTW, I love your ROM-dump program.

--T
 
Sounds like you need some Memory!

Guess it's possible to make a RAMpack with easy plug in & have
the program utilsing this memory.

Don't know how you're goin' to do that though.

CP/M User.
 
The machine has 640K of memory, so I'm pretty good. The major problem for TCP/IP is that a maximum sized packet is around 64K. And of course as packets go through the network they can become fragmented. It is the responsibility of the receiving machine to put everything back together again.

Imagine getting a few fragmented packets and trying to put them back together .. you would run out of memory fairly quickly if they were large packets. I have to look this up to be sure, but I don't think that you can tell at the beginning how big the full IP packet was supposed to be - you have to wait until you receive the last packet, which has an indicator that says it is. Which means that you always have to allocate a worst-case buffer to reassemble in.

The solution is that machines that are tight on memory don't do packet reassembly. You impose a restriction - any IP traffic to the machine has to fit within an Ethernet frame, or bust.

I know you program quite a bit - ever tackle networking code before?
 
"Beats shuffling floppies .."

I realize you're not doing this simply to facilitate network file transfers...and in that case I would simply recommend a $4 null-modem adapter!
You might want to check out the linux-86 mailing list. Activity is low, but there is at least 1 goorue on there - Alan Cox. I've gotten mails with his signature and I was like wtf.
As time allows, and I hope that to be soon, I want to learn something about network programming, but more then likely it'll be in a UNIX environment. I need to catch up on alot of things come to think of it...
 
A $4 null modem adapter isn't going to let the machine transfer at 16KB/sec. Ethernet does ..

As for Alax Cox, he's quite famous in the Linux community. Linux kernel mailing lists have nothing to do with what I'm doing.
 
mbbrutman said:
A $4 null modem adapter isn't going to let the machine transfer at 16KB/sec. Ethernet does ..

As for Alax Cox, he's quite famous in the Linux community. Linux kernel mailing lists have nothing to do with what I'm doing.
Click to expand...
...or token ring. A great feat would be to get token ring and ethernet to talk to each other. That way whenever the PCjr would recieve a packet it would have a much smaller packet to deal with.

http://www.cisco.com/en/US/tech/tk331/tk332/technologies_tech_note09186a00800a6163.shtml

Or you may want to go totally with token, since I know there's not a really good way to make them compatible. The network would be wired up differently, yes, and there's a lot of work involved... so maybe it isn't such a good idea after all.

Sorry, I just really like token ring and always consider it as a network solution when others aren't practical. Something to think about.

Nathan
 
Ethernet packets can be pretty small too. The minimal sized Ethernet frame is about 64 bytes. The Ethernet header is 14 bytes. The minimal sized IP header in that frame is 20 bytes. So the actual payload might be quite small.

Ethernet and Token ring will never talk together. Something has to pluck the IP packet out of the Ethernet and put it in a Token ring packet, and vise versa. These devices were known as routers, bridges, etc.

You really should look at Token ring and Ethernet has equivalent .. It's a different way of getting the data around, but it is packet based and fairly high speed. Quite unlike point-to-point serial connections. If Ethernet is a bad solution for something, chances are that Token ring is equally bad.

On the PCjr there is a fascinating option called 'Cluster' that I've only explored very briefly. Cluster was a Ethernet-like (but not Ethernet) network from IBM in the 1982 or 1983 timeframe. Cluster is natively supported on PCs, XTs, ATs and Jrs. The problem with Cluster is that it was quickly supplanted by 'PC Network', then Token Ring and Ethernet. Cluster adapters for the PCjr are fairly common, but ISA Cluster adapters are very hard to find.

Since Cluster is just a generic packet based transport (like Ethernet and Token ring), you could run TCP/IP over it. Then on another machine you could have Cluster and Ethernet, and do the bridging there. The most practical thing would be to put it on a low end 486 and try to shoehorn Linux onto it, and then write a Linux device driver for the old Cluster adapter. That way you could use all of the Linux networking code nearly transparently.

Another project for another year .. I'm more likely to graft a simple Ethernet chipset directly only the bus of the machine to speed things up.
 
"As for Alax Cox, he's quite famous in the Linux community. Linux kernel mailing lists have nothing to do with what I'm doing"


From: Alan Cox <alan@lxorguk.ukuu.org.uk>
To: editor@lwn.net, scoop@freshmeat.net, kernel@linuxtoday.com,
kernel@linuxfr.org, arthur@tucows.com, chris@linuxdev.net,
kernel@linuxhq.com, staff@appwatch.com, kernel@linsight.com,
sde@linuxlinks.com, dml@tonnikala.net, editors@newsforge.com,
kenneth.g@linux.nu, linux-kernel-announce@vger.kernel.org
Subject: Linux 8086
Date: Mon, 1 Apr 2002 01:30:03 +0100 (BST)


Do you wish you could run Linux on your old IBM/XT or indeed the original IBM
PC itself ?

We are pleased to announce that Linux 8086 has now reached the point where
it is ready for some wider testing. Harry Kalogirou has the TCP/IP stack
functioning...
 
I'm writing TCP/IP for DOS so that I can squeeze it into a telnet BBS and directly control the performance by writing the code myself. They are trying to cram an entire OS in. Other than for the OS having some networking, it's not related.

Also, from http://elks.sourceforge.net/:
08 May 2006: ELKS revisited and back in action!

ELKS has been in virtual hibernation for two or three years now. The CVS access is broken, the website has been left untended, and 0.1.3-pre1 never really got released! After a new maintainer stepped up to the plate, the breath of life has slowly crept into ELKS once again. Downloads are temporarily hosted by Jody Bruchon on his personal equipment due to an unexplained CVS breakage for ELKS anonymous CVS access. As always, any assistance you can offer will be greatly appreciated!
​
There is hope, but I wouldn't call this a viable project. It was been dormant for quite a while.
 
the simple point was there's people on their who are knowledgeable with network coding. And the fact that it is pretty inactive (though not entirely, I've gotten a few dozen posts since subscribing earlier this year) is a very good reason to perhaps sign up. They could use the traffic, and I'm sure there's people who'd appreciate an exchange.
 
Maybe I misunderstand .. I'm interested in Elks as an alternative OS to DOS, but joining the mailing list of a dormant project trying to shrink Linux onto an 8088 class machine just to tap their networking expertise doesn't seem like the right thing to do. They are having a hard enough time keeping the project alive; tangential networking questions are not what they need, even if it does generate some traffic.

Especially since I'm not having any trouble with the networking part. ;-)

Alan Cox is interested in the Linux kernel. Although he posted that announcement years ago, he is not on the ELKS development team, and I wouldn't think of hassling him unless it were on topic.
 
Some food for thought ..

Xmodem was good in the day, but it wasted a lot of potential bandwidth. For those who don't know or remember xmodem, each time the computer sends a 128 byte data packet to the other machine, it has to wait for the other machine to say 'I got it!' before sending the next packet. Given the round trip time of a 2400 bps modem and the slow machines, there was a lot of dead time on the line.

TCP/IP has a similar problem. If you waited for a separate ack packet for each packet of data sent, you wouldn't make very good use of your dialup, leased line, or ethernet. So TCP/IP implements two bandwidth friendly devices - a sliding window and piggybacking. The sliding window trick allows a machine to send data without waiting for an immediate ack back, up to a certain point. Piggybacking makes better use of full duplex connections.

So I was thinking about my Xmodem over UDP implementation here, and it doesn't use a sliding window or piggybacking. I thought about the sliding window, but then decided against it. Seems like a good idea, so why not do it?

Well, here is the problem. My target machines are old DOS PCs ranging from 4.77Mhz and a floppy drive to maybe 486s. The server machine would be running Java on a more modern machine. If the old DOS machine sends a packet to the faster machine, chances are that the faster machine has processed the packet and sent an ACK even before the old DOS machine can get around to looking for the ACK. Or put another way, the server machines that people would probably use (anything capable of running a small Java program) are so fast that it just doesn't matter.

The sliding window really only matters if you've got a huge round trip latency, and a lot of that latency was caused by the other machine being busy doing other things. But if you throw a 4.77Mhz PCjr against a 1Ghz machine, the round trip latency from the perspective of the PCjr is non-existent. Chances are that the server is going to go into power suspend and sleep for a bit before the PCjr gets ready to send the next packet. ;-)

I might add the sliding window just to prove the point .. any thoughts on this? Does it make sense, or am I smoking something bad?
 
As usual, you're way over my head, and whatever you're smoking must be some good sh!t...(I can barely get my mind around Modem7 & checksumz...fagiddabout X-, Y-, Z-Modem, & CRCs, etc).

--T
 
Last edited:
Many years ago I wrote code to do communications through parallel and serial ports, but never Ethernet cards.
However the basic principles of comms are the same.

The design has to allow for the fact that the CPU may be occupied when a data packet arrives.

If your code is directly talking to the Ethernet card, there maybe a hardware buffer on the card that will hold the ACK packet for collection.
If there's no hardware buffer on the card, you may need to configure the card to generate an interrupt on packet arrival, and write an interrupt processor that buffers received packets.

If your code is talking to a packet driver (or some other low-level code), then a receive buffer may already be in place.
 
mbbrutman said:
I might add the sliding window just to prove the point .. any thoughts on this? Does it make sense, or am I smoking something bad?
Click to expand...

Based just on your discussion it seems like it would be a waste of time - unless you were trying to prove the point. :)
 
I was always a serial port programmer myself. The serial port makes things pretty simple - things arrive in order, even if they do get corrupted.

UDP presents some special challenges because it is only 'best effort':
  • Packets can be lost without a trace
  • Duplicate packets can arrive after the originals

The client machine (DOS) uses a packet driver for the network card. The user software controls the buffering. Right now I've allocates space for 10 buffers, but never have more than 3 in use at a time even on my slowest machines. The card itself is buffered and interrupt driven, so the buffer management code has to be interrupt safe.

My thinking is based on two things:
  • the DOS machine is single tasking, and is doing nothing but this file sending task. It's never going to be busy doing other things for a long period of time.
  • a modern machine that somebody would use as a server might be doing something, but in general will be far more responsive than the DOS machine.

So far it seems to be true. The only real latency I have is the transmission time on the Ethernet itself, so doing a small sliding window might improve that. The real way to figure out if it is worth doing is to write a test .. I should try to time one packet going over and getting a reply back from the server, with nothing else happening to get the 'bare wire' speed.
 
Erik said:
Based just on your discussion it seems like it would be a waste of time - unless you were trying to prove the point. :)
Click to expand...

Sometimes you think more when you talk at other people .. :) I'm going to cobble up a quick test to see just how fast I can do a round trip on the Ethernet from the old machine to the new machine, and then decide if that latency is worth hiding.

The 386-40 running this code is only pushing around 150KB/sec. I used data in a RAM disk to cut the diskette drive out of the loop, and that is still pretty slow. (ISA bus speed is much higher, and 10 Mb/s Ethernet should be around 1200KB/sec.) The latency of the round trip might not be an issue for a 4.77Mhz machine, but on something like the 386-40 it might be accumulating.
 
I used to install/configure the hubs/switches/routers in a building, and so I know a little about network protocols.

I would rarely choose to use UDP. If I was sending outside temperature info to a remote display, UDP is a good choice. When packets get dropped/repeated/unordered, I doubt that anyone looking at the display would notice. The criticality is just not there.

For everything else, choose TCP so that it handles the dropped/repeated/unordered packets.

But I can see that as a programmer, you're giving yourself the challenge of coding your own TCP-like protocol.

------------------------------------------------------------------------------------------------
Some stuff you may already know:

The similarities with serial port programming are there. No matter what layer 1 you use (serial [232/422 etc.], Ethernet, Token Ring, RF, etc.) flow control and adequate buffering is needed to cater for communicating nodes that have differing data processing speeds.

And even on serial networks (eg. X.25), packets sometimes disappeared on the network (eg. caused by a router with bad RAM). And so no matter what layer 1 you use, you can't assume that the packet sent by your hardware will see guaranteed delivery.
 
The goal of the project is to write a reasonable TCP/IP for small PCs. I normally wouldn't use UDP for anything mission critical, but before I write the TCP part of the stack I want to test what I have. So far I have a basic ARP implementation, IP header, and UDP. I need ICMP, IP header options, and routing before I start TCP.

I'm doing an Xmodem-like protocol over UDP to stress the existing code and give me confidence that I'm building on a good foundation. It also gives me a chance to analyze the performance of the existing code, and makes it useful to me immediately. (I hate shuffling floppies.)

TCP is the goal, but TCP is complicated due to the sliding window and piggybacking. I can't swallow the elephant except in small bytes. :) And in a lot of ways, what I'm doing with this xmodem protocol will directly apply to TCP - the TCP code has to handshake, detect errors and retry packets just like this Xmodem-like code I'm doing. So I look at this as a practice run for the big target ..
 
You must log in or register to reply here.
Back
Top