• Please review our updated Terms and Rules here

TCP/IP Update

M

mbbrutman

Associate Cat Herder
Staff member
Joined
May 3, 2003
Messages
6,432
I'm still working on my TCP/IP for small PC clones. It's been a while since I posted an update, so here is what it looks like.

The code supports the following:

  • Connecting to another machine ("active" connect)
  • "Listen" support for accepting incoming connections ("passive" connect)
  • Multiple open socket connections
  • MTU size setting for different network types/topologies
  • User defined receive buffer .. used for getting max performance
  • Correct TCP 'window size' advertising; helps with flow control
  • A good tracing and logging facility


Still to do:

  • Better TCP 'reset connection' support
  • I need to revamp the UDP support - it's been ignored for months
  • DNS requests over UDP
  • Performance work - I need to put in a high performance memcpy routine.


The code runs on any IBM clone with a packet driver. It has tested with a generic 386-40 with an NE2000 card, an IBM PCjr with a WD8003 card, an IBM XT with a 3Com 3C503 and an IBM XT with an Intel EtherExpress 8/16.

Even though I'm adding features and required support, performance is still very good. The last round of changes show the 386-40 receiving a file from another machine at 451KB/sec, which includes the overhead of buffering and writing to disk. The same code on an XT with the original 10MB hard drive runs at about 25KB/sec, and the PCjr is in the same range. After the faster memcpy code goes in it will be far faster.


Right now there are two apps:

  • NetCat: a version of the Linux utility. Lets you send and receive data that you pipe in/redirect from stdin and stdout. I've been using this for my file transfer tests.
  • A simple 'echo server' which just sends back whatever you send it. This handles up to 5 incoming connections simulataneously and it is used to test the listen support.

Both apps are under 50KB in size, and the total memory requirement is well under 128K including the packet driver.

I've been working on this for almost a year now. A year ago I couldn't get the packet driver to send a byte on the Ethernet. This has been the most difficult home programming project that I have ever attempted.

The ultimate goal is a telnet BBS running on some vintage hardware (PCjr?) that will use this. The code is stable enough were I could start thinking about BBS features, but I really want to polish the TCP/IP code first to make sure that it doesn't cause me hard-to-debug problems later.
 
that's awesome! i'm glad to hear it's coming along. i'd love to use it for my programs once you get DNS resolution working. i can just communicate with it via interrupt calls like wattcp and tcpdrv, right?

if you give me the interface specs i would love to write quickbasic code to access it.

on a similar note, do you have documentation for the DNS resolution protocol? i can't find any decent information on it. i've been trying to do so for a long time since there isn't anything built into either wattcp or tcpdrv, afaik. i know there isnt for sure in tcpdrv.

EDIT: i meant to ask what kind of hard drive you're using in the 386/40 when you test file transfer and include disk write overhead. it makes all the difference in the world. with speeds like that, i assume it's an IDE right? is it a newer one or and older one?
 
Last edited:
It's a C library that you link to. So your QuickBASIC code is not going to be able to use it in it's current form. I can give it a software interrupt interface like WatTCP and Trumpet, but that won't be soon. That would also require some significant changes. (See below)

To use it you are going to have to work in a language that will let you link against the library. Turbo C++ 3.0 is guaranteed good ... There may be others, like a Borland PASCAL of the same vintage that will be able to link the library in and use it.

The problem with a TSR interface like WatTCP and Trumpet is that they can't use heap storage - everything is dynamically allocated. Performance in a lot of cases is a function of buffer space, and if you are confined into small buffer spaces you potentially lose a lot of performance. The NCSA Telnet code (which has an excellent FTP server) uses the same approach that I do. WatTCP and Trumpet could have let the user allocate the space for them, but that complicates things.

My DNS reference is in text books. I'm using Douglas Comer's "Internetworking with TCP/IP", which is good but in some parts dated. (It is a 15 year old book already.) For the most up to date reference I am using Charles Kozierok's "TCP/IP Guide", which is very new, 1500 pages, and $80. :)

I though that TCPDRV and NTCPDRV could resolve addresses - you have to tell it the DNS server address.

The hard disk on the 386-40 is a 1.2GB Western Digital on a Promise 16 bit IDE controller. Nothing fancy. Beats the living daylights out of the XT, but it is nothing special for a 386. If I skip the hard disk writes I was getting over 600KB/sec, but that number will be lower now because of the additional memcpy overhead that I've added. (It was unavoidable if I wanted the code to be usable by humans.)
 
Btw, I'm looking for other C programmers. If you want to pick up a good language that is still in widespread use and is the core of most current operating systems, C is the way to go.

C or C++ on an old DOS machine is a little more challenging than C using a modern Linux distribution. Pointers are only 16 bits, and you have wrapping problems because of the segmentation. Most of the common library calls are there, so they are not a problem. For some of the stranger stuff you wind up using DOS or BIOS interrupts, which the compiler makes pretty easy to do.

My development environment is Turbo C++ 3.0 on a 386-40. The Watcom toolchain is supposed to be pretty good too. The compilers generally require a 386 because of their memory usage, but they generate code that runs on the 8088.

I'm willing to talk with Pascal bigots as well. It's close enough .. :)

You are doing great stuff with QB, but based on what I've seen in your code you are being inhibited by the language and you may not realize it. The way that you use QB would be very natural in C.
 
Mike C,

If it's just a matter of not having the program, I've got a copy of Turbo C that I'll never use. It's buried in storage right now, but if ya want to try it, I'll dig it out for ya, for postage cost. I think it's version 3.0, but I ain't positive.

--T
 
Yes, we all have our biases. I wasn't trying to hide mine very well. :)

To me Pascal is like programming in a strait-jacket. Borland's extensions for the PC are tolerable because they allow you to get at things that straight Pascal doesn't allow for.

On the other hand, C is like playing with razor blades. Great for expert barbers, not recommended for small children. I've been programming in C for 16 years, 14 years as a professional. After getting used to reasonable C compilers on OS/2 and Linux, going back to the DOS environment with it's 16 bit pointers and 16 bit integers was a hard re-adjustment. And trying to interface to the packet driver (like I did for this project) required dropping into assembler, which was even ickier.

If you program under Windows or Linux you generally don't have to use assembler or worry about interrupt handling. Going back to DOS programming and doing this level of code has been a brain bender at times.

(I've been doing DOS apps for years. Just nothing that required such a high level of technical expertise. It can be much worse.)
 
Terry Yager said:
Mike C,

If it's just a matter of not having the program, I've got a copy of Turbo C that I'll never use. It's buried in storage right now, but if ya want to try it, I'll dig it out for ya, for postage cost. I think it's version 3.0, but I ain't positive.

--T
Click to expand...

thanks for the offer terry! i'll see if i can find a copy some other way (wink wink) but if i can't i will PM you. :)
 
mbbrutman said:
You are doing great stuff with QB, but based on what I've seen in your code you are being inhibited by the language and you may not realize it. The way that you use QB would be very natural in C.
Click to expand...

thanks. yeah i know the language is a bit lacking. it's mostly the lack of speed that'll drive you batty if you're using an older computer. there are ways to get around some of its other quirks... like reading var pointer locations to use strings as a sort of "byte array", etc...

but yeah. i think i'm going to get my hands on turbo C++ and start working with it. hopefully i can pick it up fairly quickly.

btw, i can't wait for your DOS telnet BBS server! i'm going to be running that :)

will it support multi-threading and door games?
 
Let's take the language discussion offline .. I kind of wanted to keep this thread for periodic status updates for the code. I've offered Mike a copy of Turbo C++ before, so if he can't get it easily there is a copy waiting for him.

Will it support multi-threading and door games?
Click to expand...

Now there is an interesting question. What exactly does multi-threading mean on a DOS machine? It's not like DOS supports threads. ;-)

But to answer the question, the BBS will allow for multiple users to be logged in at the same time, so yes, it is multi-threaded. To do something like this you have a data structure for each active connection/user that tells you what the state of that connection is. At a low level the TCP/IP socket data structure maintains the state of the connection, while a different data structure will maintain the state of the user. ie: What menu are they in, what message might they be reading, how long have they been idle, etc.

The main loop will have to poll the connections and see which ones need servicing. When a user hits a key on their telnet client far far away, that keystroke gets sent to the machine and buffered. If the main loop detects a non-empty buffer that means the user is doing something. Depending on what is going on, you might just have to copy the keystrokes to a string or you might have to act on the keystroke and do something important (like finally post the msg they've been composing for 20 minutes.)

If the TCP/IP code is good it will be able to keep receiving and buffering keystrokes even while the 4.77Mhz PCjr is busy doing something else, like accessing the disk to retreive a msg.


Door games? Given infinite time, sure. ;-)

The bigger question is what should a BBS provide. I'm thinking just a basic message board. File downloading doesn't make a lot of sense because nobody is going to have a client that supports that. (How many telnet clients do you know of that support a file transfer inline? None) Old terminal emulators let you do this, but no modern telnet client lets you. File downloads are probably better done using something standard like HTTP or FTP.


I wrote BBS software 20 years ago that ran for about 2 years in NYC. Back then of course it was single user and dialup. The icky part about writing BBS software is the message editing software. Unless you make ANSI codes a requirement, full screen editing is not possible. Most people won't deal with line editing, and even with ANSI codes full screen editing is a pain to do.

I don't want to make ANSI codes required .. that would hurt the feelings of our C64 friends. :)
 
A 2K custom character set should be able to emulate ANSI graphics quite well on the C64. I'm quite sure it has been done multiple times over the years as well.
 
I suspect that there are not too many C64s running TCP/IP, either over SLIP or Ethernet. There are a few though: http://www.dunkels.com/adam/tfe/

I think that straight ASCII (those chars that map to PETSCII) with 40 columns is best for our Commie friends.
 
Latest performance numbers. These are sends and receives of a large file, using real disk I/O. ie: Instead of just throwing the packets away to measure the speed of my code, I'm measuring actual file transfer speed:

386DX-40 with IDE hard disk, NE2000 card, 3.1MB file

Send: 417KB/sec (14KB disk read buffer)
Receive: 472KB/sec (14KB disk write buffer)

IBM PC XT with original 10MB hard disk, 3COM 3C503, 1.3MB file

Send: 26.5KB/sec (14KB disk read buffer)
Receive: 24.7KB/sec (16KB disk read buffer)


Performance is fairly good with read and write buffers as small as 4KB on the XT, and 10KB on the 386-40.

Just for comparsion, I used NCSA Telnet on the XT and measured it's FTP performance. NCSA can receive at 16KB/sec, and send at 21KB/sec. The NCSA code is fairly good, so I'm pretty happy to have beaten it by such a wide margin.

Next up .. fix my TCP/IP reset handling code, add some code to cleanup half completed connections, and then I'll make this code (netcat) available for download. After that, it's BBS time.
 
mbbrutman said:
Let's take the language discussion offline .. I kind of wanted to keep this thread for periodic status updates for the code. I've offered Mike a copy of Turbo C++ before, so if he can't get it easily there is a copy waiting for him.



Now there is an interesting question. What exactly does multi-threading mean on a DOS machine? It's not like DOS supports threads. ;-)

But to answer the question, the BBS will allow for multiple users to be logged in at the same time, so yes, it is multi-threaded. To do something like this you have a data structure for each active connection/user that tells you what the state of that connection is. At a low level the TCP/IP socket data structure maintains the state of the connection, while a different data structure will maintain the state of the user. ie: What menu are they in, what message might they be reading, how long have they been idle, etc.

The main loop will have to poll the connections and see which ones need servicing. When a user hits a key on their telnet client far far away, that keystroke gets sent to the machine and buffered. If the main loop detects a non-empty buffer that means the user is doing something. Depending on what is going on, you might just have to copy the keystrokes to a string or you might have to act on the keystroke and do something important (like finally post the msg they've been composing for 20 minutes.)

If the TCP/IP code is good it will be able to keep receiving and buffering keystrokes even while the 4.77Mhz PCjr is busy doing something else, like accessing the disk to retreive a msg.


Door games? Given infinite time, sure. ;-)

The bigger question is what should a BBS provide. I'm thinking just a basic message board. File downloading doesn't make a lot of sense because nobody is going to have a client that supports that. (How many telnet clients do you know of that support a file transfer inline? None) Old terminal emulators let you do this, but no modern telnet client lets you. File downloads are probably better done using something standard like HTTP or FTP.


I wrote BBS software 20 years ago that ran for about 2 years in NYC. Back then of course it was single user and dialup. The icky part about writing BBS software is the message editing software. Unless you make ANSI codes a requirement, full screen editing is not possible. Most people won't deal with line editing, and even with ANSI codes full screen editing is a pain to do.

I don't want to make ANSI codes required .. that would hurt the feelings of our C64 friends. :)
Click to expand...

yeah, years ago when i was about 13 or 14 i wrote a multinode BBS proggie in qbasic. it actually WORKED, too :)

i received about 3 to 5 calls a day on average. didn't support doors, though. i eventually had to stop adding features after qbasic decided the .BAS was too big to load. (i think 32 KB is the limit?)

i wish i knew about quickbasic 4.5 back then :(

and btw, hyperterminal supports inline file xfer :)
 
How good are you guys with programming TCP/IP on windows?

In the next few months I will be trying to create a WinAMP plugin that instead of passing the MP3 file to be played will transmit it over a TCP or UDP/IP socket to the Altair. The Altair will have a STA013 to decode the MP3. All we need is 16k/s for a 128kbit MP3.

Optionally it would be nice to transmit auxillary data to the Altair like the ID3 tag, as well as ask the Altair the sense switch positions to control the WinAMP program.

This would be cool. :)
 
I don't do Windows. :)

And that's is actually more of a WinAMP question, isn't it? Windows programming is bad enough, but then you need to interface to WinAMP.

How are you going to connect the Altair and what software are you going to run on it? UDP is pretty easy on a small machine, but a real TCP/IP socket is going to be very tight and somewhat slow. (TCP likes 32 bit ints.)
 
Probably UDP/IP with a custom sequence and acknowledge packet structure. Since its intended to be used on a local network and switches are so cheap the Altair shouldn't have any trouble. MAC addresses can be filtered without downloading the packet from the ethernet chip. Just peek at the packet and throw it away if we don't need it. Then its just an exercise of transfering bytes from the ethernet buffer to the MP3 decoder buffer.
 
You must log in or register to reply here.
Back
Top