Call for ideas/fixes for the next mTCP version

[mbbrutman]

Thanks. I know that manually renaming the files should work, but having to change imported source code is always a hassle. The 'unzip' utility also has the "-LL" argument to force lower-casing of all file names when extracting (this is also the most convenient solution for mass-renaming).

If you could consider this at some point in the future, it would be appreciated. Downstream projects often include a copy of mTCP, making them needlessly harder to cross-compile unless the maintainer does the work himself.

I saw the -LL option as a cheat that somebody else posted and I thought it was cute, and it is easier to do that the multiple passes of renaming. After that only the makefiles needs to be fixed.

At some point I'll do it, but I'd rather do that in a special update where there are no other changes. (Kind of a like a source code formatting only change.) It makes the review so much easier.

 

[Krille]

The Packet Driver interface is not designed to handle multiple concurrent users, but nothing prevents you from running two mTCP applications attached to different packet drivers simultaneously. You could write a Packet Driver 'shim' to provide multiple entry points (interrupts), but you'd need to figure out how to assign incoming packets to them. The easy solution is to set the hardware into promiscuous mode, assign a separate MAC address to each interrupt, then filter by incoming MAC address. Essentially, each application would become its own system on the network. (Note: This is not compatible with wireless networks.)

The packet driver interface supports multiple applications so long as they use different protocols. An application can register itself as a receiver of a certain EtherType or, as in the case of mTCP, register itself to receive all packets regardless of type. This is, in my opinion, actually a bug in the mTCP applications - they attempt to register themselves as a receiver of all EtherTypes, and if that fails (because some other protocol is already registered), then they won't even try to register themselves for only the protocols they actually need (e.g. IP and ARP).

Of course, this is rarely a problem in practice because in DOS you can only have one foreground application so the other networking software has to be a TSR. Also, these days most people (applications) only use IP and ARP anyway so it's kind of a moot point.

 

[mbbrutman]

What you are calling a bug was a conscious design decision at the time. It simplified the code and that's what I was looking for.

The only problem with it that I see is that I did not foresee something like EtherDFS which would use a reserved ethertype. Somehow people are using that at the same time as mTCP so I suspect many packet drivers are not honoring the restriction anyway.

 

[Krille]

Another possible problem might be that older packet drivers (written to conform to specification versions prior to 1.07) will not work with mTCP at all. I'm not sure how common that problem is though.

 

[mbbrutman]

In the 15 years I have been widely distributing mTCP programs I have not had a report of a packet driver that it did not work with, until NetDrive last year which the 3Com 3C905 (PCI) driver hated. I worked around that by experimenting and changing the way I hook interrupts.

When that kind of thing happens I debug it and make changes. So far I've never had a report of a packet driver not working with one of the standard mTCP programs.

 

[Krille]

OK, so here's a couple of bugs.

Under DOS 2.11 DHCP.EXE writes a bogus timestamp to the MTCP configuration file which makes the rest of the applications complain about the DHCP lease being expired. A workaround is to use a static IP but that might not be an option for some of us die-hard DOS 2.x users.

NETDRIVE.SYS; If the server responds with an error to a write-with-verify request then the driver will ignore the error and proceed to verify the data in the response. I'm not actually sure if this is a bug or by design but it seems a bit strange.

EDIT: Also, what is the purpose of fakeStat() in NETDRIVE.EXE? It doesn't appear to be needed.

 

[mbbrutman]

OK, so here's a couple of bugs.

Under DOS 2.11 DHCP.EXE writes a bogus timestamp to the MTCP configuration file which makes the rest of the applications complain about the DHCP lease being expired. A workaround is to use a static IP but that might not be an option for some of us die-hard DOS 2.x users.

NETDRIVE.SYS; If the server responds with an error to a write-with-verify request then the driver will ignore the error and proceed to verify the data in the response. I'm not actually sure if this is a bug or by design but it seems a bit strange.

EDIT: Also, what is the purpose of fakeStat() in NETDRIVE.EXE? It doesn't appear to be needed.

There is nothing strange or different about the code under DOS 2.1 vs. any other version of DOS. To actually be able to debug this I need to know the sequence of what you did.

• What was the system time set to before DHCP ran?
• What was recorded in the DHCP file?
• Did you run SNTP to set the system time after running DHCP or set the time some other way?

It's a known problem that if you boot the system and the time is not set, running DHCP and then setting the system time with SNTP makes the DHCP lease look ancient. The workaround for that (if you did that) is to run DHCP a second time after running SNTP.

 

[Krille]

Nevermind. I just realized it's probably a machine specific clock driver that doesn't work under QEMU. I'm using "Microsoft MS-DOS 2.11 [Leading Edge OEM] (5.25-360k).7z" downloaded from... somewhere that I don't recall. The time is always reported as 11:11:11.11 and the date changes completely with just seconds between running date. So, apologies for wasting your time.

Though, what about the write-with-verify thing in NETDRIVE.SYS? Is that intentional?

 

[mbbrutman]

Good catch, that's a bug but not a serious one - it doesn't affect correctness or break anything. I will fix it because source code should not be confusing.

It would be more efficient to check the return code first and not even bother comparing the contents of the received packet, but honestly, unless DOS asks for a sector that doesn't exist, the image is read-only, or there is a server error you are never going to get a bad return code. If you do then it will cost some extra time but the end result will be the same - the write is not done.

I also fixed the Compaq DOS 3.31 "fixme" recently .. FAT16B will work correctly in that version with the next code.

 

[Krille]

Good catch, that's a bug but not a serious one - it doesn't affect correctness or break anything. I will fix it because source code should not be confusing.

Well, if it was hard to write it should be hard to read, right? :D

It would be more efficient to check the return code first and not even bother comparing the contents of the received packet, but honestly, unless DOS asks for a sector that doesn't exist, the image is read-only, or there is a server error you are never going to get a bad return code. If you do then it will cost some extra time but the end result will be the same - the write is not done.

I haven't checked the server source code but I assume the server doesn't send data back for verification if there's an error. If it does then that would be a problem with this bug.

I also fixed the Compaq DOS 3.31 "fixme" recently .. FAT16B will work correctly in that version with the next code.

I was looking into that myself (I've been following the vogons thread) and was looking in RBIL and found this;

    Note:    to reliably determine which variant of the request block for
          functions 04h,08h,09h has been passed to the driver, check
          the length field as well as the word at offset 14h.  If the
          length is 1Eh and 14h=FFFFh, use the DWORD at 1Ah as the
          starting sector number; if the length is 18h, use the DWORD
          at 14h; otherwise, use the WORD at 14h.

It seems that the request header length is 22 (16h) bytes in both DOS 2.x and DOS 4.x (search for DRDWRHL) so now I'm left wondering which DOS versions uses header lengths of 18h and 1Eh bytes. (I'm trying to find the request header lengths for all DOS versions so I can figure out the most efficient way to do this).

 

[Krille]

I will fix it because source code should not be confusing.

I found something else that makes me confused;

       ; This is a successful read.  Copy the payload, but only copy what we
       ; requested.  If the number of received bytes doesn't match the expected
       ; bytes then we mark this as bad.

       mov ax, [bx].pkt_udp_len
       xchg al, ah                  ; Never forget, network byte order for UDP fields.
       sub ax, Udp_Nd_Len
       jz receiveExit

You subtract the UDP and NetDrive header lengths to get the length of the payload but then just exit without marking it as bad if the payload size is zero. It seems to me that the "jz receiveExit" at the end should not be there as the expected size should be the amount of requested sectors in bytes (and you check for that just below this code).

 

[mbbrutman]

It's not a bug. I think that case covers a zero length read. It's not supposed to be possible, but I coded defensively. I wasn't consistent everywhere with it, but as the code matures that will improve.

This thread is for gathering ideas for the next mTCP and making the occasional release announcement. Please send me a bug report if you really have a bug.

 

[Egzoset]

Hello,

I've been playing with the mTCP applications from FreeDOS v1.4 just recently and noticed some need to transfer parameters via DOS environment variables when giving MSKermit a try, because that's what my custom-made .INI file is expecting. It's not many, only 4 which are MyIP, Gateway, Nameserver and Netmask - which is from memory and could require a correction but that's easy to deal with from a batch-file anyway. I do have a means to get those IP numbers from DHCP using a fine tiny tool called IPRead, i think, yet i figure that might improve mTCP's conveniency, or perhaps it's there already and escaped my attention so far. Another thing which made me wonder turned out to be about DHCP.EXE exactly, because when i tried the Novell NetWare 16 bits LSL/DOSODI drivers plus packet shim it consistently refused to work for me, complaining in those terms: « Could not initialize TCP/IP stack ». It wasn't a big issue on one other vintage machine but i found it unconvenient while trying to make comparisons in terms of FTP transfer speeds and memory consumption. Lucky me the packet driver that is no DOSODI creature happened to perform way much better, yet it could have been the other way around and i find only those 2 scenarios satisfactory. Another strange behaviour was an imposition of the vector, sometimes 0x60 but 0x69 most often than not. Nothing critical, i could still connect the laptop, an Acer Aspire One D250 with either the ATODI.COM or L1CPkt.EXE drivers as candidate options, for a Qualcomm/Atheros AR8132 chip which made me search real hard at 1st. Actually i was about to give up when clues from a few Linux tools provided enough details for this riddle to unfold...

Well, that's what i had on my mind when i saw your present thread some short while ago. Good day, have fun!!

And thank you for sticking around this long.

 

[mbbrutman]

Hi, it sounds like you have some issues related to your choice of hardware. While many people can use a shim to convert NDIS or ODI to the packet driver specification, the most reliable thing to do is to find the native packet driver for your Ethernet hardware and use that. (If it even exists ...)

FreeDOS really should provide utility programs to make the mTCP DHCP configuration available to other programs. I know that https://github.com/ulrich-hansen/M2WAT exists for WATTCP based applications. It would be trivial for somebody to write the equivalent to help MSKermit. (I don't include either in mTCP because it is not in scope for the project and I have other things higher on the priority list to work on.)

Also keep in mind that FreeDOS is usually behind with what it ships due to the way the project is tested and packaged. You probably do not have the latest mTCP code if you got it from a FreeDOS distribution.

 

[Egzoset]

...it sounds like you have some issues related to your choice of hardware.

Not anymore, the tabletop PC has a Kingston EtheRx PCI 10/100 Fast Ethernet Adapter (KNE110TX) which been chosen after its performance was compared with that of multiple other candidates at hand exactly. While there's just no choice to reconfigure the laptop's internal hardware at all.

FreeDOS really should provide utility programs to make the mTCP DHCP configuration available to other programs. I know that ... M2WAT exists for WATTCP based applications.

Nice memories!

Well, actually this reminds me of what happens in case of problems affecting the input-file which mTCP relies on. If it's absent the file 'C:\FreeDOS\Bin\WATTCP.CFG' simply ain't created and we get this moot warning:

« Error opening mTCP config file C:\FreeDOS\mTCP.CFG »​

Or worse, if it IS present but somehow inaccurate then its contents are swiftly transferred with this rewarding message, nonetheless:

« WATTCP.CFG updated successfully. »​

E.G. No error trap here...

So when i questionned Google's IA this afternoon, relatively to such matter of validating this DOS network configuration, it suggested the following procedure as part of my preferred solution:

Place all three files (FIND_PKT.ASM, FIND_PKT.BAT, and FIND_PKT.TXT) in the same directory. Run FIND_PKT.BAT. The batch file will first create FIND_PKT.COM from FIND_PKT.ASM and then execute it to find the interrupt vector. The result (the interrupt number) will be displayed in the console. If no driver is found, it will display "Packet driver not found.".​

The 3 files are archived as an attached file called 'Find_Pkt.ZIP', in case other readers would be curious too.

It would be trivial for somebody to write the equivalent to help MSKermit.

My solution for MSKermit worked fine in 2001, still did in 2005 after a minor update and then again 20 years later. Thanks for your concern anyway! Besides, i did state those issues were reported since the thread was about « gathering ideas » i think, presumably to help improve mTCP's conveniency. Hence my apologies as the intent wasn't to interfere with anyone's own priority list. Please rest assured i'll deal with those critical IP numbers effectively and possibly even validate them further in the process, hopefully using the interrupt vector as it would also enable the use of 'Termin.COM', etc.

Good day, have fun!!

 

[mbbrutman]

Not anymore, the tabletop PC has a Kingston EtheRx PCI 10/100 Fast Ethernet Adapter (KNE110TX) which been chosen after its performance was compared with that of multiple other candidates at hand exactly. While there's just no choice to reconfigure the laptop's internal hardware at all.



Nice memories!

Well, actually this reminds me of what happens in case of problems affecting the input-file which mTCP relies on. If it's absent the file 'C:\FreeDOS\Bin\WATTCP.CFG' simply ain't created and we get this moot warning:

« Error opening mTCP config file C:\FreeDOS\mTCP.CFG »​

Or worse, if it IS present but somehow inaccurate then its contents are swiftly transferred with this rewarding message, nonetheless:

« WATTCP.CFG updated successfully. »​

E.G. No error trap here...

So when i questionned Google's IA this afternoon, relatively to such matter of validating this DOS network configuration, it suggested the following procedure as part of my preferred solution:

Place all three files (FIND_PKT.ASM, FIND_PKT.BAT, and FIND_PKT.TXT) in the same directory. Run FIND_PKT.BAT. The batch file will first create FIND_PKT.COM from FIND_PKT.ASM and then execute it to find the interrupt vector. The result (the interrupt number) will be displayed in the console. If no driver is found, it will display "Packet driver not found.".​

The 3 files are archived as an attached file called 'Find_Pkt.ZIP', in case other readers would be curious too.



My solution for MSKermit worked fine in 2001, still did in 2005 after a minor update and then again 20 years later. Thanks for your concern anyway! Besides, i did state those issues were reported since the thread was about « gathering ideas » i think, presumably to help improve mTCP's conveniency. Hence my apologies as the intent wasn't to interfere with anyone's own priority list. Please rest assured i'll deal with those critical IP numbers effectively and possibly even validate them further in the process, hopefully using the interrupt vector as it would also enable the use of 'Termin.COM', etc.

Good day, have fun!!

No problem at all .. it would have been more accurate for me to say MSKermit just wasn't on my radar because I don't use it.

Parsing the mTCP config file and generating a *good* WATTCP config file or the MSKermit environment variables should not be difficult, and I agree, there should at least some error handling. I would just really like to see the FreeDOS project take these kinds of projects up, as they are shipping multiple pieces of unrelated code and there is room for some basic integration work to make things easier for FreeDOS users.

 

[Egzoset]

Parsing the mTCP config file and generating a *good* WATTCP config file or the MSKermit environment variables should not be difficult

Although that ain't the savviest setup around i've finally made %ErrorLevel% useful just earlier today, which happens to depend on quite a few batch helpers, starting with 'PktChk.COM' also providing the software interrupt vector as a bonus. Then there's 'XEcho.COM', 'Strings.COM', 'TFind.COM' and the 'Var.BAT'/'VarSet.COM' duo.

My plan with the batch-file work is to take advantage of the 'C:\FreeDOS\Bin\FDNetPD.BAT' (custom) user-defined option while keeping it independent, e.g. even with other scenarios in mind like to pick IBM/PC DOS v3.3 instead. Who knows, i might want to put some old 10 Mbps ISA "combo" card in a venerable PC-XT that's been waiting for its turn these last couple decades - and this means dealing with disquettes, eventually with the benefit of an external hard-disk connected to its parallel port...

In the meantime having to rely on 'mTCP.CFG' integrity hardly felt satisfying so it's going work in reverse: by creating a new one each time FDNetPD is called, plus generation of more support file as required and of course a way to feed MSKermit with DOS environment variables and hence leave it's .INI file alone. The virtual modem RLFossil is on my ToDo list, and since TelNet BBSes rarely support the Kermit protocol perhaps i'd also make room to experiment with alternatives where ZModem happens to be functional. Anzio in on my mind but those memories are so vague at the moment... Which is more reason to fill my tool box pre-emptively rather than ditching away the accumulated sum of work every time a difficulty will arise.

...there should at least some error handling.

A reasonable amount of focus was put on the need to avoid situations where drivers get loaded blindly, idem for their unloading, when allowed...

I would just really like to see the FreeDOS project take these kinds of projects up...

FDNetPD and its helper files is going to remain hardware-specific, i'd hesitate to promise a universal solution as the drivers vary in features. For example using the "-U" flag to unload instead of "/U", etc... At best i may share it in hope someday it's put to good use by a guy who can adapt it, but i wouldn't dream of full integration that eventually calls for a database meant to deal auto-magically with every possible scenario!