Self Modifying Code... Best Practice or avoid?

[cjs]

I was making the point about the benefits of Smart Keyboard controllers with buffers, handshaking, and interrupts, which a rich enough person might have on a high-end terminal in the 1970’s...

Can you give some examples of such systems from the 1970s? And how did these terminals deal with making sure that the computer didn't drop the characters after the terminal had sent it? Or am I looking at things the wrong way here?

I'm rather surprised by all this; I'd not realised that multi-byte keyboard and serial buffers were so common i the 1970s that it would be thought to be an obvious missing feature on computers like the Apple II.

 

[Eudimorphodon]

Can you give some examples of such systems from the 1970s? And how did these terminals deal with making sure that the computer didn't drop the characters after the terminal had sent it? Or am I looking at things the wrong way here?

Professional terminals like the DEC VT series had transmit/receive buffers and hardware handshaking in the early 1970’s. And here’s the manual for a MITS Altair serial card explaining how it supports handshaking and interrupts.

I’m not sure why you’re obsessing so much on this. The people calling machines like the Apple II or TRS-80 “toys” were comparing them, very unfairly, to hardware costing five or ten times as much.

 

[cjs]

Professional terminals like the DEC VT series had transmit/receive buffers and hardware handshaking in the early 1970’s. And here’s the manual for a MITS Altair serial card explaining how it supports handshaking and interrupts.

I’m not sure why you’re obsessing so much on this. The people calling machines like the Apple II or TRS-80 “toys” were comparing them, very unfairly, to hardware costing five or ten times as much.

I'm not at all concerned about the how much the units cost or whether anybody considered the Apple II or TRS-80 a toy. I'm interested because this is whole area of late '70s computing history I didn't know existed. I was well aware that terminals and computers supported flow control, but I've looked at a few CP/M BIOSes and none of them that I recall made any attempt to use that. (Nor did anybody I know care about this.) The only place I ever saw typeahead being common was on minicomputers and multi-user micros.

So what were the, e.g., CP/M systems that supported typeahead via flow control for terminals with buffering? What were the UARTS (if any) before the 16550 that supported multi-character input buffers? What were the systems with built-in keyboards that supported this? I'd like to have a closer look at these.

(Feel free to start a new thread for this if you feel that's appropriate.)

 

[cj7hawk]

No need to open a new thread. This one wanders on and off topic

I've thought of using the video counters for refresh since my raster generators go through the full 8 bit sequence, whether displayed or not, and because of that, many times I've considered whether I should have arbitrarily limited vertical display lines to 192... But I didn't use DRAM because it wasn't fast enough in the era I was targetting... Only SRAM hit those speeds at a reasonable cost.

Also, I've often wondered what the strategy should be for implementing a type-ahead buffer for my own project and figured serial with handshaking isn't too bad a solution. I was going as far as thinking that clocked serial that is timed at 9600 would be even better, since it makes allowance for an external clock to be replaced with a simple timing circuit to allow sync to be used asynchronously. But I've done little more than muse about ideas, and mostly I figure a cheap CPU or MCU is still the best way to go.

Serial chips and uarts aren't needed. I remember making many serial comms boards that acted like a fixed uart in the past. A shift register in, one out, and some clock circuitry. And the good thing about designing the entire async board in common TTL chips means it's ready to send/receive without any config... Which supports other applications as well.

And I too am curious as to which CP/M machines supported type-ahead. And especially what are good choices for buffer sizes, and when the system should assume all additional characters should be dropped to avoid the DOS like issue of the keyboard holding buffer for minutes in some cases IIRC

 

[Eudimorphodon]

I was well aware that terminals and computers supported flow control, but I've looked at a few CP/M BIOSes and none of them that I recall made any attempt to use that

Dumb question: if you have a serial port that’s wired for hardware handshaking and configured for such what are you expecting in the BIOS? My vague understanding (CP/M is far from my favorite thing) is CP/M’s keyboard polling involves calling CONST regularly to see if a character is ready (this would be reading a UART flag for a serial port in this serial case), and calling “CONIN” to read the character. *IF* you have hardware flow control reading that character should assert the CTS flag and automagically inform a buffered keyboard/terminal that it’s free to send any additional characters it might be sitting on, right? CP/M doesn’t know a single little thing about type-ahead, sure, but would it not be a thing you’d get if your terminal had it?

 

[Chuck(G)]

I recall that in my CP/M BIOS, we buffered up to 50 characters ahead, but flushed said console buffer if a Ctrl-C was typed. Problem with CP/M was that there was no way to signal an abort by the BIOS routines--it was up to the application to detect and act on a pre-arranged signal.
We had the Trap signal wired to a key on the keyboard so at least things could be stopped when they got out of hand. Even MS-DOS "break" sequence is sort of dicey.

 

[Eudimorphodon]

I recall that in my CP/M BIOS, we buffered up to 50 characters ahead, but flushed said console buffer if a Ctrl-C was typed. Problem with CP/M was that there was no way to signal an abort by the BIOS routines--it was up to the application to detect and act on a pre-arranged signal.

Did you have interrupt-driven input hardware to allow your BIOS buffer routine to maintain that buffer independent of CONST/CONIN?

 

[Chuck(G)]

On my particular installation, everything was interrupt-driven, right down to the console display and keyboard, disk drives, etc. If you had a hard program hang, the screen went blank (8275 CRTC and 8257 DMAC, not running in auto-initialize mode). The TRAP key could at least restore the display and give you a debug prompt.
Life was interesting...

 

[Eudimorphodon]

Nice. I assume with this system your implementation of CONST just checked to see if the input buffer length was non-zero, and CONIN popped the oldest character in the buffer? I'm curious if someone knows of a specific vendor building machines that relied on serial terminals which used interrupts in their serial drivers to implement the same strategy to do a type-ahead buffer independent of whatever might be in the input device. (I would guess at least some did, since even MIT's serial cards could do interrupts, but like I said, CP/M isn't really my bailwick I can't point to a particular example.)

In any case, I never made the case that all S100 machines were "more sophisticated" than machines like the TRS-80 or Apple II in this area; if you had something like a Poly-88 with its VTI console card you basically had the video display of a TRS-80 married to the keyboard interface of an Apple II. It wasn't owners of machines like this that would be calling TRS-80's "toys".

Going back a bit:

In other areas, though, I think that certain design decisions that saved a bit of hardware probably hurt Tandy. Not being able to bank out the ROM of the Model I and Model III meant that they couldn't run CP/M without end user modification, and being able to run CP/M on such a relatively cheap machine might have made the Model I/III series more successful in the later '70s and very early '80s.

I've never bought this argument that not running CP/M was a serious problem with the TRS-80 line. I mean, sure, some people griped about it, and there was a market for CP/M modifications, but the thing you've got to keep in perspective here is even as late as 1979, when getting disk drives for your TRS-80 was becoming a "mainstream" option, the installed base of CP/M just wasn't that big. (Or, to flip it around, how can you argue the TRS-80 was hurt by the lack of CP/M when its market share in the US was easily 10x that of any single CP/M vendor's?) Sure, the TRS-80 never got a native port of Wordstar, but I don't see any evidence that there were gaping holes in the TRS-80's software base that needed to be filled by CP/M; this was less true when you're talking about the Apple II, which as you separately pointed out, didn't pass the trash-80 in market share until almost 1983. (At which point the TRS-80 was already in decline, arguably in large part because Radio Shack dropped the ball by fragmenting their computer line with the introduction of the Model III and Color Computer.) CP/M was a zombie by that point in the US, I don't think the TRS-80 would have lived longer with it than without it. (In fact the Model IV kind of proved that adding CP/M to the feature list was futile at this point.)

 

[Chuck(G)]

Well, our (up to 4) remote terminals were all interrupt-driven, but I don't recall if we kept a typeahead buffer on those. I don't think so.
The general idea with everything being interrupt driven was that we were deploying a multi-user system (not running CP/M as its primary OS), so polling wasn't really a viable option. Even had deadman timers on the motherboard I/O, so if a non-existent or unresponsive device was addressed, the OS knew about it in milliseconds.
Not bad for a single 8085 system.

 

[Eudimorphodon]

I could imagine possibly running into some entertaining situations if you had both a software typeahead buffer on the host *and* a send buffer on the terminal. Just think of all the havoc you could stack up.

 

[Chuck(G)]

I suppose, but there was little benefit to a send buffer--I'll have to look at some of my old terminal code (code inside a terminal) to refresh my memory as to how far that was carried out. I did do a terminal on contract that allowed uploading executable code to the terminal, which was interesting...I think it may have been used for manufacturing QA.
So much time, so many bits...

 

[Eudimorphodon]

Some terminals even in the 70’s had switchable “page modes” that let them do offline editing and submit complete blobs of data at once, I wonder if anyone ever leveraged that with CP/M. Considering how it only really natively understands teletype-level streams it wouldn’t have been the craziest thing in the world for some vertical market integrator to leverage whatever editing functions their chosen terminal offered for their custom software.

 

[Chuck(G)]

My first text editor used a Beehive Super Bee in page mode on CP/M. Took a bit of massaging as regards coding. I recall that end-of-line was 1F. But it was simple--basically, you spewed your text up, the terminal went offline, you did your editing, then hit "Transmit".

I think a couple of early WuPros used the same system. Early CPT perhaps? Text on disk is screen-indexed.

 

[cjs]

*IF* you have hardware flow control reading that character should assert the CTS flag and automagically inform a buffered keyboard/terminal that it’s free to send any additional characters it might be sitting on, right?

Oh, I see the problem: you're thinking that these UARTS will automatically deassert RTS once they've received a character, and not re-assert it until the input character register is read.

No, that's not how they work. The 8250, 6850, 6561, 65C62 and even the 16450 (introduced in 1987) all simply have a bit in a control register that software toggles in order to deassert (or assert) RTS. So you really only use it when you need to pause during bulk data receipt, such as after having received hundred or even thousands of bytes that you then need a big chunk of time to write to disk.

Here's the relevant excerpt from the 16450 datasheet:



Also, that toggling RTS on a character by character basis is risky, at best; you should generally be prepared to receive at least one or two more characters before the other end responds to the flow control signal and stops sending. With the UARTs above software transmitting at full speed is not going to be able to stop before at least one more character goes out: it will have loaded the transmit register as the (separate) transmit shift register is shifting out the previous character, and the transmit shift register will then immediately load and start transmitting the next character that was waiting in the transmit register. (And of course with XOFF you definitely need to be able to receive a handful of characters after sending it, since even ignoring the remote's processing time for that it will still take an entire character period to send.)

This is why even more modern UARTS with FIFOs generally interrupt at a high water mark a few characters below the receive FIFO being full. The 16550A, for example, has a 16 byte FIFO, but lets you set the receive interrupt to happen at 1, 4, 8 or 14 characters.

 

[cjs]

I've never bought this argument that not running CP/M was a serious problem with the TRS-80 line. ...the thing you've got to keep in perspective here is even as late as 1979, when getting disk drives for your TRS-80 was becoming a "mainstream" option, the installed base of CP/M just wasn't that big.

Well, from the figures I have for North America, though the TRS-80 had by far the largest market share in 1977 and 1978, in 1979 the "other" category (mainly CP/M) rose to equal the TRS-80 market share, and was pretty near double the TRS-80 in 1980 and 1981. (In 1981 "other" was 43%, Atari 8-bit was 21%, TRS-80 was 18%, and PET, Apple II, and IBM PC all combined were 21%.)

(Or, to flip it around, how can you argue the TRS-80 was hurt by the lack of CP/M when its market share in the US was easily 10x that of any single CP/M vendor's?)

Any single CP/M vendor, sure; but there were dozens of them. It seems reasonable to me that Tandy could have grabbed a reasonable chuck of all those vendors' market shares if they'd had a fairly cheap CP/M system available. A 32K two-drive Model I was going for about $2500 in 1979, which is somewhat less than I recall the prices being on similarly capable CP/M systems (though I could be wrong about that; I'd have to go back to old magazines to confirm), and even at a similar price, Tandy had a huge distribution network.

Sure, the TRS-80 never got a native port of Wordstar, but I don't see any evidence that there were gaping holes in the TRS-80's software base that needed to be filled by CP/M....

That may be a reasonable point; I don't have any good information on what all those CP/M systems out there in '78 and '79 were doing. I'm guessing many of them were doing accounting, and would have been systems installed by specialised vendors or systems integrators, who perhaps would not have gone with a TRS-80 anyway.

 

[Eudimorphodon]

Well, from the figures I have for North America, though the TRS-80 had by far the largest market share in 1977 and 1978, in 1979 the "other" category (mainly CP/M) rose to equal the TRS-80 market share

Those figures all come with massive grains of salt, and it’s also vey clear if you look at the notes that assuming that other category was “mainly CP/M” is a stretch(*), but sure, let’s pretend they sold as many CP/M machines as trash 80’s in 1979: so what? CP/M as a platform was fragmented and extremely user unfriendly, with annoying disk and terminal compatibility problems between vendors, and the market for it utterly faceplanted when better options came along. Radio Shack tying itself to an anchor wouldn’t have helped them swim any farther when the 8-bit Titanic hit the IBM iceberg.

*(Interpreting the “other” number as “mostly CP/M“ gets especially tenuous after 1980; even before that you have to factor in other home computers like the TI99/4 that were selling in decent numbers, and in 1981 the VIC-20 came out and sold like crazy. There were a few minor successes late in CP/M’s lifecycle, mainly cheap portables like Osborne and Kaypro, but people only bought them because they were cheap for a computer that could do “business things”, not because they wanted CP/M in particular. It was *never* a mainstream “home computer” option in North America.)

I’d say the main reason for the TRS-80’s decline post-1980 was Radio Shack badly mishandling their computer lineup in general, with bad product choices like splitting the line into *three* incompatible families, replacing the Model I with a more expensive machine that was less flexible and hardly improved (with severe compatibility issues on top), and their hostile attitude towards third party software developers. Many (potential) users had no idea there was so much software available for the TRS-80 because Radio Shack wouldn’t sell it through their stores, and general computer stores were disincentivized to stock it because they couldn’t sell the computer to go with it. They had all the built in advantages of having a huge retail distribution network when the wave started in 1977, but they systematically blew it when computers became more than just selling the unit itself.

CP/M wasn’t the problem. Not even a little.

 

[Eudimorphodon]

Oh, I see the problem: you're thinking that these UARTS will automatically deassert RTS once they've received a character, and not re-assert it until the input character register is read.

Yep, apparently I got that wrong. I haven’t written a serial driver so I guess I assumed these UARTs had a mode where the handshaking worked in automatic mode.

But, as has been noted, if you have an interrupt driven serial port you can handle this handshaking despite CP/M’s polling based tty model, and at least some CP/M machines *did* do this… and even explicitly enabled implemented type-ahead buffers, so... whatever. If you want to keep obsessing over one tiny point it’s okay, I’ll yield and agree that keyboard buffering probably wasn’t common on computers that cost less than, I dunno, $5000 or so?, in the late 1970’s.

 

[cj7hawk]

Yep, apparently I got that wrong. I haven’t written a serial driver so I guess I assumed these UARTs had a mode where the handshaking worked in automatic mode.

But, as has been noted, if you have an interrupt driven serial port you can handle this handshaking despite CP/M’s polling based tty model, and at least some CP/M machines *did* do this… and even explicitly enabled implemented type-ahead buffers, so... whatever. If you want to keep obsessing over one tiny point it’s okay, I’ll yield and agree that keyboard buffering probably wasn’t common on computers that cost less than, I dunno, $5000 or so?, in the late 1970’s.

UARTs since the 16c550 are usually capable of supporting automatic flow control, and some older UARTS were also capable of this, and it's something I've implemented on logic-built uarts prior and I got that idea from elsewhere at the time in the 80s.

A bigger issue for early uarts (and even some modern ones ) is whether they are double-buffered - that is, can you receive a character in part before the current character is polled by the CPU.

Otherwise, once the next start bit comes along, your data disappears or becomes corrupted, so you have to receive under direct interrupt rather than polled mode.

There was a big move to get 16c450 uarts in the early 90s when Windows apps started to work with Modems and then everyone wanted 550s. The last uart I was working with was the 16c750 which has IIRC 64 byte buffers and autoflow.

My recollection was that keyboard buffering wasn't something generally required on low-end computers in the 80s because people didn't really type on them too fast, and generally you only pressed a key when the running program was already waiting for you.

Users were trained by the software of the era, and the software of the era was pretty big on "Press a key" type prompts to let you know it was time to press a key. Complete with jokes about the any key.

I never heard of a terminal that inserts a break into the buffer at the start rather than the end though. That was something I was unaware of. I never saw anything that buffered the keyboard before the PC came along with it's own computer in the keyboard.

CP/M wasn’t the problem. Not even a little.

Inability to run CP/M was definitely a negative consideration in such machines - at least in Australia and the UK. Prior to widespread recognition of the PC to set the direction of all future computing, CP/M and even S100 were common themes that people wanted and asked for. Wordstar was still the defacto standard. So at the very least, I'd say "A Little Bit" even if I don't know how much.

Inability to run the same software on multiple platforms was a common negative theme. Often you got a platform specific version, and yes, I know many CP/M versions were *specifically* for that version and the average user wasn not going to tweak their copy of wordstar, but the widespread beliefs amongst those who didn't use CP/M was that it would allow you to use all of the software written for CP/M for almost any machine.

Incorrect assumptions like that were pretty common in the early and mid 80s.

This is why even more modern UARTS with FIFOs generally interrupt at a high water mark a few characters below the receive FIFO being full. The 16550A, for example, has a 16 byte FIFO, but lets you set the receive interrupt to happen at 1, 4, 8 or 14 characters.

And it supported autoflow ( automatic assertion and checking of CTS/RTS ) so could be used between two systems in polled mode without interrupts enabled.

By the time the 550 came out, the needs of a multitasking OS and lack of interrupt support for many apps changed everything.

But the early interrupt was because the timeslices were pretty slow and there wasn't always control over the transmitting side anyway. Serial interrupts could be masked by other activities also - just because there's an interrupt pending doesn't mean the computer is going to respond right away under all circumstances... Especially if there's disk activity going on.

 

[Eudimorphodon]

So at the very least, I'd say "A Little Bit" even if I don't know how much.

I'll admit to exaggerating somewhat there, it was certainly a question that was asked and, yeah, there was a market for CP/M modifications. But I really do think it had very little to do with the ultimate success (and downfall) of the platform. The TRS-80 crashed and burned by 1982 because Radio Shack was selling a barely-improved five year old computer for too much money; being able to run an even staler OS wouldn't have helped. (And didn't, the Model 4 proved that.) In some alternate universe where the Model III had, I dunno, just fixed the bugs without breaking backwards compatibility, added high-res graphics onto it as a standard feature and *hadn't* jacked the base price up $300 maybe it would have fought things out with the Apple II a little longer. (The Apple had color, sure, but a *lot* of Apple II's ended up being paired to mono monitors.) But probably not much.