CP/M 2.x directory structures, boot strapping, etc.

[Plasmo]

The original post asked an interesting question: how do you create a CP/M disk from scratch when all you have are collection of CP/M programs online and new BIOS for a new machine (Z280 in this case) that you try to bring up? I also wrestled with this problem 6-7 years ago when I tried to port CPM68K to 68000 which was my first encounter with CP/M. I should have done Z80 and Z280 first which have much better online supports for porting CP/M to new hardware, but I didn’t and at the time I didn’t know about CPMtools and various images archive.

The intricacies of disk parameter block was difficult to understand in the beginning; BIOS was a collection of routines that do not seem to relate to each other; and there were lots of new concepts. So when everything came together, there was an exciting ‘Eureka’ moment.

The path I took was first wrote a simple monitor for the new hardware that can load and run program in memory. The monitor should locate in the part of memory that does not interfere with CP/M BDOS/CCP/BIOS at high memory nor CPM application program at low memory. 0xB000-0xBFFF is a good place for Z80/Z280 monitor. The monitor also has a function to format the disk by writing E5 into disk directory area. Use this monitor to load BDOS/CCP/BIOS in high memory as well as XMODEM in low memory, then jump to BIOS entry point to start CP/M and ‘save xmodem.com’ to disk. If all that worked, then I used xmodem to bring in all CP/M system and application files.

This is how I brought up CP/M and created the first disk. I like to hear other’s experiences.
Bill

 

[cj7hawk]

This is how I brought up CP/M and created the first disk. I like to hear other’s experiences.
Bill

Mine is running in an emulator. Originally, I just had the emulator startup load the binaries into place and jump to 0 to start, with everything loaded, but as I built a realistic machine into the emulator, I needed to have it do more than that. Keep in mind I was also writing my own CP/M at the time, so my experience isn't quite the same.

Normally, the BIOS should include a way to reload the BDOS / CCP and should know how to load the files into memory via the reset, generally pulling the BDOS from the Disk, and then having it load the CCP, although variations I've heard of include;

* Loading both the BIOS and BDOS into memory as a single contiguous block from track 0, then resetting to load the CCP.
* Loading the CCP and BIOS and BDOS into memory and then executing to build the zero page.

In my case, I set hardware to reboot at F0000 ( CPU at 0000, but located at the top of a 1Mgb Memory ) which sets up the basic MMU, then loads both the BDOS and BIOS into memory at F000, sets up the main ramdisk and then jumps to the BIOS reset. The BIOS reset then pages out the ROM (which only exists from 0000 to 0FFF) and builds the zero page, then loads the CCP binary and executes it. I have the hardware allow the memory to be read as I/O, so the boot rom looks just like a 64K disk image, which the BIOS loads into Drive L: then it reads CCP.BIN into memory and executes it.

The first time? Yes, it's a very steep learning curve - at least it was for me. Also, I wasn't fully compatible with CP/M so I had further problems executing software, and my emulator had bugs too, so I couldn't get things to run correctly. I originally intended to load CP/M first, but it never worked, so I wrote my own OS to test the emulator. When the emulator was fully debugged, and could run CP/M software, then I tried again and could run real CP/M on it.

I had a hard time understanding the intricacies of the DPH and DPB, and how it all worked. Some I somewhat guessed, and I could see where problems would occur, but wasn't sure how CP/M treated those problems. The last major problem I fixed was multiple extents in the same directory entry. I detailed that here in the forum at the time.

Now I also test my software for CP/M on an Amstrad to ensure I don't miss any compatability requirements, and I am just putting together some hardware to run everything on a real machine that reflects what the emulator thinks it looks like.

p.s.
The vectors really had me stumped for a while too, as did how the checksum worked.

 

[bakemono]

The path I took was first wrote a simple monitor for the new hardware that can load and run program in memory. The monitor should locate in the part of memory that does not interfere with CP/M BDOS/CCP/BIOS at high memory nor CPM application program at low memory. 0xB000-0xBFFF is a good place for Z80/Z280 monitor. The monitor also has a function to format the disk by writing E5 into disk directory area. Use this monitor to load BDOS/CCP/BIOS in high memory as well as XMODEM in low memory, then jump to BIOS entry point to start CP/M and ‘save xmodem.com’ to disk. If all that worked, then I used xmodem to bring in all CP/M system and application files.

This is how I brought up CP/M and created the first disk. I like to hear other’s experiences.
Bill

My ROM monitor contains an XMODEM receive function and runs in Z280 supervisor mode, while I run CP/M in user mode so they don't have to share address spaces. My BIOS also uses the system call instruction 'SC' to access some character I/O routines in the ROM and modify the I/O page register (which can't be done from user mode). My first test of CP/M was to transfer two copies of a CCP/BDOS/BIOS image which was padded with $E5 bytes at the end. One of these images, at $80000, served as the RAMdisk, with code in the first two 'tracks' followed by the empty directory. The other got loaded into it's own 64KB user area at offset $DC00 and then executed by jumping to the BIOS coldboot vector. Subsequently I made a small program on the PC side that just takes a list of files and builds a disk image to contain them, including the OS code in the first 8KB. Then I added a routine to the ROM code that can 'boot the image' so I don't have to manipulate things manually.

 

[cjs]

There is very little in a CP/M file system which can break.
Early UNIX and FAT are notorious for easily breaking and recovery through fsck/chkdsk is slow on large disks.

Well, there's obviously something that can break. I've never seen a Unix filesystem completely and unrecoverably wiped just by hitting reset while the machine was running (and not writing to disk!).

 

[durgadas311]

Well, there's obviously something that can break. I've never seen a Unix filesystem completely and unrecoverably wiped just by hitting reset while the machine was running (and not writing to disk!).

If you go back to Unix of that era, it was quite common. I've spent hours recovering a VAX BSD-4.3 system after some simple mishap. But, I've not seen an idle CP/M system ever have a disk problem by something like a reset, other than possibly losing data on a file that was being written and wasn't closed (i.e. if a program was not well designed, such that it would wait for user input while a file was partially written). It sounds like you are talking extremes here, but we're not talking about "completely and unrecoverably wiped" but rather suffering filesystem damage that requires effort to recover from (and may result in lost data). These comments are referring to the simplicity of the CP/M filesystem and how that speaks to reliability, compared to more complex filesystems which have more ways for things to go wrong.

 

[cjs]

It sounds like you are talking extremes here, but we're not talking about "completely and unrecoverably wiped" but rather suffering filesystem damage that requires effort to recover from (and may result in lost data).

So how would one recover the diskette in that video? CP/M doesn't come with anything like fsck or CHKDSK, as far as I'm aware, so I don't know how you'd recover from that situation where it simply refuses to read the diskette except with perhaps a sector editor.

These comments are referring to the simplicity of the CP/M filesystem and how that speaks to reliability, compared to more complex filesystems which have more ways for things to go wrong.

UFS (the Unix filesystem from 4th Edition onward, varying somewhat, but essentially the same even in the BSDs) is not really more complex than CP/M is, beyond having an extra layer of linking between directory entries and i-nodes which makes it possible to lose the name of the file (without losing the file itself, however). It's still a set of directory entries/inodes in one or more disk sectors, each entry/inode containing information about the file itself and a list of block numbers. The same kinds of things that cause problems with one would cause similar problems with the other, with the caveat that trashing one sector of entries/inodes might result in a greater loss on CP/M if it can't skip past the bad sector in the list of directory sectors.

I expect that one reason you might personally have seen CP/M to be more reliable is that it immediately writes out all data to the disk before returning, while Unices by default tend to cache writes. (This is why Unix programs concerned about ensuring something is "committed" do a sync after their write call.) You can, on most Unices, use a mount option to ask the system not to return from write calls until the data are actually written, which would fix that problem, albeit at a cost in performance.

 

[durgadas311]

...
UFS (the Unix filesystem from 4th Edition onward, varying somewhat, but essentially the same even in the BSDs) is not really more complex than CP/M is, beyond having an extra layer of linking between directory entries and i-nodes which makes it possible to lose the name of the file (without losing the file itself, however).
...

You and I have drastically different definitions of "complex" then. Knowing the CP/M filesystem very well, and enough about the BSD 4.3 filesystem of that era, I see things quite differently.

Regarding that youtube video, I don't see anything in there that really tells us what happened, only the user's guess. Resetting any hardware while disk I/O is in progress is highly likely to trash the formatting on the disk, requiring a total reformat (complete data loss). That is true regardless of the filesystem. Some platforms might have allow single-track formatting and thus allowed for finer-grained recovery if one was willing to do the work.

 

[cjs]

You and I have drastically different definitions of "complex" then. Knowing the CP/M filesystem very well, and enough about the BSD 4.3 filesystem of that era, I see things quite differently.

Are you looking at the data structures, as opposed to the code? What about the UFS data structures is significantly more complex than the CP/M ones? How do they fail and how well can they be recovered in the face of various errors?

Regarding that youtube video, I don't see anything in there that really tells us what happened, only the user's guess. Resetting any hardware while disk I/O is in progress is highly likely to trash the formatting on the disk....

I'm not sure how that's relevant, given that there was no I/O in progress.

...requiring a total reformat (complete data loss). That is true regardless of the filesystem.

This is simply wrong. On BSD FFS, where there are multiple copies of the superblock, trashing a few seconds of I/O operations will leave the majority of files intact: the only ones affected can be those that were actually being changed and others with inodes in the same block. Even on UFS with a single superblock, the filesystem is just as recoverable, unless the superblock itself is trashed. And the superblock being trashed is roughly the same situation as the first CP/M directory sector being trashed, except at least with UFS you still can make some reasonable recovery if you know the disk parameters.

 

[durgadas311]

Are you looking at the data structures, as opposed to the code? What about the UFS data structures is significantly more complex than the CP/M ones? How do they fail and how well can they be recovered in the face of various errors?


I'm not sure how that's relevant, given that there was no I/O in progress.


This is simply wrong. On BSD FFS, where there are multiple copies of the superblock, trashing a few seconds of I/O operations will leave the majority of files intact: the only ones affected can be those that were actually being changed and others with inodes in the same block. Even on UFS with a single superblock, the filesystem is just as recoverable, unless the superblock itself is trashed. And the superblock being trashed is roughly the same situation as the first CP/M directory sector being trashed, except at least with UFS you still can make some reasonable recovery if you know the disk parameters.

I'm not going to keeping arguing about this. You are free to believe what you want. My experience has shown me a different reality. You and I are seeing the same thing quite differently, and I can't explain that.

 

[Svenska]

I've never seen a Unix filesystem completely and unrecoverably wiped just by hitting reset while the machine was running (and not writing to disk!).

I have. Your turn.

If your hardware is flaky enough to wipe the low-level formatting by accident (such as many 8" systems during power-on/power-off), no file system code will protect you from damage. UNIX never ran on hardware as flaky as the stuff often used in the CP/M era. Especially not then.

 

[cjs]

I have. Your turn.

I took my turn already to say that the CP/M filesystem is not, by nature of its design, more reliable than UFS. You've seen a Unix system unrecoverably wipe an FS not being written during a power cycle, just like the CP/M system did, which doesn't contradict my statement at all given that I've just posted a video of a CP/M system doing exactly that.