• Please review our updated Terms and Rules here

Anomolies around STAT - and the meaning of certain displays.

You made me go back to my emulator code and yes, you're right. Too much time has passed:

Code:
;       Move the command line, but translate to upper case and remove the
;       carriage return at the end.

        lodsb                                   ; get the count
        xor     ah,ah
        mov     bx,ax
        stosb                                   ; store the count
        add     bx,di                           ; last+1 in control line
        mov     cx,(tpa-tbuf-1)                 ; count to move
Init_Request10:
        lodsb                                   ; get a byte
        call    ToUpper                         ; convert to uppercase
        stosb
        loop    Init_Request10                  ; loop...
        mov     byte ptr es:[bx],0              ; zero last+1 in control stmt
 
I used to have a null terminator, but removed it because I thought it was not supposed to be there... :(

A question - is it a part of the BIOS get string function ( that returns a null terminator ) or is it just added to the arguments buffer at 0080?

And yes, that was the issue - :) Stat doesn't give the error anymore.
 
The BIOS does not have an input string function. BDOS function 10 (READ CONSOLE BUFFER) does NOT place a NUL terminator in the buffer. CCP adds the NUL when copying the string from it's internal buffer (used for BDOS function 10) to 0080H (slightly different format). BDOS function 10 does not place a CR (or LF) in the buffer.
 
OK, so in changing my file system to support all elements of a directory entry
The BIOS does not have an input string function. BDOS function 10 (READ CONSOLE BUFFER) does NOT place a NUL terminator in the buffer. CCP adds the NUL when copying the string from it's internal buffer (used for BDOS function 10) to 0080H (slightly different format). BDOS function 10 does not place a CR (or LF) in the buffer.
OK that makes sense - that was how I implemented it, so that will work.

Now, I have hit another problem, with respect to Logical and Physical extents.

With a logical extent, or a small physical one, the CR tracks the current record against the RC record counter in the FCB.

What tracks the "Current Extent" counter?

Or is it virtualized somehow? eg, When loading FCBs with the allocations, it only loads a few at a time, and shifts those records to match the current extent?
It seems like either the disk needs to be accessed every time a new extent is loaded, rather than having a Current Extent counter and just loading the FCB up with the allocation groups related to all extents in that FCB, or does it manipulate the R0,R1, R2 records to maintain a "Defacto" Current Extent?

I'm lost on this one.

Thanks
David
 
OK, so in changing my file system to support all elements of a directory entry

OK that makes sense - that was how I implemented it, so that will work.

Now, I have hit another problem, with respect to Logical and Physical extents.

With a logical extent, or a small physical one, the CR tracks the current record against the RC record counter in the FCB.

What tracks the "Current Extent" counter?

Or is it virtualized somehow? eg, When loading FCBs with the allocations, it only loads a few at a time, and shifts those records to match the current extent?
It seems like either the disk needs to be accessed every time a new extent is loaded, rather than having a Current Extent counter and just loading the FCB up with the allocation groups related to all extents in that FCB, or does it manipulate the R0,R1, R2 records to maintain a "Defacto" Current Extent?

I'm lost on this one.

Thanks
David
So, in the *FCB* the EX and CR fields are used to locate the actual block and record in the file (in CP/M 2.2). It's been a very long time since I did random access files using that method, but I believe that is all there is. If doing sequence read/write, you can see those fields change after each operation (if running a true DRI BDOS). The *directory entry* simply records the size of the file - the "extent" in the sense of the outer border - using EX and RC. As the file is extended (during write), the relevant fields in the *directory entry* change to indicate the last valid record in the file. Note that the directory entry *on disk* is not updated immediately, only when the extent is closed (file closed).
 
It helps to understand the roots of CP/M. In the beginning, it was very primitive.
In CP/M 1.4 and earlier, there were no random-access APIs (BDOS calls 33 and 34). You figured out what the extent and record position and plugged those into the FCB.
You really didn't even have to issue an open call for a file, as long as you had a valid FCB that matched a directory entry.
CP/M 2.0 was an incremental improvement. But things really changed with MP/M. All of the sudden, the system had to track open files and enforce file locking, because you didn't want two user jobs writing to the same file (you can see how disastrous that would be, say, on just about any accounting application). CP/M 3.0 incorporated much of the MP/M API and disallowed many of the old CP/M 1. shenanigans.
MSDOS 1.0 followed CP/M 2.2's API and didn't introduce the notion of file handles until 2.0, when subdirectory APIs were introduced. Use of FCBs was discouraged--and I don't know if they still work in DOS 7.x. 22NICE uses file handles internally and dummies up FCBs for CP/M.
 
It used to be (and may still be) the case that one could open a file in CP/M and tinker with the block numbers in the second 16 bytes of the FCB to gain access to any part of the disk; even other user's files. I don't know if CP/M 3 or MP/M plugged that security hole or not.
 
CP/M, just like MS-DOS, were never secure. MP/M may have added FCB checksums to detect corruption, but even that is easily subverted. And the password protection is also easiy subverted. There is nothing to prevent malicious actors from accessing anything they want.
 
Given that there's no CPU privileged state, executing any user-supplied binary code is flirting with disaster. One of the reasons that systems running interpreted code were far more secure--there was no way for any user to get the system to execute user-supplied machine code. We ran 5 users on an 8085 and there was no problem with security--everyone ran interpreted code. You paid a penalty in speed, but it wasn't awful.
 
Hi Doug, Chuck,

I am not concerned about the security issues with accessing the hard disk directly - more I am trying to figure out what is correct behaviour of the FCB when using files with large extent masks.

For example, in normal disks, you open the file ( or just start accessing it directly through reads without opening, as Chuck pointed out ) and you set the extent and CR and go for it. Everything works. If you want another extent, you change the extent number. CP/M copies the directory entry over the FCB from bytes 2 to 32 if you "open" the file in advance, populating the allocation group tables in the second 16 bytes, but it also copies RC, EX, S1 and S2 regardless. EX stays the same because it was used to access the entry via the FCB - and the entry on the disk matches the entry in the FCB. The CR field doesn't get copied as also the R0, R1 and R2 fields because they are used to track current position.

This is not the case in large file extents that contain more than 16K per directory entry. If the entire directory entry is copied, then the EX field is either clobbered or not copied. ( At the moment I've gone with "Not Copied" ) when opening a file. This means I'll have to "open" the next "extent" via a full disk operation once again, which will read the disk again and just return the same directory entry, but reflecting the new EX value I placed there.

Alternately, If the EX field if copied on opening a file, then a "Current EX" field of some kind is needed. It does exist in R0 and R1 ( Bit 7 and bits 0,1,2,3 respectively ) so could be tracked via R0, R1, R2 if they are simultaneouly used along with CR, then changing EX is not manually required except when opening the actual FCB.

Or CP/M might have done something completely different.

What I don't know is which way CP/M went on this, since I don't have a machine with a big enough storage medium to do the test and see what the result is. :( While I am rewriting the entire OS much like MS-DOS did, I am also trying to achieve the exact same outcome as CP/M - ie, a "Clone" CP/M operating system.

My OS presently handles disks with extents of 16K just fine, and software recognizes them without issue. It's just the large disk I have which stores 64K per directory entry that I am not sure about how this should be represented.

Or, to phrase it another way, If I have a single directory entry like this example Doug first provided,

0000 - 00 52 4F 4F 54 5F 43 50 4D 49 4D 47 03 00 00 80 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F 10 - +ROOT_CPMIMG++++++++++++++++++++

and if I set EX to 0 and open it, what will I see in the FCB?

Will I see EX as 0 in the FCB post-open, or will I see 3 as per the directory entry post-open?

Thanks
David
 
I should add the question around the RC as well - Assuming the directory entry looks like this ( slightly less than 64 K in this example - let's say 56K ) -

0000 - 00 52 4F 4F 54 5F 43 50 4D 49 4D 47 03 00 00 40 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F 10 - +ROOT_CPMIMG++++++++++++++++++++

Should opening it four times while manipulating the EX field produce the following FCB results on open? ( Have I predicted what the EX and RC fields should do respectively ) - Or have I misunderstood something?

First time: (EX=0)
0000 - 00 52 4F 4F 54 5F 43 50 4D 49 4D 47 00 00 00 80 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F 10 00 00 00 00 - +ROOT_CPMIMG++++++++++++++++++++

Second time: (EX=1)
0000 - 00 52 4F 4F 54 5F 43 50 4D 49 4D 47 01 00 00 80 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F 10 00 00 00 00 - +ROOT_CPMIMG++++++++++++++++++++

Third time: (EX=2)
0000 - 00 52 4F 4F 54 5F 43 50 4D 49 4D 47 02 00 00 80 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F 10 00 00 00 00- +ROOT_CPMIMG++++++++++++++++++++

Fourth time: (EX=3)
0000 - 00 52 4F 4F 54 5F 43 50 4D 49 4D 47 03 00 00 40 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F 10 00 00 00 00 - +ROOT_CPMIMG++++++++++++++++++++
 
Extent can be up to 11 bits. 5 bits in the byte following the extension and 6 bits in byte S2. According to my very tired mind, that's 11 bits or 2048 extents of 16K each = 33,554,432 bytes.
Note that some DRI utilities directly manipulate these (and CR) bytes, rather than using the random read/write BDOS calls.
 
Extent can be up to 11 bits. 5 bits in the byte following the extension and 6 bits in byte S2. According to my very tired mind, that's 11 bits or 2048 extents of 16K each = 33,554,432 bytes.
Note that some DRI utilities directly manipulate these (and CR) bytes, rather than using the random read/write BDOS calls.

Were S1 and S2 implemented in CP/M 2.2?

Thanks
David
 
Firstly, CP/M 2.2 did not support the extension of EX to more than 5 bits.

How accurately you reproduce CP/M behavior depends on what set of programs you want to run. Most, if not all, of the DRI applications are well behaved in this respect. Most major application vendors' programs are also well behaved. This is somewhat similar to the issues that arose for CP/NET, where the FCB was essentially managed by the server (usually MP/M, but could be something else entirely). A wide variety of programs run fine on CP/NET.

The other fields in the directory entry, and FCB, had various purposes. There was a bit in one of those fields that was used to indicate a "dirty" file - one that needed the FCB fields written back to the directory entry (mostly anything that had been written to). The CP/M 2.2 version of SUBMIT used that and some other "features" to implement simple handling of the $$$.SUB temporary file.

So, EX and RC are pretty important. But other fields are less so.

I have a couple simulators (H8/H89 and Kaypro) that run "pure" DRI CP/M, and you could use those for reference to some of this more-obscure behavior. If you want, I can help you set one of those up. Both have options for harddisk, which would allow you to explore larger disks and files. Although, the H8/H89 is probably better for this as you can use CF-size disks (within the limits of CP/M). One advantage of ths simulators is that you can try out various things in CP/M and then go to the host PC and dump parts of the disk image to see what happened. They also allow tracing and various other debug features like dumping memory.
 
I have a couple simulators (H8/H89 and Kaypro) that run "pure" DRI CP/M, and you could use those for reference to some of this more-obscure behavior. If you want, I can help you set one of those up. Both have options for harddisk, which would allow you to explore larger disks and files. Although, the H8/H89 is probably better for this as you can use CF-size disks (within the limits of CP/M). One advantage of ths simulators is that you can try out various things in CP/M and then go to the host PC and dump parts of the disk image to see what happened. They also allow tracing and various other debug features like dumping memory.

Thanks for the offer - I would greatly appreciate it if I could trouble you to do so -

I had my OS running around Christmas time... Since then I've been fising bugs and mostly fixing things I misunderstood. I encountered the question about what to do with EX last night when I rewrote my code to support larger directory entries, and on reading, realized I had no way of tracking EX without reloading it each and every time I wanted to change extent due to the only place that held the information on how many logical extents were located in a single physical extent is on the disk surface itself, and not in memory.

And at that point, I was unsure as to what the correct behaviour was... I can make any number of assumptions that will work -and can completely head in a new direction as MS-DOS did, but what I want to do is start reconverging on how CP/M 2.2 was supposed to work rather than finding completely bespoke solutions to things.

If I could trouble you then setting up a simulator of a Kaypro with a hard disk would be great. Especially if it's running 2.2 - though I can still work with 3.0 otherwise.
 
Yes, I wasn't clear -- the extent extension into 11 bits is a CP/M 3. thing.
See here.

I'm less about being faithful to the limits of CP/M 2.2 (I do have that installed on Z80 and 8085 systems) than making CP/M programs relevant and useful.

Since my emulator was already working with 16 MB DOS 2. partitions, it made no sense to limit CP/M programs to small files. The idea behind my emulator was to seamlessly integrate CP/M-80 programs into an MS-DOS environment. It succeeds at that very well--if you want, to say use MAC, PIP or ED under MSDOS, you wouldn't even notice that they're not x86 code.

Now, when volumes get into the gigabyte range, I just dummy disk stats to the maximum possible. Same for network-mapped drives.
 
David, sent you a private message with info on my simulator. I'm tracking down a CP/M 2.2 image, for now there is just CP/M 3.
 
Regarding STAT source code .... if PLM is not your thing, there is a version of STAT for CP/M 68k that is written in C. I'd post it here, but I'm afraid it might violate VCFED standards? Look up the D. R. CP/M 68k distribution (not sure if vers. 1.2 or 1.3 would be best). I think you can find it there? Not sure how different STAT is for the Z80 vs. the 68k, but it might be a good starting point?

Roger
 
Regarding STAT source code .... if PLM is not your thing, there is a version of STAT for CP/M 68k that is written in C. I'd post it here, but I'm afraid it might violate VCFED standards? Look up the D. R. CP/M 68k distribution (not sure if vers. 1.2 or 1.3 would be best). I think you can find it there? Not sure how different STAT is for the Z80 vs. the 68k, but it might be a good starting point?

Roger

To be honest, I was kind of hoping for assembly, even 8080 - which I am not familiar with, but can convert to z80 easily and then understand. Even if I have to go through scan to text... But as it turned out, the problem was already figured out - it was a missing null terminator on the end of the arguments string when copying it to 0080 - That fixed it... Mostly... I still get some anomolous behavior, but I also get correct behavior and both are somewhat predictable, so I'll work with that for the moment.

Most recently it depends on which disk I run the application from, but I've made so many disk system code changes of late, I've probably introduced some new bug...

You know,


7c6bbf73f430a9c2f176484e44a61ef1-3996811917.jpg
 
Well, I'm still getting some strange results from STAT - Sometimes it works perfectly, sometimes not so.

EG;
M>stat *.* $S Size Recs Bytes Ext Acc 256 128 16k 1 R/W M:DRIVE-L:.IMG 64 128 4k 1 R/W M:L:_DRIVE.DSK 256 128 16k 1 R/W M:ROOT_CPM.IMG 32 32 1k 1 R/W M:RST-#08H.INT 32 32 1k 1 R/W M:RST-#10H.INT 32 32 1k 1 R/W M:RST-#18H.INT 32 32 1k 1 R/W M:RST-#20H.INT 32 32 1k 1 R/W M:RST-#28H.INT 32 32 1k 1 R/W M:RST-#30H.INT 32 32 1k 1 R/W M:RST-#38H.INT 128 256 8k 2 R/W M:Screen_1.IMG 512 256 32k 2 R/W M:VIDEO1.IMG Bytes Remaining On M: 41k

So the .INT files are all 4K and are placeholders for individual code to follow external interrupts that can be hooked by code to do something without affecting the TPU

The L: drive is 64K the Video is 128K and the root CP/M is 64K.

The size in records is correct, but the records is not.

Meanwhile, it says I have 41K on M:

yet;

M>stat DSK: A: Drive Characteristics 5056: 128 Byte Record Capacity 632: Kilobyte Drive Capacity 128: 32 Byte Directory Entries 32: Checked Directory Entries 128: Records/ Extent 16: Records/ Block 32: Sectors/ Track 1: Reserved Tracks L: Drive Characteristics 464: 128 Byte Record Capacity 58: Kilobyte Drive Capacity 64: 32 Byte Directory Entries 0: Checked Directory Entries 128: Records/ Extent 8: Records/ Block 32: Sectors/ Track 241: Reserved Tracks M: Drive Characteristics 8192: 128 Byte Record Capacity 1024: Kilobyte Drive Capacity 128: 32 Byte Directory Entries 0: Checked Directory Entries 512: Records/ Extent 32: Records/ Block 32: Sectors/ Track 0: Reserved Tracks M>STAT M: Bytes Remaining On M: 732k M>

This part works OK -

And so does this;

M>stat m: DSK: M: Drive Characteristics 8192: 128 Byte Record Capacity 1024: Kilobyte Drive Capacity 128: 32 Byte Directory Entries 0: Checked Directory Entries 512: Records/ Extent 32: Records/ Block 32: Sectors/ Track 0: Reserved Tracks M>


And even if I call it from another drive; eg;

A>stat m: Bytes Remaining On M: 732k

It works. ( Stat isn't even on A: or M: - it's just on L: which is on the "search" path for the CCP. )

So I'm not sure why it doesn't like working on the default drive. Seems I still have some inconsistencies with real CP/M - At some point I might need to get real 2.2 working in my emulator.

David
 
Back
Top