Difference between revisions of "File systems"

From Ninerpedia
Jump to navigation Jump to search
 
(13 intermediate revisions by 3 users not shown)
Line 8: Line 8:
|-
|-
! 00
! 00
| rowspan="5" colspan="2" | Volume name.
| rowspan="5" colspan="2" | Volume name  
| rowspan="5" class="descr" | May not contain a dot (".") because this is used as the path separator. Padded with spaces.
| rowspan="5" class="descr" | May not contain a dot (".") because this is used as the path separator. Padded with spaces.
|-
|-
Line 169: Line 169:
|-
|-
! 12
! 12
| colspan="2" | Level 3 record count
| colspan="2" | Level 3 record count (little-endian)
|-
|-
! 14
! 14
Line 195: Line 195:




The '''Extended record length''' field (offset 0x0A) is used for record lengths beyond 255; up to 255 bytes the length is given by the byte at offset 0x11. The HFDC introduced this field, but it is unknown whether it was ever used, and whether indeed both bytes (0x0A, 0x0B) are significant; maybe the byte at 0x0A is assumed to be zero. In any case we can say that the record length calculates as RL = AABBCC, where AA is at 0x0A, BB is at 0x0B, and XX is at 0x11.
The '''Extended record length''' field (offset 0x0A) is used for record lengths beyond 255; up to 255 bytes the length is given by the byte at offset 0x11. The HFDC introduced this field, but it is unknown whether it was ever used. According to [1], the Extended Record Length field defines the length of records of data files when the ''Logical Record Length'' field (offset 0x11) is 0.
 
The '''File status flags''' are defined as follows (see also [[TIFILES format]]):
 
{| class="tfiflags"
|-
!
! MSB
!
!
!
!
!
!
! LSB
|-
| 0
| FIXED
| rowspan="2" style="background-color:#eeeeee"| Reserved
| normal
| Unmodified
| Unprotected
| rowspan="2" style="background-color:#eeeeee" | Reserved
| DISPLAY
| Data
|-
| 1
| VARIABLE
| [[Emulate File format|Emulate File]]
| Modified
| Protected
| INTERNAL
| Program
|}
 
The '''EOF offset''' is the location in the last sector where the EOF marker is found. Only variable length data files have an EOF marker (0xff). For program files and fixed length files this field points to the first byte after the file contents. If the EOF offset contains 0, the last sector is completely filled with data.
 
To recreate the actual file length,
 
* the total number of sectors must be multiplied by 256
* the number of bytes past the EOF marker must be subtracted (if the offset is not 0).
 
That is, if the number of sectors is 10 and the EOF offset is 36, the file length is (10*256) - (256-36) = 2340. If the EOF offset is zero, we get the full 2560 bytes.
 
 
The '''Level 3 record count'''  (offset 0x12 and 0x13) specifies the number of [[records]] in [[data files]]; this means the number of times you can execute an INPUT operation on that file. In the case of fixed length records, the field "Number of Level-3 records" contains the highest record actually written to. If the last sector is filled as far as possible, we have
 
:''L3 = Records/Sector * Total number of sectors''
 
This number is required to determine the highest record number, especially if the last sector is not filled completely.
 
In the case of variable length records, it contains the highest sector actually written to and should therefore be equal to the field "Total number of sectors".
 
For program files, 0x0000 is usually found in this field.
 
'''NOTE''': The bytes in this field are in '''reverse order''' (little-endian).
 
 
The '''Creation''' and '''Update time specification''' are two 16-bit words, the bits having the following meaning:
 
* hhhh.hmmm.mmms.ssss
* yyyy.yyyM.MMMd.dddd
 
With only 5 bits for seconds, the timestamp has a resolution of 2 seconds. The years reach from 1970 (values 70..99) to 2069 (values 0..69).
 
These bytes were reserved for future expansion, and will always be 0 on a disk formatted with the TI Disk Manager Module.


The '''Level 3 record count''' specifies the number of [[records]] in [[data files]]; this means the number of times you can execute an INPUT operation on that file. Note that the value is specified in '''little endian''' format, that is, with the lower byte first, then the higher byte: The count calculates as C = BBAA with BB at offset 0x13 and AA at offset 0x12.


The '''Data chain pointers''' are used to refer to sections on the disk where fragments of the file will be found. There is always at least one data chain pointer; if there are more, the file is ''fragmented''.  
The '''Data chain pointers''' are used to refer to sections on the disk where fragments of the file will be found. There is always at least one data chain pointer; if there are more, the file is ''fragmented''.  
Line 244: Line 308:
This means: The file occupies the sectors >0036 (sector 0), >0037 (sector 1), >0044 (sector 2), ... >041b (sector 16). Note that N does not provide the length of the current chain element but the total number of read sectors after reading this chain element. Counting from 0, and together with the directory entry, we have
This means: The file occupies the sectors >0036 (sector 0), >0037 (sector 1), >0044 (sector 2), ... >041b (sector 16). Note that N does not provide the length of the current chain element but the total number of read sectors after reading this chain element. Counting from 0, and together with the directory entry, we have


:''The number of sectors which a file occupies on the disk is N<sub>last</sub>+2''
:''The number of sectors which a file occupies on the disk is N<sub>last</sub>+1''


Or you may want calculate the intervals as follows:
Or you may want calculate the intervals as follows:
Line 333: Line 397:
|-
|-
! 1A
! 1A
| colspan="4" | Sector number of DSK1 emulation file
| colspan="4" | AU number of DSK1 emulation file
|-
|-
! 1C
! 1C
Line 347: Line 411:
|-
|-
|}
|}
BS = Buffered head stepping (one bit, 0 or 1)


The W,I,N values may be present after initial formatting. Usually they are replaced by the given settings.
The W,I,N values may be present after initial formatting. Usually they are replaced by the given settings.


'''Reserved AUs''': when multiplied by 64 (0x40), declares the start AU for file contents. For instance, with a value of 0x15, file contents are stored at AUs 0x0540 and higher.
'''Reserved AUs''' (0-255): Value is divided by 64 (0x40) and declares the start AU for file contents (0-16320). For instance, with a value of 0x15, file contents are stored at AUs 0x0540 and higher.


There are 114 slots for pointers to subdirectories (from 0x1c to 0xff, 2 bytes each), so this is the maximum number of subdirectories in a directory.
There are 114 slots for pointers to subdirectories (from 0x1c to 0xff, 2 bytes each), so this is the maximum number of subdirectories in a directory.
==== Special MFM parameters ====
'''Step speed''' (0-255): Controllers like the HDC9234 on the HFDC card use the value for step timing. The number is controller-specific.
'''Reduced write current''' (0-255): Value is divided by 8 and indicates the cylinder number (0-2040) starting from which the write current in the write head is reduced.
'''BS''' = Buffered head stepping (one bit, 0 or 1): Some drives allow for fast stepping by increasing a step counter and then performing a fast, single seek.
'''Write precompensation''' (0-127): Value is divided by 16 and indicates the cylinder number (0-2032) starting from which the timing of the flux writing is changed.


=== Volume Information Block (SCSI) ===
=== Volume Information Block (SCSI) ===
Line 628: Line 700:


that is 248 MiB. Any bigger drive will stay unused beyond this point.
that is 248 MiB. Any bigger drive will stay unused beyond this point.
== References ==
[1] Myarc, Inc.: ''Hard and Floppy Disk Controller / Users manual''
[2] Western Horizon Technologies: ''Small Computer Systems Interface Host Adapter Card / Software Interface Specification'', 1993
[[Category:Programming]]

Latest revision as of 20:50, 31 August 2023

Floppy Disk File System

Floppy file systems had no notion of subdirectories for most of the time. Disk controllers like the Myarc FDC and HFDC introduced a directory scheme where the root directory may have up to three subdirectories. These subdirectories may have no subdirectories by themselves.

Volume Information Block

00 Volume name May not contain a dot (".") because this is used as the path separator. Padded with spaces.
02
04
06
08
0A Total number of sectors
0C Sectors per track "D"
0E "S" "K"
10 Protection Tracks per side
12 Number of sides Density Density may have values 0 .. 4 (see below)
14 Directory 1 name Same constraints as for file or volume name
16
18
1A
1C
1E Sector number of FDIR of dir 1 Null if no directory exists. Only on start of AU.
20 Directory 2 name
22
24
26
28
2A Sector number of FDIR of dir 2
2C Directory 3 name
2E
30
32
34
36 Sector number of FDIR of dir 3
38 Allocation bit map Each bit represents a collection of sectors on the disk (see below)
3A
3C
...
FE

Density values:

  • Single (0,1): 125 kBit/s FM
  • Double (2): 250 kBit/s MFM
  • High (3): 500 kBit/s MFM
  • Ultra (4): 1MBit/s MFM

File Descriptor Index Record

The File Descriptor Index Record (FDIR) contains pointers to the File Descriptor Records. The pointers are arranged so that the list of files has a lexicographic order (alphabetic sorting). The FDIR of the root directory is always in sector 1. The FDIRs of the subdirectories are pointed to by the respective fields in sector 0.

The end of the list is marked by the first null pointer.

00 Sector number of first file descriptor record Always sector number, not AU
02 Sector number of second file descriptor record
04 Sector number of third file descriptor record
06 Sector number of fourth file descriptor record
...
FC Sector number of 127th file descriptor record
FE 0

Unlike the FDIRs in the hard disk file system there is no pointer back to the parent directory as the floppy file system does not allow nested subdirectories, and thus the parent is always the root directory.

File Descriptor Records

Each file entry in the diretory is defined by a file descriptor record.

00 File name
02
04
06
08
0A Extended record length used for logical record lengths beyond 255
0C File status flags Number of records/sec
0E Number of sectors currently allocated
10 End-of-file offset Logical record length
12 Level 3 record count (little-endian)
14 Date and time of creation hhhh.hmmm.mmms.ssss (2 seconds resolution)
16 yyyy.yyyM.MMMd.dddd
18 Date and time of last update
1A
1C Data chain pointer blocks
1E
...
FE


The Extended record length field (offset 0x0A) is used for record lengths beyond 255; up to 255 bytes the length is given by the byte at offset 0x11. The HFDC introduced this field, but it is unknown whether it was ever used. According to [1], the Extended Record Length field defines the length of records of data files when the Logical Record Length field (offset 0x11) is 0.

The File status flags are defined as follows (see also TIFILES format):

MSB LSB
0 FIXED Reserved normal Unmodified Unprotected Reserved DISPLAY Data
1 VARIABLE Emulate File Modified Protected INTERNAL Program

The EOF offset is the location in the last sector where the EOF marker is found. Only variable length data files have an EOF marker (0xff). For program files and fixed length files this field points to the first byte after the file contents. If the EOF offset contains 0, the last sector is completely filled with data.

To recreate the actual file length,

  • the total number of sectors must be multiplied by 256
  • the number of bytes past the EOF marker must be subtracted (if the offset is not 0).

That is, if the number of sectors is 10 and the EOF offset is 36, the file length is (10*256) - (256-36) = 2340. If the EOF offset is zero, we get the full 2560 bytes.


The Level 3 record count (offset 0x12 and 0x13) specifies the number of records in data files; this means the number of times you can execute an INPUT operation on that file. In the case of fixed length records, the field "Number of Level-3 records" contains the highest record actually written to. If the last sector is filled as far as possible, we have

L3 = Records/Sector * Total number of sectors

This number is required to determine the highest record number, especially if the last sector is not filled completely.

In the case of variable length records, it contains the highest sector actually written to and should therefore be equal to the field "Total number of sectors".

For program files, 0x0000 is usually found in this field.

NOTE: The bytes in this field are in reverse order (little-endian).


The Creation and Update time specification are two 16-bit words, the bits having the following meaning:

  • hhhh.hmmm.mmms.ssss
  • yyyy.yyyM.MMMd.dddd

With only 5 bits for seconds, the timestamp has a resolution of 2 seconds. The years reach from 1970 (values 70..99) to 2069 (values 0..69).

These bytes were reserved for future expansion, and will always be 0 on a disk formatted with the TI Disk Manager Module.


The Data chain pointers are used to refer to sections on the disk where fragments of the file will be found. There is always at least one data chain pointer; if there are more, the file is fragmented.

Every data chain pointer consists of three bytes with two hex digits each, so we have six hex digits

1 2 3 4 5 6
M2 M1 N1 M3 N3 N2

from where we get the 16-bit numbers

  • M = 0 M3 M2 M1: Sector or AU number of start of chain element
  • N = 0 N3 N2 N1: Highest file sector count (starting at 0) for this chain element

This may sound a bit confusing, therefore I'm adding an example. Here are the pointers for the data chain:

36 10 00, 44 30 00, ac 62 00, 03 b3 00, 17 04 01

Using the mapping from above, we get 

M =   0036    0044     02ac     0303      0417
N =   0001    0003     0006     000b      0010

Sectors
    0:0036  2:0044   4:02ac   7:0303   12:0417
    1:0037  3:0045   5:02ad   8:0304   13:0418
                     6:02ae   9:0305   14:0419
                             10:0306   15:041a
                             11:0307   16:041b

This means: The file occupies the sectors >0036 (sector 0), >0037 (sector 1), >0044 (sector 2), ... >041b (sector 16). Note that N does not provide the length of the current chain element but the total number of read sectors after reading this chain element. Counting from 0, and together with the directory entry, we have

The number of sectors which a file occupies on the disk is Nlast+1

Or you may want calculate the intervals as follows:

intv(i).start = Mi
intv(i).end = Mi + Ni - (Ni-1 + 1)

with N-1=-1.

Important comment to sector/AU usage: For disks with capacities up to 720 KiB (2880 sectors), the M-value refers to a sector. For disks with higher capacities (1.44 or 2.88 MiB, i.e. 5760 or 11520 sectors), M-values are AUs. The N-values are always sector counts, even when the M-value points to an AU.

Allocation Bit Map

This map shows which sectors are occupied by files on this file system. The map is located starting at byte 0x38 and reaching to the end of this sector, so its size is 200 bytes. This means that for file systems with more than 1600 sectors, clusters must be formed which serve as allocation units (AU).

Bits are organized in little-endian order. That is, a value of 0x01 declares the AU with the smallest number of a block of 8 to be allocated. On floppy disks with less than 1600 sectors, the allocation map is usually preset with 0x03, reserving sector 0 and sector 1.

AU 7 AU 6 AU 5 AU 4 AU 3 AU 2 AU 1 AU 0

Disks with a capacity of at most 400 KiB have an AU size of 1 sector (all SSSD, SSDD, DSSD, DSDD, with 40 tracks, at most 1440 sectors). Disks with at most 800 KiB space have 2 sectors per AU (720 KiB, 2880 sectors), with 1.4 MiB we have 4 sectors per AU, and the disks with ultra capacity (2.8 MiB) have 8 sectors per AU.

Hard Disk File System

The file system is defined by

  • the Volume Information Block
  • the Directory Descriptor Records
  • the File Descriptor Index Records
  • the File Descriptor Records, and
  • the Allocation Bit Map

Volume Information Block (HFDC)

The Volume information block (VIB) contains information about the complete file system on this volume. It is located in the first sector of the volume. The VIB takes the role both of the volume specifier and of the root directory specifier. Therefore, we find directory information as well as volume information in this block.

00 Volume name
02
04
06
08
0A Total number of allocation units
0C Sectors/Track Reserved AUs / W
0E Step speed / I Red. write current / N
10 Sect/AU -1 Heads -1 BS Write precomp
12 Date and time of creation
14
16 # of files # of subdirectories
18 AU number of file descriptor index record
1A AU number of DSK1 emulation file
1C AU numbers of subdirectories (DDR)
1E
20
...
FE

The W,I,N values may be present after initial formatting. Usually they are replaced by the given settings.

Reserved AUs (0-255): Value is divided by 64 (0x40) and declares the start AU for file contents (0-16320). For instance, with a value of 0x15, file contents are stored at AUs 0x0540 and higher.

There are 114 slots for pointers to subdirectories (from 0x1c to 0xff, 2 bytes each), so this is the maximum number of subdirectories in a directory.

Special MFM parameters

Step speed (0-255): Controllers like the HDC9234 on the HFDC card use the value for step timing. The number is controller-specific.

Reduced write current (0-255): Value is divided by 8 and indicates the cylinder number (0-2040) starting from which the write current in the write head is reduced.

BS = Buffered head stepping (one bit, 0 or 1): Some drives allow for fast stepping by increasing a step counter and then performing a fast, single seek.

Write precompensation (0-127): Value is divided by 16 and indicates the cylinder number (0-2032) starting from which the timing of the flux writing is changed.

Volume Information Block (SCSI)

The Volume information block (VIB) contains information about the complete file system on this volume. It is located in the first sector of the volume. The VIB takes the role both of the volume specifier and of the root directory specifier. Therefore, we find directory information as well as volume information in this block.

00 Volume name
02
04
06
08
0A Total number of AUs
0C Reserved Reserved AUs / W
0E Unused / I Unused / N
10 Sectors per AU Unused
12 Date and time of creation
14
16 # of files # of subdirectories
18 AU number of file descriptor index record
1A Reserved
1C AU numbers of subdirectories (DDR)
1E
20
...
FE

Byte >10 is the same as in the HFDC VIB (first 4 bits, adding 1). Thus, a value of >F0 means 16 sectors per AU and is the maximum number. The other four bits are not used. See above for the meaning of Reserved AUs.

Directory Descriptor Records

Each directory is defined by a directory descriptor record (DDR). It is structured similarly to a VIB.

00 Directory name
02
04
06
08
0A HFDC: Total number of AUs / SCSI: unused
0C unused "D"
0E "I" "R"
10 unused
12 Date and time of creation
14
16 # of files # of subdirectories
18 Pointer to file descriptor index record (FDIR)
1A Pointer to the parent directory AU
1C AU numbers of subdirectories (DDR)
1E
20
...
FE

File Descriptor Index Records

As space in a DDR is limited, the files are referenced outside the DDR, using a File Decriptor Index Record (FDIR). There is one FDIR per directory. The FDIR is also responsible for the ordering of file names in the directory. That is, new files are inserted into the list, preserving alphabetical order.

The end of the list is marked by the first null pointer. Note that the FDIR for hard disks contain pointers to allocation units, not to sectors.

The last entry of the list (position >FE) points back to the containing directory AU. With a sector size of 256 bytes we can have 127 files per directory at most.

00 AU number of first file descriptor record
02 AU number of second file descriptor record
04 AU number of third file descriptor record
06 AU number of fourth file descriptor record
...
FC AU number of 127th file descriptor record
FE AU number of directory descriptor record of this FDIR

File Descriptor Records

Each file is defined by a file descriptor record.

00 File name
02
04
06
08
0A Extended record length
0C File status flags Number of records/sec
0E Number of sectors currently allocated (also see extended information)
10 End-of-file offset Logical record length
12 Number of level 3 records allocated (also see extended information)
14 Date and time of creation
16
18 Date and time of last update
1A
1C "F" "I"
1E Pointer to previous FDR (AU)
20 Pointer to next FDR (AU)
22 Number of AUs allocated by this FDR for the file content
24 Pointer to file descriptor index record (AU)
26 Extended information (see below)
28 Data chain pointer blocks (AU)
2A
...
FE

For hard drives, the data chain pointer blocks are structured differently. We have four bytes per chain element, two for the start AU, and two for the end AU.

The extended information is a 16-bit word ABCD, built up as follows:

A B C D
Number of allocated sectors / 65536 Number of sectors of a VARIABLE file / 65536 Sector number in AU pointing to previous FDR Sector number of AU pointing to next FDR

The file system allows for multiple chained FIBs in case the cluster list does not fit into one FIB. Unlike the original FIB, the next FIBs may obviously be located in other sectors within an AU, not in the first one: It is possible to define which sector in the given AU actually contains the next FIB. However, this only makes sense if the first sector in the AU is from the same file; otherwise, when deleting a file, it would be impossible to decide whether an AU can be freed or not without checking each other file on the medium.

Anyway, it should be a pretty rare case; I've never seen a file that was so heavily fragmented to require more than one FIB.

Allocation Bit Map

The allocation bitmap is located in sectors 1 to 31 where each bit represents one allocation unit. That is, we have 

31 sector * 256 byte/sector * 8 bit/byte * 1 AU/bit = 63488 AUs = >F800 AUs.

Caution: The bit order in the bytes of the allocation map is big-endian, different to the situation with floppy disks.

AU 0 AU 1 AU 2 AU 3 AU 4 AU 5 AU 6 AU 7

The allocation bitmap limits the maximum size of a hard drive in terms of allocation units. The maximum number of sectors in an allocation unit is determined by byte >10 in the VIB which leaves only 4 bits for this number (adding 1), so the maximum is 16.

That is, we ultimately have a maximum usable space of 

16 sector/AU * 63488 AU * 256 byte/sector = 260046848 byte

that is 248 MiB. Any bigger drive will stay unused beyond this point.

References

[1] Myarc, Inc.: Hard and Floppy Disk Controller / Users manual

[2] Western Horizon Technologies: Small Computer Systems Interface Host Adapter Card / Software Interface Specification, 1993

Retrieved from "http://www.ninerpedia.org/index.php?title=File_systems&oldid=50844"