Booting from an FFS2 filesystem

Developer Otto Moerbeek (otto@) has been working on support to boot from FFS2. He writes in with the below article, to give us a little insight into the challenges he faced while working on this.

FFS2 filesystem support has been in OpenBSD for quite a while. FFS2 has a few advantages above FFS1: large partition support, 64-bit timestamps, faster newfs(8) and faster fsck(8), but it is only used for large (> 1TB) filesystems at the moment. The only drawback is that its meta-data overhead is a bit larger than FFS1 because of 64-bit instead of 32-bit blocknumbers and timestamps.

I decided that it was time to start using FFS2 in as many places as possible, and that includes booting from it. Booting is an area where there are quite large differences between the various platforms OpenBSD supports. The boot code interacts with the platform-specific firmware and the bootstrap process uses different vendor-specific mechanisms.

Luckily not all parts are platform specific: almost all platforms use boot(8) to load the kernel from the root filesystem. boot(8) is largely system independent but contains platform specific code to access disks via the platform provided firmware. For example, on i386 and amd64 BIOS based systems it uses BIOS calls to access the disks. boot(8) can be extended to access different filesystem types. Adding FFS2 support to it is as simple as extending a table with some entries.

However, boot(8) itself has to be loaded from disk, and this is where the various platforms differ the most. For the loading of boot(8) the most simple mechanisms is: the firmware itself loads boot(8) from a native (e.g. FAT) partition on the disk. Systems like armv7, arm64 and macppc use this method and adding ffs2 boot support was very easy for them.

Other platforms have a primary boot loader that is loaded from a known position on the disk which in turn loads boot(8) from the filesystem. The primary bootloader called bootblocks load boot(8) by either using some very basic filesystem access code or using information about the block numbers where boot(8) is stored on the disk. This information is constructed and stored to the disk in a known location by installboot(8).

For these systems (like i386, amd64, sparc64, hppa and alpha) I had to adapt the primary boot loader to be able to interpret FFS2 filesystem data.

After a few weeks I have FFS2 booting now working for almost all platforms. Luckily I cold re-use some existing work from various sources, e.g. former developer pedro@. Quite some time was spent in testing: I made sure to test my changes before sending out diffs for OKs. In some cases extra testing on various hardware versions was needed and I got good help from other developers.

amd64 and i386 were "interesting" in that the changes to the primary bootloader caused some BIOS implementations to hang. We still do not know why this case was only triggered on *some* BIOS implementation and why the fixed code does not expose the issue. So if you like a challenge: the issue seems to be related to some code or data being not aligned. See biosboot.S.

sparc64 is also special, its OpenFirmware bootblocks are written in Forth. I had to update the Forth compiler fgen(1) but for that I could take code from NetBSD, retrofit the softraid(8) support stsp@ did and have bootblocks that can load the secondary bootloader from FFS1, FFS2 and softraid partitions. This diff is not yet committed.

At the moment, the only systems lacking booting from FFS2 are landisk and luna88k. landisk could be done but it has severe restrictions on the size of its primary bootloader. This bootloader must interpret an FFS2 filesystem including 64-bit arithmetic that does not fit in the few hundred bytes it has left. A possible solution involves having a separate FFS1 and FFS2 xxboot(8) but I honestly do not think that is worth the trouble for this "exotic" system.

I did not look at the luna88k bootloader apart from noticing it is very different from other systems and since I do not have such a system it will be hard for me to work on. For the other systems it shows that having an attic full of old gear is worth it :-)

Next steps (likely after the upcoming release) will be enabling FFS2 in the installer in a paced manner, so we can spot issues before converting more platforms to start using FFS2. During my tests I did find and fixed one FFS2 issue: during the early boot the remount of root from r/o to r/w had a bug if the root filesystem was dirty.

Thank you Otto for working on this project and for the article!