Encrypting the root filesystem with softraid

Stefan Sperling (stsp@) writes in with a summary on encrypting the root filesystem with softraid.

“

When softraid is used for disk encryption the root partition is usually left unencrypted to allow bioctl(8) to ask for a passphrase upon boot (as we wrote earlier).

I don't want to use such a setup for several reasons:

• I don't want to remember yet another passphrase.
• Sensitive data (such as keys stored in /etc) on the root filesystem is exposed when the system falls into the wrong hands.
• The "Passphrase:" prompt from bioctl(8) gives away the fact that encryption is being used. Instead, I want the system to make a more or less broken appearance when it is booted by an attacker.

Thankfully, in addition to passphrases, softraid also supports storing keying material on a key disk.

When a key disk is used softraid can automatically assemble volumes at boot. This way it's possible to encrypt the root filesystem, leaving only a small unencrypted boot partition that contains boot loader files and kernels.

The key disk should of course be removable. USB memory sticks or CF/SD cards are an obvious choice. The key is stored within softraid metadata of a partition of type RAID. This partition can be very small, so there is plenty of room left for other data partitions on the key disk.

With a few manual steps, it's possible to set up a fully encrypted system during installation. Familiarity with the regular OpenBSD install procedure is highly recommended before attempting this.

The following description is valid for OpenBSD/i386 4.9. It may not be applicable to other architectures or later releases. There are plans among developers to make some of these steps easier in the future.

At the (I)nstall, (U)pgrade, (S)hell prompt, pick "shell".

Set up disk partitions in the MBR. In my case, the disk is called sd0, and OpenBSD is the only OS on the computer, so running fdisk -i sd0 is sufficient:

# fdisk -iy sd0
Writing MBR at offset 0.
# fdisk sd0                                            
Disk: sd0       geometry: 8270/240/63 [125045424 Sectors]
Offset: 0       Signature: 0xAA55
            Starting         Ending         LBA Info:
 #: id      C   H   S -      C   H   S [       start:        size ]
-------------------------------------------------------------------------------
 0: 00      0   0   0 -      0   0   0 [           0:           0 ] unused      
 1: 00      0   0   0 -      0   0   0 [           0:           0 ] unused      
 2: 00      0   0   0 -      0   0   0 [           0:           0 ] unused      
*3: A6      0   1   2 -   8269 239  63 [          64:   125042336 ] OpenBSD 

Next, run disklabel -E sd0, and set up 3 disklabel partitions within the OpenBSD MBR partition. Partition 'a' of type 4.2BSD will be used for booting. Partition 'b' will be used for swap (which is already being encrypted so there is no need to put it into the softraid volume). Partition 'd' should be of type RAID and will host everything else. The result should look something like this:

# disklabel sd0
[...]
#                size           offset  fstype [fsize bsize  cpg]
  a:           136000               64  4.2BSD   2048 16384    1 
  b:          8376496           136064    swap                   
  c:        125045424                0  unused                   
  d:        116529840          8512560    RAID  

Now it's time to set up the key disk (which in my case is sd1). If device nodes for the disk do not already exist in /dev, they must be created first:

# cd /dev
# sh ./MAKEDEV sd1

The key disk needs an OpenBSD partition in the MBR, which can be small (something like one megabyte is sufficient). Use fdisk to create it.

Then, create a small partition of type RAID (using disklabel -E sd1). I used the letter 'd' for this partition:

# disklabel sd1
[...]
#                size           offset  fstype [fsize bsize  cpg]
  c:          3970048                0  unused                   
  d:            16001               64    RAID 

Now we're ready to create the crypto volume. The key will be generated automatically from data in the kernel's entropy pool:

# bioctl -C force -c C -l /dev/sd0d -k /dev/sd1d softraid0

The kernel will spit out a message like:

scsibus3 at softraid0: 1 targets
sd2 at scsibus3 targ 0 lun 0: <OPENBSD, SR CRYPTO, 004> SCSI2 0/direct fixed
sd2: 56899MB, 512 bytes/sec, 116529312 sec total

This disk cannot be used without corresponding device nodes, so they must be created:

# cd /dev
# sh ./MAKEDEV sd2

It's good practice to zero the first megabyte of the newly created crypto volume:

# dd if=/dev/zero of=/dev/rsd2c bs=1m count=1

The install script can now be run to install OpenBSD onto the sd2 disk. When the installer asks for the root disk, answer with the name of the crypto volume. Do not let the installer touch any of the other disks. DO NOT REBOOT after the install, because the system cannot boot from sd2!

# cd /
# install

Once the install script is done, the system must be made bootable.

Create a filesystem on the boot partition:

# newfs /dev/rsd0a

Install the first-stage boot blocks (platform-specific; this example is valid for i386 and amd64 -- see installboot(8) for details):

# /usr/mdec/installboot -v /usr/mdec/boot /usr/mdec/biosboot sd2

Copy the kernel to the boot partition:

# mount /dev/sd0a /mnt2
# cp /mnt/bsd /mnt2/bsd
# cp /mnt/bsd.rd /mnt2/bsd.rd

Make the kernel ask for the name of the root file system after booting. You can of course also type boot -a at the boot prompt instead.

# mkdir /mnt2/etc
# echo "set howto -a" > /mnt2/etc/boot.conf

Now it is safe to reboot.

If the key disk is present while the system boots, the kernel will assemble the softraid volume automatically. When the kernel asks for the root partition, point it at the 'a' partition within the softraid volume (in my case, sd2a). It will also ask for a swap device, which is on the 'b' partition of the physical disk (sd0b in my case).

After booting, create an image of the key disk for safekeeping on a different computer and/or an external hard disk:

# dd if=/dev/rsd1c of=keydisk.img bs=1m

For convenience, it's possible to hard-code the name of the root filesystem and swap device with a small tweak to the kernel configuration:

Index: GENERIC
===================================================================
RCS file: /cvs/src/sys/arch/i386/conf/GENERIC,v
retrieving revision 1.709
diff -u -p -r1.709 GENERIC
--- GENERIC     17 Feb 2011 20:14:30 -0000      1.709
+++ GENERIC     6 Mar 2011 17:27:29 -0000
@@ -32,7 +32,8 @@ option                PROCFS          # /proc
 option         NTFS            # NTFS support
 
 # or use               root on nfs swap on nfs
-config         bsd     swap generic
+#config                bsd     swap generic
+config         bsd     root on sd2a swap on sd0b dumps on sd0b
 
 mainbus0 at root

When hard-coding the root and swap device in the kernel configuration, it is important to also hard-code the dump device! Else, the kernel might end up trying to use the crypto volume as a dump device, potentially trashing the crypto volume when a system panic occurs.

There is no need to pass 'boot -a' at the boot loader when booting this modified kernel. The system will behave just like an unencrypted system as long as the key disk is present. When the key disk is not present, the kernel will panic with a message like "root device (sd2a) not found".

When using installation media to upgrade the system, the crypto volume will be automatically assembled at boot. The update can be performed as usual, using the crypto volume as the root disk.

Unless extra device nodes are created before the upgrade script is run, there might errors reported by installboot, such as: installboot: open: /dev/rsd1: No such file or directory This error can be ignored.

Before rebooting the system, the new boot code and kernel need to be installed on the boot partition. For example (disk names should be adjusted as necessary):

CONGRATULATIONS! Your OpenBSD update has been successfully completed!
To boot the new system, enter 'reboot' at the command prompt.

# /usr/mdec/installboot -v /usr/mdec/boot /usr/mdec/biosboot sd2
# mount /dev/sd0a /mnt2
# rm -f /mnt2/obsd
# ln /mnt2/bsd /mnt2/obsd
# cp /mnt/bsd /mnt2/nbsd
# mv /mnt2/nbsd /mnt2/bsd
# rm -f /mnt2/obsd.rd
# ln /mnt2/bsd.rd /mnt2/obsd.rd
# cp /mnt/bsd.rd /mnt2/nbsd.rd
# mv /mnt2/nbsd.rd /mnt2/bsd.rd
# reboot

When rebooting into the upgraded system, boot -a must be typed at the boot prompt or set in boot.conf to cause the new kernel to prompt for the root partition.

Similarly, when upgrading the system from source, the new kernel must be compiled and installed on the boot partition before rebooting and building userland.

To avoid potential mix-ups when the system is booted with a different number of disks inserted, it is recommended to specify partitions within the crypto volume in /etc/fstab via disklabel UIDs. This way, partitions will still be mounted correctly if the name of the crypto volume changes.

The disklabel UID can be obtained with disklabel(8):

disklabel sd2 | grep duid

Below is an example /etc/fstab file that uses a disklabel UID for partitions in the crypto volume:

5ffb14139e1548be.a / ffs rw 1 1
5ffb14139e1548be.d /var ffs rw,nodev,nosuid,softdep 1 2
5ffb14139e1548be.e /usr ffs rw,nodev,softdep 1 2
5ffb14139e1548be.f /usr/X11R6 ffs rw,nodev,softdep 1 2
5ffb14139e1548be.g /usr/local ffs rw,nodev,softdep 1 2
5ffb14139e1548be.h /usr/src ffs rw,nodev,nosuid,softdep 1 2
5ffb14139e1548be.i /usr/obj ffs rw,nodev,nosuid,softdep 1 2
5ffb14139e1548be.j /usr/ports ffs rw,nodev,nosuid,softdep 1 2
5ffb14139e1548be.k /usr/xenocara ffs rw,nodev,nosuid,softdep 1 2
5ffb14139e1548be.l /usr/xobj ffs rw,nodev,nosuid,softdep 1 2
5ffb14139e1548be.m /tmp ffs rw,nodev,nosuid,softdep 1 2
5ffb14139e1548be.n /home ffs rw,nodev,nosuid,softdep 1 2