OpenBSD adds boot(8) support for keydisk-based softraid crypto volumes

On i386 and amd64 boot(8) support has been added for keydisk-based softraid crypto volumes. Undeadly editor Sean Cody (sean<at>tinfoilhat<dot>ca) did some testing and wrote in to tell us how to use this feature.

CVSROOT:	/cvs
Module name:	src
Changes by:	stsp@cvs.openbsd.org	2013/10/20 07:25:21

Modified files:
	sys/arch/amd64/stand/boot: conf.c 
	sys/arch/amd64/stand/libsa: softraid.c 
	sys/arch/i386/stand/boot: conf.c 
	sys/arch/i386/stand/cdboot: conf.c 
	sys/arch/i386/stand/libsa: softraid.c 
	sys/arch/i386/stand/pxeboot: conf.c 

Log message:
Add i386/amd64 boot(8) support for keydisk-based softraid crypto volumes.

So far, only passphrase-based crypto volumes were bootable. Full disk
encryption with keydisks required a non-crypto partition to load the
kernel.

The bootloader now scans all BIOS-visible disks for RAID partitions and
automatically associates keydisk partitions with their crypto volume.
Attempting to boot from a volume without its keydisk currently results
in a passphrase prompt (this might be changed in the future).

There is no need to re-create existing volumes. Moving the root
partition onto the crypto disk and running installboot(8) is all that's
needed.

help & ok jsing

Sean wrote in with the following:

Using keydisk based crypto volumes is very easy to setup and fun. Adding boot support makes using keydisks less of a pain to setup since your entire root volume can now be encrypted and deciphered by the key on an external volume/disk (like a USB or SD card). Prior to this patch the way to get this to work was to setup a root volume containing only the kernel that was bootable and then do the rest of the install on the encrypted volume.

There have been a few articles on how to do this in the past such as a blog post by one Ryan Kavanagh and another blog post by Bryan Vyhmeister. One thing slightly amiss with both articles is how they give some slightly awkward advice on how to handle swap slices. OpenBSD has encrypted the swap partition by default since 2005, but not putting the swap slice in the crypto volume is well, inadvisable. The result of putting swap on the crypto volume is a performance penalty caused by encrypting twice. While I understand and respect the idea of keeping swap outside of the crypto volume, doing this can be problematic for how the kernel finds and saves crash core dumps on warm reboots with savecore(8).

By default, the kernel expects crash core dumps to be written to swap on the 'b' slice of the boot volume, and if found, savecore(8) is called from rc(8) to write the core dump to a file in /var/crash on reboot. This isn't a savecore(8) specific option, but instead, it's a kernel option, so changing the dump device requires modifying the kernel options and rebuilding. Since running GENERIC is always advised, running a custom kernel for a single feature seems a bit excessive. If you are using the GENERIC kernel and mistakenly use the 'b' slice of the boot volume for something else like /usr or /home, you might be ok since it is not defined as a swap partition in /etc/fstab, but you have lost the ability of the kernel to record crash state and therefore lost the ability to debug a kernel panics. My recommendation is to keep a swap volume on the host disk AND one on the crypto volume, then in /etc/rc.local remove the crypto volume's swap slice from the active swap set. This gives you the best of both worlds at the cost of a bit of disk space.

$ grep '^swapctl' /etc/rc.local
swapctl -d /dev/sd0b

Since /etc/rc runs before /etc/rc.local, the 'b' slice of the boot volume exits as a swap device when it's needed during warm reboots for saving crash dumps to /var/crash with savecore(8). Since you're disabling the 'b' slice on the crypto volume with swapctl(8) afterwards, you no longer suffer the double encryption performance penalty. None the less, the 'b' slice still exists if you need to write an initial crash dump with:

ddb> boot crash

Though you can use the 'dumps on' directive in the configuration file used with config(8) to change the device where crash(8) dumps are saved, the result would be you're running an unsupported, custom kernel. The default (below) uses the 'b' slice as the target device for the 'dumps on' directive:

$ grep '^config' /usr/src/sys/arch/i386/conf/GENERIC
config          bsd     swap generic

Another thing to consider is how you are going to manage your keydisk. Having only one copy of the key is, well, ill advised. If the USB stick fails, your machine is no longer bootable and your data (reasonably) unrecoverable. USB sticks are only slightly more reliable than floppies, and the reliability of floppies is horrible.

A safe way to manage keydisks is to create and test multiple keydisks and also keep a disk image of the key volume in a safe place. This is pretty easy to do if you plan out how you are going to manage your keydisk. For instance I would prefer to have one keydisk drive for all my systems, so I don't have to fumble with a stack of flash drives. I'll create a custom label and use /dev/sd0a for system A, and /dev/sd0d for system B etc. I'll also have another MSDOS partition on the USB stick to store copies of the slices and make it easy to transfer to a trusted backup host. Disk space doesn't matter much for the keydisk slices. You can make them 1MB with no issues, and go as small as 64 sectors (i.e. 16KB). Smaller may be possible but I have not tried personally. From looking at the softraid(8) source for a bit, (specifically softraid.h,softraid_crypto.c), it appears the minimum space is roughly 1KB since you need at least 32 keys times 32 bytes per key (256bits for AES-XTS-256). This is a bit opaque to those like myself unfamiliar with the softraid code base. The point is to not be too skimpy. I would suggest sticking with 1MB as it isn't really that big relative to the available flash media today and it's easier to handle. For example, a keydisk I'm using to play with this feature is a 16GB stick which is laid out like:

$ sudo disklabel -p m sd1
# /dev/rsd1c:
type: SCSI
disk: SCSI disk
label:
duid:
flags:
bytes/sector: 512
sectors/track: 63
tracks/cylinder: 255
sectors/cylinder: 16065
cylinders: 1946
total sectors: 31277056 # total bytes: 15272.0M
boundstart: 0
boundend: 31277056
drivedata: 0

16 partitions:
#                size           offset  fstype [fsize bsize  cpg]
  a:             1.0M         29296882    RAID
  b:             1.0M         29298930    RAID
  c:         15272.0M                0  unused
  i:         14305.1M                2   MSDOS

A clean and safe way to backup and restore the keydisk softraid metadata is to use dd(1) and seek in 8192 bytes so as to avoid the disklabel on the drive.

Backup:

dd bs=8192 skip=1 if=/dev/rsd1a of=backup-keydisk.img

Restore:

dd bs=8192 seek=1 if=backup-keydisk.img of=/dev/rsd1a

To visually check or inspect the keydisk the handy hexdump(1) can be used.

$ sudo hexdump -C /dev/sd1a | head -15  
00000000  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00002000  6d 61 72 63 43 52 41 4d  05 00 00 00 00 00 00 00  |marcCRAM........|
00002010  b9 18 57 73 b4 30 47 eb  b2 71 2e ba b9 05 34 d2  |..Ws.0G..q....4.|
00002020  01 00 00 00 00 00 00 00  01 00 00 00 00 00 00 00  |................|
00002030  fe ff ff ff fe ff ff ff  00 00 00 00 00 00 00 00  |................|
00002040  4f 50 45 4e 42 53 44 00  53 52 20 4b 45 59 44 49  |OPENBSD.SR KEYDI|
00002050  53 4b 00 00 00 00 00 00  30 30 35 00 00 00 00 00  |SK......005.....|
00002060  c6 41 01 52 4c dc 9e 39  fd 2b 7a 89 70 dd 31 e9  |.A.RL..9.+z.p.1.|
00002070  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00002090  01 00 00 00 00 00 00 00  01 00 00 00 00 00 00 00  |................|
000020a0  00 00 00 00 00 00 00 00  43 00 00 00 00 00 00 00  |........C.......|
000020b0  73 64 30 61 00 00 00 00  00 00 00 00 00 00 00 00  |sd1a............|
000020c0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
$ 

Note: All softraid volumes start at 0x2000 (8192) bytes from the beginning of the volume and start with the string 'marcCRAM'. Also note the device ID at 0x20b0, while the softraid driver should manage the devices moving around (ie. detected out of order). Just keep it in mind in case you are having issues (ie. keep the slice arrangement consistent on your keydisks... may not be necessary but I had some issues when playing with it myself.

The final proviso is that your keydisk needs to be discoverable from the BIOS/EFI. If the machine you want to play with can boot from USB then you should be ok.

If you wonder why using a keydisk would be useful, then consider how the keydisk is required to boot the system and can be removed immediately after the encrypted volume is mounted. From a physical security perspective, this is a pretty handy feature assuming you have a UPS and don't mind traveling to the machine location to boot the system. Another neat use is for laptops, as this can give you full disk encryption without having to resort to TrueCrypt or similar. You only need to have the keydisk (USB, SD card etc.) plugged in to boot, so if you are disciplined about ejecting the keydisk, then the laptop data is somewhat more secured if stolen.

Like most use of RAID, a keydisk setup is only as reliable as your ability to recover from it, so make sure to backup what you care about, and of course, practice and validate your procedures and tools.