OpenBSD Journal

Getting OpenBSD running on Raspberry Pi 3

Contributed by pitrh on from the Pi the puffy birdhouse dept.

Ian Darwin writes in about his work deploying the arm64 platform and the Raspberry Pi 3:

So I have this empty white birdhouse-like thing in the yard, open at the front. It was intended to house the wireless remote temperature sensor from a low-cost weather station, which had previously been mounted on a dark-colored wall of the house (reading were really high when the sun reached that side of the house!). But when I put the sensor into the birdhouse, the signal is too weak for the weather station to receive it (the mounting post was put in place by a previous owner of our property, and is set deeply in concrete). So the next plan was to pop in a tiny OpenBSD computer with a uthum(4) temperature sensor and stream the temperature over WiFi.

unbirdhaus
Figure 1. The un-birdhouse, slightly un-finished

The Raspberry Pi computers are interesting in their own way: intending to bring low-cost computing to everybody, they take shortcuts and omit things that you'd expect on a laptop or desktop. They aren't too bright on their own: there's very little smarts in the board compared to the "BIOS" and later firmwares on conventional systems. Some of the "smarts" are only available as binary files. This was part of the reason that our favorite OS never came to the Pi Party for the original rpi, and didn't quite arrive for the rpi2. With the rpi3, though, there is enough availability that our devs were able to make it boot. Some limitations remain, though: if you want to build your own full release, you have to install the dedicated raspberrypi-firmware package from the ports tree. And, the boot disks have to have several extra files on them - this is set up on the install sets, but you should be careful not to mess with these extra files until you know what you're doing!

Here's a more detailed look at these files and the booting process, based on an email from jsg@ and https://wiki.sel4.systems/Hardware/Rpi3

  • ROM in the SoC (System on a Chip) loads file bootcode.bin into the GPU from FAT16/FAT32 (initially on the MicroSD or "uSD")

  • bootcode.bin loads start.elf from FAT16/FAT32 into the GPU (this is ThreadX OS, a tiny OS used to manage the hardware)

  • start.elf parses config.txt on FAT16/FAT32, loads u-boot.bin and the device tree from FAT16/FAT32 into one of the ARM cores and starts the ARM core running

  • u-boot.bin loads the OpenBSD bootloader (BOOTAA64.EFI) from FAT16/FAT32

  • BOOTAA64.EFI loads the OpenBSD kernel from a BSD FFS filesystem

A mini version of the FFS filesystem exists in the miniroot and a larger version is what you will create when running the OpenBSD installer.

Thus: This article tells you how to install the current snapshot version of OpenBSD on the Raspberry Pi 3. It's aimed at people that have already installed OpenBSD on their laptop, desktop, server, or other computer a few times.

So we don't cover the basics of running the OpenBSD installer - this is not an ideal first platform to install on.

But wait! Before you read on, please note that, as of April 1, 2017, this platform boots up but is not yet ready for prime time:

  • there's no driver for SD/MMC but that's the only thing the hardware can level-0 boot from, so you need both the uSD card and a USB disk, at least while getting started;

  • there is no support for the built-in WiFi (a Broadcom BCM43438 SDIO 802.11), so you have to use wired Ethernet or a USB WiFi dongle (for my project an old MSI that shows up as ural(4) seems to work fine);

  • the HDMI driver isn't used by the kernel (if a monitor is plugged in uBoot will display its messages there), so you need to set up cu with a 3V serial cable, at least for initial setup.

  • the ports tree isn't ready to cope with the base compiler being clang yet, so packages are "a thing of the future"

These limitations will presumably be fixed in due course. The hardworking OpenBSD developers on this platform have done some great work getting things going (thanks guys!) and will no doubt continue to do so. As for when, and in what order, progress will come: 'Things happen when they happen.'

All that said, here's how to proceed, assuming you have OpenBSD up and running on a more conventional desktop or laptop. The basic idea is that we'll boot from a mini-root filesystem on the MicroSD card and install OpenBSD onto a USB-connected disk.

But wait - there's more! The "USB disk" can be a USB thumb drive, though they're generally slower than a "real" disk. My first forays used a Kingston DTSE9, the hardy little steel-cased version of the popular DataTraveler line. I was able to do the install, and boot it, once (when I captured the dmesg output shown below). After that, it failed - the boot process hung with the ever-unpopular "scanning usb for storage devices..." message. I tried the whole thing again with a second DTSE9, and with a 32GB plastic-cased DataTraveler. Same results. After considerable wasted time, I found a post on RPI's own site which dates back to the early days of the PI 3, in which they admit that they took shortcuts in developing the firmware, and it just can't be made to work with the Kingston DataTraveler! Not having any of the "approved" devices, and not living around the corner from a computer store, I switched to a Sabrent USB dock with a 320GB Western Digital disk, and it's been rock solid. Too big and energy-hungry for the final project, but enough to show that the rpi3 can be solid with the right (solid-state) disk. And fast enough to build a few simple ports - though a lot will not build yet. I then found and installed OpenBSD onto a "PNY" brand thumb drive and found it solid - in fact I populated it by dd'ing from one of the DataTraveller drives, so they're not at fault.

rpi3
Figure 2. The Raspberry Pi 3 with (clockwise from upper left) serial debug cable, MSI ural(4), uthum(4) underneath, PNY usb drive, Ethernet, power (bottom)

So here's what you have to do:

  • Acquire a 3.3V "FTDI" USB-to-serial cable like this one (this may take a few days, that gives you time to read the instructions in the following steps): https://www.adafruit.com/products/954. Note that some cables have the 4 or 5 wires baked into a single inline connector; you instead want the one that has them separated out, as in the photo. Note also that this is the same cable you'd want for the same purpose to set up, e.g., a BeagleBone Black.

  • Read the latest notes at https://www.openbsd.org/arm64.html

  • Consider reading the official installation instructions at https://ftp.openbsd.org/pub/OpenBSD/snapshots/arm64/INSTALL.arm64 but they are somewhat generic to various 64-bit ARM systems

  • Download the snapshots from https://ftp.openbsd.org/pub/OpenBSD/snapshots/arm64/

  • Verify the downloads with signify, replacing the XX version with two digits, e.g, for OpenBSD 6.1, use '61'
    	signify -C -p /etc/signify/openbsd-XX-base.pub -x SHA256.sig
    

  • On your computer, insert a uSD card either into a USB-based microSD-card reader, or into an adapter which you then insert into your computer's SD slot or similar. This card can be almost any capacity, as the miniroot is under 20MB.

    Note very carefully what SD drive this creates, by looking in the console output or in dmesg. I get something like:

    	scsibus4 at sdmmc0: 2 targets, initiator 0
    	sd1 at scsibus4 targ 1 lun 0: <SD/MMC, USDU1, 0020> SCSI2 0/direct removable
    	sd1: 15343MB, 512 bytes/sector, 31422464 sectors
    

    On that computer, sd0 is the real hard drive that I boot from. If I were to make the error of typing the wrong disk number in the next few steps, it would completely destroy my working system, so when I say "double check", just do it, ok?

    The name "sd1" tells us it's the second SD drive (they number from zero, of course). Double check the sd drive number, then copy the miniroot filesystem to the SD card (check it again before you press Return). On OpenBSD the "c" partition of a hard disk refers to the entire disk, so we use "rsd1c".

    	doas dd bs=1m if=miniroot61.fs of=/dev/rsd1c
    

  • Now remove the uSD card from your computer and insert it into the uSD slot on the back of the RPI3 card.

  • connect the serial cable between your computer and the rPI3. Use this diagram: https://developer.android.com/things/images/raspberrypi-console.png

  • Plug a USB charger or the RPI3 official power supply into the MicroUSB charging slot (avoid the temptation to run this off your laptop's USB port; for reliable operation, rpi3 needs more juice than most USB ports give). Power up! The red LED should come on. You should see some chatter from u-boot, the low-level boot process.

  • Plug the USB disk into the PI3. I suggest you use the port nearest the Wired Ethernet jack, in case you plug in a second USB drive containing the install sets (this will make your install target sd0 and your sets drive sd1).

  • Eventually OpenBSD should boot into the familiar "installer/shell" choice. Select Install. Fill in the first few choices as you wish. The disk should show up as sd0 this time (AS OF 2017-04-04; this may change in future) as there is no driver for the SD/MMC so the MicroSD card - even though we're booted from it - isn't accessible. The kernel only knows it was booted from rd0, an in-memory "ramdisk".

  • From here on, follow the normal install steps as you've done for other systems including your laptops, desktops, servers. You might as well install all the sets.

This is long, but you've seen it before, right? I don't need to repeat all that here.

I used the default partitioning - whole disk and auto-layout. Note that the MS-DOS partition "suggested" by auto-layout is required for booting!

When it asks you to reboot, do so, but do not remove the MicroSD card; that is still needed to boot. As an aside, the article cited above about the problems with some USB flash drives also mentions an alternate configuration mechanism that involves booting once from the uSD card with a configuration file change that permanently enables USB booting; you can read about that with a web search for program_usb_boot_mode. If you're brave enough to follow that path, you can then remove the MicroSD card to boot. I haven't done this yet.

When the rpi3 reboots, we want to change the configuration on the MicroSD card to enable USB booting. During the reboot, you're given a few seconds to type any character to prevent it from auto-booting the uSD card. Go ahead and type a character this time, you character you!

U-Boot 2017.03 (Apr 01 2017 - 04:27:09 -0600)

DRAM:  944 MiB
RPI 3 Model B (0xa22082)
MMC:   bcm2835_sdhci: 0
reading uboot.env
In:    serial
Out:   lcd
Err:   lcd
Net:   Net Initialization Skipped
No ethernet found.
starting USB...
USB0:   Core Release: 2.80a
scanning bus 0 for devices... 5 USB Device(s) found
       scanning usb for storage devices... 1 Storage Device(s) found
       scanning usb for ethernet devices... 1 Ethernet Device(s) found

Hit any key to stop autoboot: 2 1 0

U-Boot>

You should now be in Uboot, identified by the "U-Boot>" prompt. Let's tell the PI to boot automatically off the new USB drive (if it's present, else fall back to MicroSD, PXE or DHCP) in future.

setenv boot_targets usb0 mmc0 pxe dhcp
saveenv
boot

If you're cautious, you can omit the saveenv the first time. The saveenv is required to save your changed command across reboots. Note that saveenv will probably give a couple of warnings about invalid filesystem entries; these didn't seem to cause any grief on my system.

This time it should boot up into your new OpenBSD system!

.uboot and dmesg from booting after installation

USB device 0:
    Device 0: Vendor: Kingston Rev: PMAP Prod: DataTraveler 2.0
            Type: Removable Hard Disk
            Capacity: 7446.0 MB = 7.2 GB (15249408 x 512)
... is now current device
Scanning usb 0:1...
Found EFI removable media binary efi/boot/bootaa64.efi
reading efi/boot/bootaa64.efi
75737 bytes read in 76 ms (972.7 KiB/s)
## Starting EFI application at 01000000 ...
Scanning disks on usb...
Scanning disks on mmc...
MMC Device 1 not found
MMC Device 2 not found
MMC Device 3 not found
Found 5 disks
>> OpenBSD/arm64 BOOTAA64 0.2
boot>
booting sd0a:/bsd: 3734688+568700+500888+672784 [86+438360+232975]=0x7c2c68
[ using 672704 bytes of bsd ELF symbol table ]
Copyright (c) 1982, 1986, 1989, 1991, 1993
        The Regents of the University of California.  All rights reserved.
Copyright (c) 1995-2017 OpenBSD. All rights reserved. https://www.OpenBSD.org

OpenBSD 6.1 (GENERIC) #0: Sun Apr  2 17:05:03 AEST 2017
    jsg@arm64.jsg.id.au:/usr/src/sys/arch/arm64/compile/GENERIC
real mem  = 989855744 (944MB)
avail mem = 931606528 (888MB)
mainbus0 at root: Raspberry Pi 3 Model B Rev 1.2
simplefb0 at mainbus0: 1024x768
wsdisplay0 at simplefb0 mux 1
wsdisplay0: screen 0 added (std, vt100 emulation)
simplebus0 at mainbus0: "soc"
bcmintc0 at simplebus0
bcmdog0 at simplebus0
pluart0 at simplebus0
com0 at simplebus0: ns16550, no workingcom0: console
dwctwo0 at simplebus0
agtimer0 at simplebus0: tick rate 19200 KHz
simplebus1 at mainbus0: "clocks"
usb0 at dwctwo0: USB revision 2.0
uhub0 at usb0 configuration 1 interface 0 "Broadcom DWC2 root hub" rev 2.00/1.00 addr 1
uhub1 at uhub0 port 1 configuration 1 interface 0 "Standard Microsystems product 0x9514" rev 2.00/2.00 addr 2
smsc0 at uhub1 port 1 configuration 1 interface 0 "Standard Microsystems SMSC9512/14" rev 2.00/2.00 addr 3
smsc0: address b8:27:eb:xx:xx:xx
ukphy0 at smsc0 phy 1: Generic IEEE 802.3u media interface, rev. 3: OUI 0x0001f0, model 0x000c
umass0 at uhub1 port 3 configuration 1 interface 0 "Kingston DataTraveler 2.0" rev 2.00/1.10 addr 4
umass0: using SCSI over Bulk-Only
scsibus0 at umass0: 2 targets, initiator 0
sd0 at scsibus0 targ 1 lun 0: <Kingston, DataTraveler 2.0, PMAP> SCSI4 0/direct removable serial....
sd0: 7446MB, 512 bytes/sector, 15249408 sectors
uhidev0 at uhub1 port 5 configuration 1 interface 0 "Ten X Technology, Inc. TEMPer sensor" rev 1.10/1.50 addr 5
uhidev0: iclass 3/1
uthum0 at uhidev0
uhidev1 at uhub1 port 5 configuration 1 interface 1 "Ten X Technology, Inc. TEMPer sensor" rev 1.10/1.50 addr 5
uhidev1: iclass 3/0
uthum1 at uhidev1
uthum1: type ds75/12bit (temperature)
vscsi0 at root
scsibus1 at vscsi0: 256 targets
softraid0 at root
scsibus2 at softraid0: 256 targets
bootfile: sd0a:/bsd
boot device: sd0
root on sd0a (690c6f679c7e7768.a) swap on sd0b dump on sd0b
WARNING: CHECK AND RESET THE DATE!
Automatic boot in progress: starting file system checks.
/dev/sd0a (690c6f679c7e7768.a): file system is clean; not checking
/dev/sd0l (690c6f679c7e7768.l): file system is clean; not checking
/dev/sd0d (690c6f679c7e7768.d): file system is clean; not checking
/dev/sd0f (690c6f679c7e7768.f): file system is clean; not checking
/dev/sd0g (690c6f679c7e7768.g): file system is clean; not checking
/dev/sd0h (690c6f679c7e7768.h): file system is clean; not checking
setting tty flags
pf enabled
starting network
DHCPREQUEST on smsc0 to 255.255.255.255
DHCPACK from 192.168.1.254 (00:0d:b9:41:68:4c)
bound to 192.168.1.173 -- renewal in 43198 seconds.
reordering libraries: done.
starting early daemons: syslogd ntpd.
starting RPC daemons:.
savecore: no core dump
checking quotas: done.
clearing /tmp
kern.securelevel: 0 -> 1
creating runtime link editor directory cache.
preserving editor files.
starting network daemons: sshd.
starting local daemons: cron.
Tue Apr  4 12:54:38 EDT 2017

OpenBSD/arm64 (darpie.darwinsys.com) (console)

login:

When it starts booting, make sure it says GENERIC and not RAMDISK; the latter would mean it failed to find your USB disk.

You'll want to log in and make a few changes. First, since the PI doesn't have a real-time clock (RTC), you need to set the date each time you boot. You also probably don't need certain daemons on a tiny system, but that will depend on what you're planning to use it for. Quick way: edit /etc/rc.conf.local and set at least the first of these:

ntpd_flags=-s
pflogd_flags=NO
smtpd_flags=NO
sndiod_flags=NO

The -s tells NTPD to set the time immediately when you boot up the system.

Note that most "conventional" USB devices are supported by the GENERIC kernel; you may have noticed my uthum temperature sensor for the "birdhouse" temperature monitoring in the dmesg output. Maybe I'll do another article on that when it's all working.

And again, remember that this is beta stuff at this point, so there may be some flakiness. Be patient, both with the Pi and with the developers. Good things come to those who wait. Better things come to those who help out.

P.S. Thanks to jsg@ for feedback on a draft of this article

(Comments are closed)


Comments
  1. By Andrew Chambers (122.56.24.194) andrewchamberss@gmail.com on acha.ninja

    Thank you for this post and everyone else who has put in the hard work. I have a similar project that currently runs linux on an rpi3 (which I am unhappy about) which I can't wait to port to OpenBSD.

  2. By kraileth (212.77.224.251) kraileth@elderlinux.org on http://www.elderlinux.org

    Great post, thanks for sharing that info! It's always great to read about this kind of progress. :) And of course thanks to everybody involved in making this possible in the fist place!

  3. By Ian Darwin (ian) ian@darwinsys.com on http://www.darwinsys.com/

    Thanks for the comments, but I just tried a few things and wrote down what I did and what worked. The kernel guys deserve all the credit for what you see here!

  4. By sthen (82.68.199.143) on

    Just so people are aware: OpenBSD on this platform is under heavy development - please use snapshots and keep them updated if you're trying it.

  5. By Anonymous Coward (68.151.50.251) on

    Very cool, I can't wait for EspressoBin support next!

  6. By Anonymous Coward (92.221.146.192) on

    Thanks for writing this. I followed the guide,and made it trough.
    Any idea when multicore cpu will be supported.
    I find no packages, will compiling work?

    Comments
    1. By sthen (82.68.199.130) on

      > Thanks for writing this. I followed the guide,and made it trough.
      > Any idea when multicore cpu will be supported.

      One step at a time... When it's ready.

      > I find no packages, will compiling work?

      There are ~500 in snapshots/packages now, more will follow. Some things will build straight-off, some won't.

      Comments
      1. By Anonymous Coward (82.68.199.130) on

        > > Thanks for writing this. I followed the guide,and made it trough.
        > > Any idea when multicore cpu will be supported.
        >
        > One step at a time... When it's ready.
        >
        > > I find no packages, will compiling work?
        >
        > There are ~500 in snapshots/packages now, more will follow. Some things will build straight-off, some won't.

        Now 6186 packages. I'm rebuilding a few which have since been fixed and to catch some random failures but as a first full ports build on a new architecture this isn't bad at all (i386 has 9698).

        Comments
        1. By sthen (82.68.199.130) on

          > > > Thanks for writing this. I followed the guide,and made it trough.
          > > > Any idea when multicore cpu will be supported.
          > >
          > > One step at a time... When it's ready.
          > >
          > > > I find no packages, will compiling work?
          > >
          > > There are ~500 in snapshots/packages now, more will follow. Some things will build straight-off, some won't.
          >
          > Now 6186 packages. I'm rebuilding a few which have since been fixed and to catch some random failures but as a first full ports build on a new architecture this isn't bad at all (i386 has 9698).

          [... and no, this wasn't done on an rpi3 but, as always with OpenBSD, it was a native build :)]

          Comments
          1. By sthen (82.68.199.130) on

            > [... and no, this wasn't done on an rpi3 but, as always with OpenBSD, it was a native build :)]

            ... and up to 7243 now.

            If you're interested, they're currently being built on a SoftIron OverDrive 1000, which boots like this (asciinema).

    2. By Ian Darwin (ian) on https://darwinsys.com/

      > Thanks for writing this. I followed the guide, and made it through.

      Glad to help.

      > Any idea when multicore cpu will be supported.
      > I find no packages, will compiling work?

      Like Stuart said, some packages will build. But I would only build seriously with a "real" hard disk or SSD, not on a USB thumb drive :-) For example, "make package" in devel/git took a few hours to build all its basic dependencies and the package (as always, be sure your snapshot and your ports tree are fully up to date!!).

      Both MP and WiFi are in the queue (there's even a datasheet for the WiFi!), but the people doing the work have other things in their lives and there is no way they are going to commit to having major features like this working by a certain time, nor to give people false hope.

  7. By Ian Darwin (ian) ian+udderly@darwinsys.com on http://www.darwinsys.com/

    the article cited above about the problems with some USB flash drives also mentions an alternate configuration mechanism that involves booting once from the uSD card with a configuration file change that permanently enables USB booting; you can read about that with a web search for program_usb_boot_mode. If you're brave enough to follow that path, you can then remove the MicroSD card to boot. I haven't done this yet.

    So I finally did this, editing boot/config.txt to add the magic line, and rebooted. The sky did not fall.

    Now my RPI3 will boot off the PNY USB stick without the uSD card being present.

    However it will not boot off my USB SATA hard drive, but I'm guessing it's some kind of drive compatibility issue. I have one WD3200AAKS SATA drive described in the article, and two SATA-USB docks, a Sabrent and a no-name (SATA+IDE); In the former it gives various diagnostics; on the latter it doesn't even start UBoot. Either one will boot up the WD disk fine if I put the uSD card back in, but that kind of defeats the point of this part of the exercise.

    At any rate, it seems like the rpi3+OpenBSD+PNY USB stick combo will work for the birdhouse application once I get some solar cells and a battery to power it.

Credits

Copyright © - Daniel Hartmeier. All rights reserved. Articles and comments are copyright their respective authors, submission implies license to publish on this web site. Contents of the archive prior to as well as images and HTML templates were copied from the fabulous original deadly.org with Jose's and Jim's kind permission. This journal runs as CGI with httpd(8) on OpenBSD, the source code is BSD licensed. undeadly \Un*dead"ly\, a. Not subject to death; immortal. [Obs.]