Developer blog: dlg: ahci(4) and atascsi

ahci(4) is a driver supporting hardware conforming to the Advanced Host Controller Interface for SATA. The hardware is becoming more and more available, and soon it may be necessary to have this driver to use newer machines.

I started the driver in the tree about 3 months ago, but I was stuck on get the SATA ports initialised. Since then Chris Pascoe (pascoe@) got involved and he has now got it to the point where it's doing IO. Since it's basically working now on the JMicron SATA controllers it was enabled in GENERIC on i386 and amd64 machines.

Edit: The change is in the tree now, with code to make some of the SATA drivers attach to this driver instead, making some SATA devices appear as sdX instead of wdX. In case this really isn't what you want on your -current boxes, you can always use config(8)/boot -c into UKC> and from there "disable ahci" on the kernel to revert to the old driver. Remember to update your /etc/fstab if this does apply to your machine.

ahci(4) is important for a couple of reasons, the main one I've already covered: there is hardware coming out now which we'd like to support. That isn't a very surprising reason to work on something, and I'm sure it's not something I have to explain. The other reason I wanted to write a driver for ahci is less to do with AHCI hardware itself, and more to do with pciide(4), wdc(4), and wd(4).

pciide(4) is the driver that attaches to basically all the PCI IDE controllers we support, and wdc(4) supports all the other IDE controllers. Actually, pciide(4) and wdc(4) are mixed together a bit so it's really one and a half drivers. The drivers for devices on an ATA bus (wd(4), and atapiscsi(4)) attach directly to wdc(4) and pciide(4).

This is a bit different to the SCSI stuff, where there are maybe 40 to 50 drivers in the tree for all the different SCSI controllers out there (and we don't support all of them yet). We also have the scsibus(4) device sitting between the drivers for the 40ish controllers we have and the drivers for our SCSI devices. It simply provides a layer of abstraction and management between the devices that generate a SCSI command, and those that put the commands on the bus.

Because of this abstraction it is also extremely easy to emulate SCSI on top of devices that don't actually understand SCSI commands. Examples of this would be ami(4), gdt(4) and the sdmmc stuff. None of those devices actually understand SCSI, they just take a SCSI command via the scsibus(4) and translate it into something appropriate to the hardware they sit on top of.

Anyway, back to ATA stuff. The model for pciide/wdc/wd/atapiscsi sort of makes sense since all IDE controllers are fairly similair to each other, and this is reflected by how they work. There's a generic core for the standard IDE controller state machine, but with the option to provide exceptions to the core functionality for specific chipsets.

The problem with this is that the need for exceptions on more modern controllers is becoming the norm, rather than the exception. This causes the IDE drivers to become increasingly more and more complicated to cope with this new hardware. On top of this ATA gear is starting to get some smarter characteristics that look more and more like how SCSI works. Things like Native Command Queuing in newer SATA disks looks suspiciously like the tagged queing stuff SCSI has had for years.

To deal with the extra complication that newer controllers are bringing to pciide, an obvious solution would be to restructure these drivers to provide some level of abstraction. An obvious model to follow would be the one we use for SCSI. There have been discussions for a long time about how to do this. The first idea was to slowly rework the IDE code and get it to a point where we could introduce an atabus(4) between pciide(4)/wdc(4) and wd(4)/atapiscsi(4). This idea sounds good, but in the real world noone was willing to step up and spend the time working with the behemoth that is the pciide/wdc code. It's some very twisty code, and whoever worked on it would have had to modify how it works without breaking it. A lot of people use this hardware (this is an understatement), and potentially breaking their disks is scary.

The next idea (and my favourite for a long time) was to implement atabus(4) as a separate codebase to the existing IDE drivers. As a start it would only support a couple of controllers, but again, that would be separate to the existing code. The reasoning behind this is you can work on something without breaking existing device support. Once it got to a point where it was usable, we could just switch the drivers for that controller from pciide to the new atabus based code.

I eventually realised that this approach would be simply copying how scsibus(4) works and changing the structures that represent the command from SCSI commands to ATA frames. marco@ had been arguing with me saying that scsibus(4) has all the semantics we wanted in a modern pciide replacement, such as the abstraction between the disks and the controllers, and things like tagged queuing and hotplug. We also already emulate SCSI on a lot of devices that actually have no knowledge of SCSI, so why not do the same for IDE?

Thus ahci(4) and the atascsi layer were born.

The language that scsibus(4) talks is encapsulated in a struct called "scsi_xfer". It contains the SCSI command, pointers to the buffers containing the data you want transferred between the computer and the SCSI device, pointers to which driver is responsible for the buffers, timeouts for handing device errors, flags for how the controller should operate on this command, and so on. For devices that don't understand those SCSI commands (eg ami and sdmmc) we translate them in the controllers driver.

If we are only going to have ahci(4) attach a scsibus and do the translation, then leaving the translation in ahci itself would have been ok. However, we intend to do the same thing to other controllers in the future, so it makes sense to split this SCSI to ATA translation layer out, so that's what I did. I implemented an equivilent to a scsi_xfer called an ata_xfer, which is basically the same thing except it carries a representation of an ATA command instead of a SCSI one, and a few more flags for ATA specific things.

This means that IDE drivers can be structured similairly to our SCSI drivers, except instead of attaching a scsibus directly, they hook an instance of the ATA to SCSI translation layer, and instead of chewing on scsi_xfers, they take ata_xfers. The code that is responsible for the attachement of scsibus to an ATA driver, and for translating the two types of xfers, is a couple of files in src/sys/dev/ata called atascsi.c and atascsi.h.

ahci(4) itself is actually pretty boring if you can get over the complexity of dealing with the hardware. All the SCSI hardware I've worked on deals with the init of the bus and devices on the bus for me. I simply init the chip, give it some memory, and start pushing commands down. AHCI is different, the hardware itself is incredibly dumb, which means the driver has to be very smart. It is responsible for initialising the port, the phy, getting the device on the end to wake up, stopping the controller on error and asking the device what went wrong, and so on and so on. I wasn't coping with that too well, so I asked pascoe@ to look into it and he managed to figure it out and then basically finished the driver.

Fundamentally though, ahci(4) works much like our SCSI drivers now, which is the goal we were aiming for. It simply inits the hardware and hooks up with atascsi. That in turn attaches a scsibus(4), and then takes scsi_xfers from the midlayer and emulates a scsi disk by talking ata_xfers to ahci(4). It also passes through ATAPI commands to devices that support them (like SATA cd/dvd drives). This means we've cut out wd, atapiscsi, pciide, and all the wdc code. This all happens in about 1000 lines of code in atapiscsi.

As a side note, linux and solaris do something similair to atascsi too. The linux emul layer (libata) is about 13000 lines.

So that's the story of ahci(4). Well, it was more the story of atascsi and how it let us write ahci(4) like a SCSI driver. I spose you all want to see what it looks like now. This is the JMicron controller on my dev box:

ahci0 at pci4 dev 0 function 0 "JMicron JMB361 IDE/SATA" rev 0x02: AHCI 1.0: irq 11
scsibus0 at ahci0: 32 targets
sd0 at scsibus0 targ 0 lun 0: <ATA, ST3320620AS, 3.AA> SCSI2 0/direct fixed
sd0: 305245MB, 305245 cyl, 64 head, 32 sec, 512 bytes/sec, 625142448 sec total

We are working on supporting other controllers with ahci(4), such