Contributed by merdely on from the fscking-softraid-crypto dept.
Mark Uemura (mtu@), back by popular demand, has been gracious enough to share his experiences at c2k8, The OpenBSD General Hackathon in Edmonton. Here is his first summary:
Mark's continues below.
In the n2k8 series, I mentioned that there are many individuals that I follow on misc@. They are like heroes on the OpenBSD stage and people that I admire for what they have done in OpenBSD and for the community. Otto Moerbeek (otto@) is one of those guys. Theo has spoken very highly of Otto in the past and ever since then, I've paid closer attention to what Otto has to say on misc@ and what he has done in OpenBSD. His most recent work on ffs2 support in OpenBSD is really impressive. With drives getting bigger and cheaper, large partitions are becoming that much more important. I'm sure that I can speak for everyone and say, "We really appreciate your work!".
Here is what this kind and polite gentleman had to say about this work at the c2k8 hackathon:
malloc(3). Actually, most of the work on malloc has been done in the last two years: first lots of studying of the current malloc and thinking how to improve it. Then, spread over some months, I did implement it at home and here in Edmonton I worked on the final pieces.
The problem with current malloc is that its main datastructure (the page directory) originally was designed to deal with contiguous memory as obtained by sbrk(2). Some years ago, it was adapted to use mmap(2), which on OpenBSD returns pages with random addresses. The page directory does not handle that well: its overhead, both in time and memory usage, for random page addresses is pretty bad.
The new malloc implementation (omalloc for short) uses a page directory which does not degrade when handling random page addresses. It also handles various edge cases better: it never needs memory to free memory. It also randomizes sub-page sized allocations by default; the current malloc only does that when the 'G" options is used. I knew there was room for improvement, but I was pretty amazed that omalloc improves performance by tens of percents in some cases. The code is also quite a bit smaller and cleaner, in my opinion.
bbq, beer and being able to work full time on things I like 100%) is that these suggestions could be discussed face to face, immediately, and so I was able to finish omalloc in far less time than I expected. Of course omalloc needs more testing, if you are running -current (and only -current!), you can help by giving the latest version a spin.
The other thing I worked on is fsck_ffs(8) for really large filesystems. fsck_ffs needs to store a few bytes of data about each inode in the filesystem being checked. Depending on the limitations of the platform, this means that filesystems having lots of inodes cannot be checked. Using a diff originally started by Dale Rahn (drahn@), I could reduce memory consumption by 20%. This diff has been committed. I am also working on a diff to actually not store all inode data in memory simultaneously; but this work is not yet ready. Although, in a certain test setup, I could reduce memory usage by a factor of 4 -- and there's more to be gained.
One of the small things I worked on with Ken Westerback (krw@) is disklabel(8). We are taking steps to make it present less confusing data in interactive editing mode to make it more understandable. This is also work in progress: it needs more tweaking before it hits the tree.
The hackathon has been fun and fruitful for me!
bio(4)) and the sensors framework. Marco doesn't stop there. Now we have softraid(4), which is also really exciting; and more recently, support for softraid with crypto support! Oh, and that's not all!
He has done quite a lot of work with Jordan Hargrave (jordan@) and others to make ACPI work on OpenBSD for those poor souls that do not have an X40 and can't suspend. ;-) I brought along my evil ACPI machine, a Sony UX series computer, so that Marco could iron out some more issues with ACPI. I thought that it would eventually be a potential Zaurus replacement. Theo bought two of them last year to give to developers mainly because of the ACPI. Needless to say, it is pretty nicely supported now thanks to Marco and Jordan.
Here is what Marco had to say about his time at the c2k8 hackathon:
softraid crypto implementation while djm@ added a new crypto algorithm that should work a whole lot better for block devices (AES-XTS). hshoexer@ pushed me to finally implement the delete softraid volume and thanks to that activity, I found an ancient bug that had been plaguing us (disks are shut down normally yet, they complain that they have unclean metadata upon reboot).
The other large project I started working on is rewriting the softraid metadata from scratch because we want to be able to deal with foreign metadata. Since it needs a rewrite, djm@ and I completely rewrote the softraid on-disk format. I am dealing with the fallout of that which is quite the task. This will keep me busy for weeks to come.
(c2k8 hackathon summary to be continued)
Thanks to Mark, Otto and Marco for sharing their experiences at c2k8. We plan to publish several more installments from c2k8 over the next week or so. Please show your appreciation by donating, buying a CD set or convincing the company you work for, that benefits from OpenBSD development, to contribute to the project.
(Comments are closed)