OpenBSD Journal

g2k14: Martin Pelikan on ext4, filesystems in general

Contributed by pitrh on from the files from fishy penguins dept.

Martin Pelikan writes in with this report from g2k14:

My initial plan was to bring our base to a state where LLVM's libcpp could be compiled, giving us C++11 support. After I read up on the latest POSIX locale additions, other developers made it clear that more library version cranks will be necessary in order not to break ports. After the first diff was ready, I set up a base system build to check if it breaks. And then my life has changed...

Few days before the hackathon I decided to reinstall my laptop and put Linux, Windows and OpenBSD next to each other. One of the locale-related articles was left on the Linux partition and I wanted to open it in Austria, which is just full of tunnels without the internet. Our kernel didn't like the ext4; being too lazy to reboot, I decided to take a look at it "if I have a spare moment", which base builds certainly are.

No wonder -- ext4 uses extents, which weren't supported at that time. Quick look at FreeBSD showed they already have read-only support which became more or less functional on my OpenBSD kernel on Wednesday evening. No HTree directory indexes, no 64-bit block numbers, no journal or snapshots or multi-mount protection. But I could finally read the PDF without rebooting, and then also copy a file bigger than 4 GB or open a directory with 50,000 subdirectories in it. Never even looked at filesystem code before, and now a working port in a few hours? Life was great!

The question now was how to integrate it. OpenBSD developers don't like huge diffs for a very good reason. After fixing the inode format and adding new flags, Ted pointed out the ancient rule "whoever touched it last now becomes the maintainer" was valid even here. It was obvious I had to gain more knowledge by reading design papers and other systems' code before I can do valuable and correct commits. Legacy remnants like hard-coded (and wrong) limit on file sizes went first. Parts of the code were pretty badly comprehensible but FreeBSD has managed to split them up somehow. After our code paths looked similar enough, the ext4 extent support hit the tree.

Despite the fact that hackathons' purpose should be to write code, I had to literally learn to work with a subsystem I've only seen very vaguely once before (with FUSE). Because FreeBSD has already had these bits and getting them to work was so easy, this was a very good place to learn how to write filesystem code myself. During the next days I wrote code parsing and following the journal and ended up deliberately breaking my filesystem to observe what should we do about recovering it in seconds and how to order writes to support them in the future. Let's try to continue going down this road.

All this wouldn't have been possible without Phillip Guenther who reviewed my diffs after I kept distracting him from way more important things to do. Ted, Theo, Ken and Bob gave me lots of valuable input and explained how are things supposed to really work. Stephan Sperling gave me access to one of his sparc64 machines and gladly kept the tree up-to-date, so my breakages were relatively few and minor. Discussions about the network stack helped narrowing our focus to a direction and explaining our concerns about other solutions. Mitja must have been very busy with organizing the event so that things would go as well as they did. Well done everyone, thanks!

Now let's hope this filesystem enthusiasm will last longer, because now I need to switch back to GSoC in an area I'm already familiar with. Breaking my filesystem now will hopefully help repairing yours one day...

(Comments are closed)


Comments
  1. By Anonymous Coward (77.109.141.138) on

    awesome! excellent news, and excellent report. thank you!

Latest Articles

Credits

Copyright © - Daniel Hartmeier. All rights reserved. Articles and comments are copyright their respective authors, submission implies license to publish on this web site. Contents of the archive prior to as well as images and HTML templates were copied from the fabulous original deadly.org with Jose's and Jim's kind permission. This journal runs as CGI with httpd(8) on OpenBSD, the source code is BSD licensed. undeadly \Un*dead"ly\, a. Not subject to death; immortal. [Obs.]