N2K8 Hackathon Summary Part 4

Mark Uemura (mtu@) continues his recap of the Network Hackathon with Part 4 of the series:

Network Hackathon (Part 4) - May 5-10, 2008, Ito, Japan At AsiaBSDcon 2008, Claudio Jeker (claudio@) gave a fascinating overview of OpenBSD's network stack internals (paper, slides). After the talk, I couldn't think of anything closer to brain surgery than having to work with OpenBSD's network stack. If you have a look at the slides and paper, you will have a better understanding of what I mean.

More from Part 4 (with lots of pictures) below.

During the conference, Claudio did mention to me his desire to start working on MPLS sometime soon. After the talk, I realised that this was no trivial task. Yet, it is amazing what happens when you have a few key networking developers in the same room for a week :-) Another big surprise that came out of the hackathon was MPLS support in OpenBSD!

Here is what Claudio had to say about his work:

During n2k8, my main target was to reduce the count of M's in my local trees. The other big time consuming thing was all the MPLS work and especially discussions I had with Esben Norby (norby@) and Pierre-Yves Ritschard (pyr@). MPLS support made a major step forward: it is possible to inject traffic into the MPLS cloud with pyr@'s mpe(4) interface. I cleaned up the forwarding code a bit more so it was possible to ping two OpenBSD hosts via MPLS at n2k8. During all this hacking and the incredible work by norby@, who started with looking at LDP, we realized that our current approach to MPLS is incompatible with the way LDP works. Having the main MPLS developers around, it was easy to discuss the further steps in the onsen: throwing ideas back and forth until a doable solution crystallized.

Apart from the MPLS work, the biggest thing I committed was the routing priorities changes I worked on for about almost 2 years now. The idea for routing priorities came up at the r2k6 hackathon as a solution to the synchronization problem we have with all the userland routing daemons. Currently all userland routing daemons do a more or less good job at keeping the kernel routing table in a consistent state. There are a few scenarios where this currently fails. In particular, a conflict arises if more then one routing daemon tries to include the same route into the kernel. With routing priorities this is now solved in the kernel. Every userland daemon gets a particular priority assigned which is used for all prefixes added to the kernel table. In case of a conflict the kernel will sort the entries depending on the priority so userland daemons don't need to keep track of all changes anymore. While the kernel part is in, no userland daemon uses these priorities -- they just set the priority now -- but in the end, most of the routing table code needs to be rewritten.

Last but not least, I removed routed from OpenBSD. routed is now replaced with ripd(8), which is a nicely privilege separated daemon.

Interestingly, almost no IPv6 related changes went into the tree during n2k8, though we were in Japan. It is sad that no one of the IPv6 enthusiasts are willing to help cleaning and maintaining the IPv6 network stack. It is necessary to have a strong team of people in that area because more and more security and reliability related issues are being found. But as we know, security considerations are not covered by the standard.

---

For some reason, at the evening dinners there was a tendency to pick on the rather large contingent of Germans at the hackathon. Perhaps this was at times provoked by Henning Brauer (henning@), who is also German. Since Claudio is fluent in German, even though he is Swiss, he was also part of that motley crew of Germans. I'm being facetious, of course. Well, Esben Norby, being Danish, was also thrown into this group, for what reason, I don't know.

Esben seemed to really enjoy the Japanese experience; history, culture and food. He was so looking forward to eating sushi that we specifically asked that one of the dinners during the week had to be raw fish. We were expecting this to be served on the last night at the ryokan; however, because of some oversight, we were served something very different. Realising this, I made quick arrangements for another course to be served. After almost an hour later, the sashimi dishes appeared and Esben got to eat his fish. In hindsight, I guess he was expecting sushi rather than sashimi but he got raw fish nevertheless. ;-)

Here is what Esben had to say about his work:

My main objective with this hackathon was startup work on an MPLS stack for OpenBSD. MPLS is getting increasingly more and more popular, especially in the VPN business.

With a lot of help from claudio@ ,we now have a simple but functional MPLS stack. It is possible to make IP packets enter and leave the MPLS domain with the new mpe(4) device from pyr@.

Furthermore, I spent quite some time figuring out how to implement a decent LDP daemon for OpenBSD. LDP is Label Distribution Protocol, primarily used for automatically assigning MPLS labels within a MPLS network. I will hopefully start writing actual code for a LDP daemon, shortly after this hackathon.

---

One of the nicest comments that I heard came from Pierre-Yves. On his way back to the airport on his way to Paris, he said, "I am leaving my dream job. I just spent a week working on OpenBSD and taking onsens whenever I wanted. Thank you!". That made my day. :-)

At the hackathon, the technical discussions were enthralling and captivating, but they were just as interesting after hours when the hacking stopped and the beer drinking began. I recall one conversation having to do with cheese. Pierre-Yves couldn't believe that people actually eat cheese made from homogenised milk. Others couldn't believe that people (Frenchman) actually eat stinky cheese. These were priceless moments of laughter and comradery. Hackathons are not just technical events but an opportunity to build relationships, understand each other better and inevitably create stronger bonds. Most importantly, we have fun! "It has to be fun", as Theo would say.

Besides the amazing stuff that Pierre-Yves has done bringing relayd(8) to OpenBSD, he continued to impress me with what he pulled off during the hackathon. Here is what Pierre-Yves had to say about his work:

I originally came to n2k8 to work on relayd to continue our ongoing joint effort with Reyk Flöter (reyk@). I also wanted to spend some time with Esben Norby and Claudio Jeker to help MPLS support.

I started off with a simple bug, an important reliability fix, to relayd which improves its layer 7 asynchronous engine.

I then caught up to date with the current state of the MPLS subsystem and how things should be done in OpenBSD. One of the specific questions I wanted to address is how packets destined for the MPLS stack should enter it. We already had shared some thoughts on the subject with Claudio and Esben and we concluded that a dedicated virtual network interface was the way to go. Since Claudio and Esben were busy implementing label switching and operations, I started to write an interface dedicated to let standard IP traffic enter the MPLS stack.

After a good day of hacking, the interface started to look functional. We then spent an afternoon with Esben integrating the interface with the stack. The following day, at the expense of some hacks, the interface could be used for entering and leaving the MPLS stack! The mpe, which stands for "MPLS Provider Edge", with the help of routed, can be used to let packets enter the stack with a specified MPLS input label. This is a work in progress and some work still needs to be done, like bpf(4) integration to allow setting promiscuous mode on the interface for use with tcpdump(8).

I needed a break before going back to more serious things and tested many diffs that were being sent out, including the very important one by Reyk which enables DSR (Direct Server Return) for relayd. I also brushed up and committed a simple sshd(8) diff which enables the use of the AllowAgentForwarding keyword, much like the AllowTcpForwarding keyword, which prevents normal agent forwarding. This feature can, for instance, prevent agent and associated keys to be available on shared machines.

I then started implementing a relayd feature I think will be useful to many people: on-the-fly HTTP authorization. This feature will allow relayd to authenticate incoming HTTP connections against the bsd_auth(3) mechanism. This is useful for building authenticating transparent proxies, may they be forward proxies or reverse proxies. The diff is ready but still needs a bit of polishing, which I will do in the next few days, so it is not in the tree yet. As far as configuration goes, only one directive was added to the configuration file parser to keep things simple, as always.

Of course besides this, hackathons are hackathons and remain a great way to share our ideas, to ensure we don't follow paths that lead to dead ends, and have fun! On top of that, this hackathon was a cultural - and culinary, I might add - experiment for many of us who were in Japan for the first time. We will make sure to come back for more onsens, sake, unagi and nato (for the adventurous).

         - Pierre-Yves.

(n2k8 hackathon summary to be continued)

Many thanks to Mark for putting these stories together and sharing them with us.