OpenBSD Journal

g2k16 Hackathon Report: Florian Obser on httpd, networking, acme-client, and more

Contributed by rueda on from the Let's Rename dept.

Next up in the series of g2k16 hackathon reports is this one from Florian Obser, who writes:

One of the first things to do when attending a hackathon is travel planning.

For me this started a few months back when I got annoyed that getting there would waste a whole day. And I live somewhat close by! Shall I go by train? But how do I figure this out? I could fly, but to where, and then what, need to figure out train options...
While mulling this over I notice that Mark Kettenis (kettenis@) arrives early in the morning by train. So he either has a plane leaving at stupid o'clock in the morning or he travels overnight on a ferry. So I drop him a mail and he confirms that it's a ferry and he sends me a link to the booking page. Neat!

On arrival at Harwich the harbormaster informs me: "It's a shilling to tie up your boat at the dock... and I shall need to know your name." I offer three shillings and we forget about the name.

After what I understand to be a pretty uneventful train ride for UK standards (we were only 1 hour late!) we arrived at the hack room with a few people already busy.

I quickly got the social necessities out of the way (handshakes all around the room), sat down and got busy by committing an already lined up diff: Over the years (sigh, yes it has been this long! Sorry about that!) people have reported that httpd(8) naively and wrongly assumed that the HTTP response header is fully contained in the first fastcgi stdout record. I finally got around to look into this and fix it.

I had some other diffs lined up for the hackathon, namely an update for nsd(8) 4.1.11 which Stuart Henderson (sthen@) quickly OK'ed during the hackathon. At the end of the week nlnetlabs published 4.1.12 with a little reliability fix which did not effect OpenBSD due to the way we build nsd(8). But we pulled in the update anyway. Two nsd updates in a week, whee!

On the topic of name servers, PowerDNS released their spring cleaning / refactoring / rewrite effort in the form of 4.x. These days they test OpenBSD themselves so the code actually compiles without local patches. That makes things easier. As package maintainer my job now only consists of shoving it through the ports infrastructure. With 4.x they switched to c++11. I cargo culted the needed Makefile machinery from another port some time ago, but well... it was cargo culted and looked weird / not quite right. I shelved it for the hackathon, sent a mail to ports@ and within minutes sthen applied the necessary clue bat.

This was a weird hackathon for me. All my previous hackathons followed the tradition of bringing some things that are ready to commit, get that out of the way, have something planned to do during the hackathon, which then never happens and find one thing to work during the hackathon. Not this time. I did bring things, I had things planned, but I was multitasking a lot. So much so that I was live locked at times. But that could be fixed with a pint of bitter...

My main focus of work was on the unification of ping6(8) with ping(8), acme-client(1) and slaacd(8). More on these later.

One of the great things of a hackathon is that everybody is focused on OpenBSD so your questions usually have a very short round trip time.

At every hackathon since I moved sending of router solicitations into the kernel, Christian Weisgerber (naddy@) is pestering me that all the machines are now sending one every 60 seconds. This makes debugging things with tcpdump(8) pretty hard since you get a lot of noise. And if I could please fix this.

Hacking in the kernel is always interesting so I spent some brain cycles on this. However, lunch first! Back from lunch I had a look at the source code, couldn't find anything that was in the way and quickly implemented timeouts depending on the preferred life time (pltime) of the router advertisements.

Sebastian Benoit (benno@) was sitting across from me and his work on log.c unification for our network daemons reminded me of David Gwynne's (dlg@) lerr() etc. family of logging functions. They are shared between identd(8), slowcgi(8), tftp-proxy(8) and tftpd(8). I'm quite fond of them ever since I discovered them for slowcgi(8). Unfortunately they are missing printf-like format attribute annotations. I quickly added them and even found a bug, yay!

Since some time TLS for OpenBSD.org is a thing. It uses a certificate authority that implements the "Automatic Certificate Management Environment (ACME)" protocol. This being OpenBSD we are not too fond of running a client that depends on half of the python ecosystem - especially if it's security critical.

Conveniently Kristaps Dzonsons of mandoc(1) fame already wrote a client that's very close to OpenBSD's base standards (https://kristaps.bsd.lv/acme-client/). It's privsep'ed, chroot(2)'ed and pledge(2)'ed.

Of course the old adage is true: "There are only two hard things in computer science: cache invalidation, naming things, and off-by-one errors." Kristaps originally named his tool after a particular implementation of the ACME protocol. We felt that it is inappropriate for inclusion in base and tried to come up with a different name. Once the dissenting voices were silenced, uhm that is to say convinced, we settled on acme-client(1). Kristaps followed suit and apparently around the same time said certificate authority contacted Kristaps and "suggested" that he should rename his client. (https://github.com/kristapsdz/acme-client/commit/69ae73d0077386d6b49032d30212f592d633d5ed)

With the hard problem solved it was time for dinner. Coming back Theo de Raadt (deraadt@), Benno and I went on a little rampage. I imported it, added a bsd.prog.mk based Makefile, cleaned up the man page, created default directories with mtree(8) and removed some compat code (we like to keep base clean of compat code and add it to the portable release). Benno plowed through the code and fixed whitespace issues. Theo fixed shadowing compile warnings and removed a sandboxing abstraction. pledge(2) calls are now in the normal place like in the rest of the tree. After two hours we were done and it was time to sleep. Meep meep!

The next day Jason McIntyre (jmc@) joined with more man page clean up and Theo Buehler (tb@) pointed out that we imported yet another base64 implementation. He switched it to the libc version.

Unifying ping(8) and ping6(8) is a long running project of mine. Some time ago I merged ping6(8)'s engine into ping(8). It has the advantage that it does considerably less in signal handlers, it just sets flags that the main for(;;) loop handles.

Shortly before the hackathon Todd C. Miller (millert@) pointed out that we still call summary() in a signal handler. The function has to jump through some hoops to be signal safe. He shuffled code around to pull summary() out of the signal handler. Theo (deraadt@) then quickly moved summary() to stdio. With a bit more code shuffling and bringing a bugfix from ping(8) to ping6(8), summary() is now the same in both programs. Whee!

A big blocker of unification was the -n flag in ping(8). And more importantly my inability to give a coherent explanation on why it's stupid and completely unneeded and why you won't see a difference in normal operation. Since I failed to explain this to fellow developers for nearly a year I won't try to do it here. If you are interested go read the diff. It's rev 1.144 of ping.c. So the new ping(8) default is now as if -n had been provided. This makes it possible to make pr_pack() address family (AF) independent.

With that out of the way I could continue to chip away at the differences. Work continues...

Another main area of work for me was the start of slaacd(8). To help Martin Pieuchot (mpi@) keep his sanity the plan is to rip out all router advertisement processing and router solicitation sending code from the kernel and move it to user land. This is part of a much bigger effort by multiple people to handle IPv4 / IPv6 address and DNS resolver configuration. Details about that are out of scope for this report. Using the chroot(2), privdrop, logging and libevent framework laid down by slowcgi(8) I quickly had a daemon, pledge(2)'ed from the beginning, that could parse a router advertisement with all options - including domain search lists! With that working I hooked it up to a routing socket to get a notification when the layer two address of an interface changes (think ifconfig lladdr random) but it turns out that that (no longer?) works and I wandered off into the kernel to see what's going on. At this point the last three standing were kicked out of the hackroom and g2k16 concluded.

Thanks to Gemma Gordon, Anil Madhavapeddy (avsm), the OpenBSD Foundation, the Computer Laboratory Cambridge, and everybody else who made this hackathon possible.

Thanks for the report, Florian!

(Comments are closed)


Credits

Copyright © - Daniel Hartmeier. All rights reserved. Articles and comments are copyright their respective authors, submission implies license to publish on this web site. Contents of the archive prior to as well as images and HTML templates were copied from the fabulous original deadly.org with Jose's and Jim's kind permission. This journal runs as CGI with httpd(8) on OpenBSD, the source code is BSD licensed. undeadly \Un*dead"ly\, a. Not subject to death; immortal. [Obs.]