OpenBSD Journal

OpenBSD goes to Venice

Contributed by dhartmei on from the report-from-the-asylum dept.

Thanks to Ed for his report from OpenCON 2005 in Venice:

What happens when you put a dozen of developers on a little island with their la ptops, power, and an internet connection?

During the first week of November some OpenBSD developers met in a little island in the Venice's laguna to hack on the ports system. This was probably the first ports hackathon and was followed by OpenCON, an european conference fully dedicated to OpenBSD.

Read on for his full story including (work safe ;) pictures.

OpenBSD goes to Venice

What happens when you put a dozen of developers on a little island with their laptops, power, and an internet connection?

During the first week of November some OpenBSD developers met in a little island in the Venice's laguna to hack on the ports system. This was probably the first ports hackathon and was followed by OpenCON, an european conference fully dedicated to OpenBSD.

Starting from Saturday 30th developers travelled to San Servolo, a tiny island previously used as leprosarium, and later as lunatic asylum. It's now a cool place for conferences, art expositions, and students dormitories. During the hackathon it was renamed "The Little Alcatraz". A prison is not so bad when you want to get some code written ;-)

Hackers at work

As usual hackathon are a good opportunity to meet face to face, so that the link among people can become more deep, and also they favor ideas exchange. As Theo de Raadt would later point out, there is also the secret ingredient, namely beer. It seems that meeting face to face, and get drunk together is a great strategy to let developers reveal their most secret ideas for software improvements or new features. And once they made them public, they have to write the code! Ask Henning about something called OpenBGPD...

Hackers at work

So, in the first week of November a little crew of developers worked on the ports system. Here the names list:

  • Aleksander Piotrowski (alek@)
  • Bernd Ahlers (bernd@)
  • Christian Weisgerber (naddy@)
  • David Cathcart (cathcart@)
  • Marc Balmer (mbalmer@)
  • Marc Espie (espie@)
  • Niall O'Higgins (niallo@)
  • Nikolay Sturm (sturm@)
  • Peter Stromberg (wilfried@)
  • Peter Valchev (pvalchev@)

Hackers at work

The conference was planned for Saturday 4th and Sunday 5th, so around Friday the team was joined by Theo de Raadt, Henning Brauer, Uwe Stuehler, and Wim Vandeputte with his portable OpenBSD bazaar.

Some autographs on OpenBSD 3.8 Poster
This poster could be a map for a treasure... do you see the subliminal message?

A funny thing I have to point out is that when the acronym "F.A.Q." is read in italian, it sounds very similar to that english word part of the other acronym "RTFM". So if you are visiting an italian software conference and hear "fu*k... website... fu*k... documentation...", well, they are just talking about the FAQ on the website.

Saturday

There was a single track, so I listened to most talks starting from Saturday afternoon. Theo had a one hour slot, but after a few slides he called Henning Brauer that started talking about PF and OpenBGPD improvements.

As usual there is always a lot of work that happens without any visible change, so users don't see what is happened. This release, for example, comes with a lot of tiny bug-fixes and improvements in PF. Henning explained all the hard work he and Ryan McBride had to do to create an interface abstraction to implement interface groups and support hot-plug interface. By the way, during April 2004 Hackathon they implemented a simple anticongestion feature in PF; packets that don't match any state entry are dropped until the flag that indicates cpu exhaustion is unset.

Also he underlined that OpenBGPD has a strong interaction with the kernel, starting with the routing table to PF. It supports IPv6, and it seems 5-10 times faster than Quagga. This could explain why the three biggest European Internet Exchange Point are planning to deploy a route server with it. There is also a plan to add a restricted socket (/var/www/logs/bgpd.sock) in the Apache chroot, so that it could be accessed by a cgi script via web-browsers.

Finally he listed some "crazy ideas for future releases". They have a lot of ideas... you can checkout the slides directly on the internet, but I would like to underline the fast path through the network stack with the chance to offload some work to specialized hardware, a great rework of PF data structures, and the use of states for other subsystems.

Then Marc Espie talked about the ports infrastructure and its evolution. For years the code used to manage ports and packages was the same used by FreeBSD, with a lot of fixes and hacks. This made things hard, so there was a revolution. Eh? Wait! A revolution in OpenBSD? This is not possible, OpenBSD believes in evolution, not revolution! At this point a voice from the first row:"But you are french, after all". Ok, some smile here :-D

Marc explained us that while the user interface has not changed so much, the tool were completely rewritten using Perl, that comes in the base system, and this let them write compact code. The good thing is that now, this code can be reused using the OO paradigm, without the need to rewrite each time the same functions to open archives, build things and so on.

If you have built some ports recently you have probably noticed that the system build a package, and then installs it. This is exactly the opposite order of FreeBSD's tools. The idea behind this strategy is that, first, packages should always work, and second, we make all the check we can before installing a package. This will be really usefull even when upgrading a package.

Marc Espie
Marc Espie, OpenBSD's Chief Revolution Officer, has been leading the (r)evolution of the ports system

At the moment there are around 500 new ports added every year, and there is some hope to see system updates via ports not too far in the future.

After Marc's talk all the people at the conference, around 50 men and 5 women, took the boat to Venice.

Venice by Night
Around 18 o'clock and everything is dark. You are in a little boat travelling to Venice... everyone hopes that the boat is not called Titanic...

Then we walked in the city tight streets until we reached the Bus stop. Two bus later, we were at the dinner party. There isn't much to say about that, except that that club had a funny policy about peanuts. They give you all the peanuts you want (really!) and you can use them to "interact" with other people. Ok, yes, I'm guilty, we were divided in two long tables, so some peanuts battles took place... Henning and Wim should remember something ;-)

Sunday

Peter Valchev presented us some results of this ports hackathon:

  • updated the port of gettext; it has been out of sync for years because it was used by hundreds of other ports.
  • great work on OpenOffice; Peter wrote hundreds of patches. The new malloc work spotted a lot of off-by-one bugs ("it works on Linux by luck").
  • better interaction between the ports system and the packages archive, that is one system can call the other if needed. You can learn more searching for FETCH_PACKAGES, a feature developed by Nikolay Sturm.
  • some brainstorming about version numbers. Sometime a tool uses multiple different type of notations to mark stable or unstable releases. You can find the discussion on ports@.
  • a bit of auditing of libexpat, a XML parsing library. It is duplicated since it is parts of X11 and in the ports tree too.

Marc Espie showed how to make a simple port, and suggested to try this at home if you don't dare to play with kernel internals. It can give you good results, so that you could choose to start contributing to the project as a ports maintainer. For more details I suggest that you contact the ports@ mailing list.

Later Uwe Stuhler and Niall O'Higgins talked about porting OpenBSD to new platforms [PDF]. It was really interesting, especially if you would like to make some test by yourself with an ancient system you have, but you don't know where to start from. They explained that the first step is to choose a similar architecture already supported, and then start hacking a bit. Obviously everything begin cross-compiling, but as soon as it's possible you should start using native builds. The first goal is booting the kernel with a working serial console, so that you can debug and play with it. Sometime you use a JTAG module, sometime boot(8), that can be launched under Linux and replace it in memory. The initial work is done building and rebuilding bsd.rd, the RAMDISK bundle for obvious reasons.

Once the system is stable enough you can start porting drivers from other architectures or maybe even from other BSD projects. Writing drivers is always a delicate work, and having access to good docs is much better. Uwe and Niall presented the history of the Zaurus port, that came with good docs and so its hardware is now almost completely supported. This good result has been achieved in less than one year thanks to the contribution of large group of developers, that is if you want to see a port being maintained you need to be sure that at least twenty developers have access to hardware and documentation. This is very important, especially if you want daily snapshots and a broad range of ported applications.

Finally, Theo gave his traditional talk on Exploit Mitigation Techniques. He presents it often, but he updates it for every release, so you'll find some new slides about the malloc() work included in OpenBSD 3.8. Talking about W^X, Theo said that some upcoming i386 cpus (VIA, Intel) will feature a NX bit accessible via PAE, but this is not as good as the X bit found on other cpus. Also he doesn't like the idea to implent PAE just to use it, because this would mean supporting two MMU modes in the same architecture and make the pmap module complex; using a compile time option is not acceptable too. So, he is trying the hard way: persuading some venders into implementing a 32bit NX extension.

Talking about privilege revocation and separation, he pointed out that the new Sendmail maintainer has been following some good coding practice, and in fact Sendmail X is very similar to qmail, as a group of small processes that interact. Maybe this means that OpenBSD could import it in a near future... who knows?

Theo de Raadt
Theo de Raadt, OpenBSD's Chief Blowfish Officer

After this last talk, the conference ended. There were some final words by organizers (OpenBEER/OpenGeeks staff), some group photos, and everyone followed his way to home.

Free as free beer!

I hope to see more conference like this one, where people can join them for free (no money was asked to partecipate), and developers have a more active role, hacking a bit with other teammates, doing a bit of advocacy among users, and meeting face to face. As far as I know there was people from more than a dozen of contries, in Europe and outside. This is cool, because it probably means that they spent money on the travel, instead of spending 300-1000 euro just for the conference ticket!

(Comments are closed)


  1. By Antonios Anastasiadis (147.102.51.43) on

    Where can we find marc espie's porting tutorial? I've skimmed quickly through ports@ but haven't found any similar mail.

    1. By Anonymous Coward (213.118.238.25) on

      http://www.openbsd.org/papers/ven05-espie/
      http://www.openbsd.org/ports.html

      1. By guly (82.119.211.78) guly@openbeer.it on

        slide is incoming, check back opencon website

    2. By Marc Espie (62.212.102.210) espie@openbsd.org on

      It's the two ports that were committed as chmlib and xchm... not really a big tutorial, but a quick example of how fast it can take sometimes... about an hour from start to finish, and roughly twenty minutes of presentation. We are going to have more documentation in the near future...

  2. By Venture37 (217.22.94.66) venture37@hotmail.com on www.geeklan.co.uk

    WoW, great read, sounds like fun was had by all, great news about the large ISP's implementing OpenBGP :)

    1. By dsp (62.1.231.170) on http://dcs.ics.forth.gr/~dsp

      yes that's true ;) some of us were from greece too ;) . and we have started thinking about a greek summer hackathlon or smth *if there is interest* ;) , sun/sea/beers/beers

      1. By Miod Vallat (82.101.10.8) miod@ on

        Beer? In Greece? Are you out of your mind? Developers will want to drink ouzo.

        1. By philipp (85.75.88.188) pB@ on

          there's very good greece beer. trust me. ;-)

          1. By dsp (62.1.231.72) on

            who said ouzo will not be available? specially if the con is held in crete where i work (http://dcs.ics.forth.gr)(crete == greek island) we will have raki too ;) http://www.sfakia-crete.com/sfakia-crete/raki.html

    2. By Lennie (82.75.30.141) on

      Those are not ISP's, those are exchange points, places where ISP's come to exchange data.

      It's very expansive to connect to a large number of ISP's to exchange data with them, so they come to exchange points.

      Although they could also transport there traffic over other providers, higher up in the chain, upstream if you like to call it that, but as you can imagine there is a cost involved with that.

      Most ISP's are connected to a few of these exchange points.

      1. By Venture37 (217.22.88.124) venture37@hotmail.com on www.geeklan.co.uk

        I take it that you mean peering??

        1. By Lars Hansson (203.65.246.6) on

          No, he means an exchange. Peering happens at an exchange but peering also happen between providers outside of an exchange.

  3. By Anonymous Coward (68.124.58.147) on

    Thanks for the report. ;-)

Credits

Copyright © - Daniel Hartmeier. All rights reserved. Articles and comments are copyright their respective authors, submission implies license to publish on this web site. Contents of the archive prior to as well as images and HTML templates were copied from the fabulous original deadly.org with Jose's and Jim's kind permission. This journal runs as CGI with httpd(8) on OpenBSD, the source code is BSD licensed. undeadly \Un*dead"ly\, a. Not subject to death; immortal. [Obs.]