OpenBSD Journal

A Week of OpenBSD Hacking In Slovenia: Developers Report From s2k11 (Part 1)

Contributed by pitrh on from the roomful of puffies dept.

s2k11 T-shirt design For a few days in September (16th through 23rd), the Slovenian capital Ljubljana was also the World Capital of OpenBSD hacking, hosting the s2k11: General hackathon, with 25 developers participating. Undeadly editor Mitja Muženič was there, but he was too busy organizing the event (by all accounts doing an excellent job) to write about it, so we asked each of the developers who participated to send us a short summary of what happened at the hackathon.

Those summaries form the base of this series of articles (we have several more in process). If you're not folloing -current, this is an early preview of what may appear in the next release, or in some cases of features that will be tentatively included for testing in upcoming snapshots.

We start off with Sebastian Reitenbach (sebastia@), who writes

I was in the middle of a holiday trip in Hungary, but since Ljubljana is nice, and always worth a trip, I managed to get to the first two days to the hackathon. As always, working on some ports. I added some GNUstep based ports, and updated the opencachemanager and its dependant webkit-sharp. Generally works well, but afterwards found some bugs, that I need to resolve before or at the p2k11.

Further I was working on getting XEP-167 (Jingle RTP Streams) and XEP-0177 (Jingle Raw UDP) into coccinella for video streaming via xmpp. It generally works, but still needs some work.

This sounds like a good chunk of work done on bits of the package system, and a proper warmup for the ports hackathon that's scheduled for later this year.

Next up is Ingo Schwarze (schwarze@), who continues to work on mandoc(1) and related text processing tools:

So, here's what i did:

  • New functionality:

    • Major rewrite of the horizontal spacing of tables in mandoc(1), for tables both with and without vertical rulers, making this part of our tbl(7) implementation compatible with groff(1).
    • Merge kristaps@' patch to support the roff(7) .tr request into our mandoc(1), improving the rendering of dashes in manuals generated by pod2man(1), in particular Perl manuals. This patch caused loud cheering by espie@. :)
    • Merge kristaps@' new eqn(7) parser into our mandoc(1). For complicated formulas, the rendering is still somewhat ugly, but having a good parser is a solid basis for further improvements in this area.
    • Start an implementation of mandoc(1) -Tman, an mdoc(7) to man(7) translator. That work is still far from complete, but the goal is that authors of highly portable software packages can write their documentation using the modern mdoc(7) language, but can still provide automatically generated man(7) code for old- fashioned operating systems that still fail to support mdoc(7).
  • Documentation improvements:
    • Add MACRO OVERVIEW sections to the mdoc(7) and man(7) manuals, making it easier to find the right macro for a given purpose.
    • Move the nearly identical LANGUAGE SYNTAX sections from mdoc(7) and man(7) to roff(7), and move the very technical MACRO SYNTAX sections down to the bottom, making the mdoc(7) and man(7) manuals much more accessible for casual documentation authors.
    • Remove the incomplete, partly outdated, and partly misleading mdoc.samples(7) manual now that mdoc(7) is complete and of good readability.
    • Make sure that all manuals, even those specific to other architectures and other GCC/binutils tools chain versions, get installed on all architectures (with deraadt@).
  • Bugfixes:
    • Several fixes related to mandoc(1) terminal output: Nicer page header lines for very long titles in mdoc(7). Multiple indentation fixes affecting various kinds of blocks. The man(7) .RS macro must reset and check the right margin. Fix a segfault by man(7) next-line arguments on the same input line.
    • Some fixes related to the roff(7) parser: Fix an infinite loop caused by recursive .ds requests. Hyphens are only breakable between alphabetic characters. Even a breakable hyphen may be bold or underlined.
    • Put in my sed(1) s/// partial rewrite fixing multiple bugs related to the replacement of zero-length matches (with otto@) and improve the related regression tests.
  • Preparations for future functionality:
    • Merge kristaps@' work on making the mandocdb(8) interface more similar to makewhatis(8). This is one of several steps in my ongoing quest to get us a better apropos(1).

Locale handling is interesting, useful and irritating, and Stefan Sperling (stsp@) among other things keeps working on getting the OpenBSD version right:

I was working on locale support and a few other things in libc.

Roughly 10 years ago, the default "C" locale in OpenBSD was changed to classify printable characters based on ISO-8859-1 rules instead of ASCII. E.g. the letter "ä" (a with umlaut, ISO-8859-1 value 0xE4) has since been considered a printable character by the isprint(3) function.

This caused problems in the UTF-8 locale because single bytes above the ASCII range such as 0xE4 do not form valid UTF-8 sequences. They are not recognisable when printed to a UTF-8 terminal. Most programs in the base system are not locale-aware and rely on isprint(3) to determine printability. The default is now again ASCII, which fixes the display glitches in UTF-8 locales and also makes our "C" locale conform to POSIX again. This also fixes problems with applications relying on isprint(3) to filter 8-bit characters (e.g. isakmpd(1) uses isprint(3) to verify peer names specified by the user, which should only contain ASCII).

However, this means people who want to use characters which are not in ASCII must now configure a locale if they have not already done so. I've written an FAQ entry that explains how to do this: http://www.openbsd.org/faq/faq10.html#locales.

I have written an implementation of wscanf(3) based on the existing scanf(3) implementation and code from FreeBSD. This hasn't been committed yet since I am waiting for review and a good opportunity to bump the libc library version number. Once it has been committed porting applications that use wscanf(3) will be much easier. We are getting closer to having most of the standard wide-character APIs available in OpenBSD's C library.

Lastly, I have taken a stab at porting Bill Rowe's new fnmatch(3) implementation to OpenBSD. This was released in APR-1.4.3 as a fix for CVE-2011-1928, a DOS vulnerability which causes fnmatch(3) to keep trying to match a large pattern space in all possible ways, wasting CPU time. We have a fix for our existing fnmatch(3) which makes sure the number of matching attempts is bounded. However, Bill's implementation doesn't use recursion at all, so in theory it is safer. But since it has been written from scratch it can potentially have other bugs. This new code needs a lot of additional review before it can be trusted and committed to OpenBSD. I also spotted two additional instances of the original problem which have now been fixed (see r1.6 of usr.sbin/httpd/src/ap/ap_fnmatch.c and r1.37 of lib/libc/gen/glob.c).

Next we turn to Ariane van der Steldt (ariane@), who is keeping a good pace at improving OpenBSD memory handling:

I spent the first day sleeping (driving 14 hours straight makes you tired). :)

Once properly awake, I spent a lot of time on the compatibility mode for the new vmmap allocator (of which I intend to speak on EuroBSDcon).

Uwe Stühler (uwe@) and I decided that we should also get into rthreads and start pushing that into a state that we can replace uthreads with it.

I also worked on the 3rd attempt to get the non-inline trees in a state where it's acceptable for commit.

Somewhere in between I managed to visit a castle (scary scary stairs in the tower, didn't make it to the top) and enjoyed the Slovenian restaurants. After that Mitja mentioned he was organizing a hike, but after seeing the preview (pictures of steep mountains) I passed; I can only handle so much height during a hackathon. :D

Somewhere during the hackathon I also managed to add to my todo list, to remove the biglock around the two most expensive system calls in OpenBSD: mmap and page fault resolution (the latter is not a real system call, but very expensive none-the-less).

After the hackathon, I spent another 14 hours of driving to get home and went straight to bed.

Mitja did an awesome job on organizing the hackathon, including bringing huge amounts of coffee, beer and other drinks into the hackroom.

The sentiments of that last sentence has been echoed by several others who wrote in, a huge thank you to Mitja for organizing this hackathon!

This is the first installment in a continuing series, several more developers have written in with summaries of their work in various parts of OpenBSD or have promised to do so, and we will be processing those into articles in the series over the next days.

(Comments are closed)


Comments
  1. By phessler (phessler) spambox@theapt.org on http://theapt.org

    Yes, many many thanks to Mitja for organizing the Hackathon. Amazingly well organized, especially for a first time!

    Comments
    1. By J.C. Roberts (jcr) on http://www.designtools.org

      > Yes, many many thanks to Mitja for organizing the Hackathon. Amazingly well organized, especially for a first time!

      Are we allowed to make "first time" jokes about our fellow editors? ;)

      Congrats Mitja!

Credits

Copyright © - Daniel Hartmeier. All rights reserved. Articles and comments are copyright their respective authors, submission implies license to publish on this web site. Contents of the archive prior to as well as images and HTML templates were copied from the fabulous original deadly.org with Jose's and Jim's kind permission. This journal runs as CGI with httpd(8) on OpenBSD, the source code is BSD licensed. undeadly \Un*dead"ly\, a. Not subject to death; immortal. [Obs.]