OpenBSD Journal

g2k14: Ingo Schwarze on manly stuff

Contributed by jj on from the runs the ministry of propaganda dept.

In the week right before the hackathon, I have done quite a bit of work cleaning up mandoc(1) warning and error messages. The goal is to provide more, more precise, and more readily understandable information to the user, in particular mentioning in the messages which section titles, macro names, and arguments each individual message is related to, and which workaround or fallback mandoc(1) has chosen, if any.
Also, I'm trying to use descriptive rather than imperative style wherever possible and unify the wording for similar issues. Some messages clobbering together unrelated kinds of issues were split, some bogus messages deleted, some overblown ones downgraded, in some cases from FATAL to a mere WARNING. In the process of looking at almost all messages, I fixed more than a dozen parsing and formatting bugs along the way, and i started providing regression tests for messages, cleanly integrated into the well-known OpenBSD /usr/src/regress/usr.bin/mandoc/ regression suite. This cleanup is not quite complete yet, but the bulk of the work has been done, maybe about 75% so far.

On the OeBB Eurocity train to Ljubljana, I already started working on man.cgi(8), the CGI interface to search and display manual pages on the web, upgrading the old Berkeley DB version rotting in the tree to use the new mandoc 1.13 SQLite backend we have in OpenBSD, so I could commit a first working version to on the first morning in Ljubljana.

After simplifying the server directory structure, the manpath.conf configuration file format, and the URI scheme, I imported the source code into the OpenBSD tree and continued development there. The main user-visible progress this week is to cleanly distinguish between man(1) and apropos(1) mode. In man(1) mode, which is the default, we now always show an actual manual page, in addition to links to other pages of the same name, if any. The search form was polished using feedback from Bob Beck@ and others. Besides, there were lots of small improvements behind the scenes:

A full rewrite of the man.cgi(8) manual that is now also used online; a cleanup of error reporting; a reasonable default for .Os; getting rid of pointless run-time configuration, using minimal compile-time configuration instead; getting back closer to the classical URI scheme; always including manpath= when printing queries, and omitting empty parameters; and a compatibility hack for the old OpenBSD "manpath=OpenBSD<blank>" query parameter format. Thanks also to Ted Unangst (tedu@) for finding the time to send two bugfix patches for man.cgi(8) among all his other work.

There is an old saying that hackathons are ideal for either starting work on a new task, to be polished afterwards, or getting an old one finished that was started long before. The man.cgi(8) replacement is one of the rare examples where a task was started right on the voyage to the hackathon *and* finished before the end of it, including deployment of the less-than-five-days-old software in production on and even on
And by the way, using queries like
you can now run semantic searches on the original CSRG 4.4BSD-Lite2 manual pages!

Partly in parallel to that, but mostly after man.cgi(8) was in production, I picked up the pod2mdoc(1) utility that Kristaps@ Dzonsons recently wrote in one of his characteristic flurries of extraordinary creativity. To help Anthony J. Bentley@ getting started with the LibReSSL manual conversion form perlpod(1) to mdoc(7), i pushed a pod2mdoc-0.0.12 release out of the door, and he promptly updated his port in the OpenBSD tree. He then started using the tool in practice, converting many manual pages, doing considerable manual postprocessing on each of them, and reporting lots of bugs and feature requests with respect to the tool.

I couldn't quite keep up with his pace, but some stuff got done by now: during the hackathon, correct handling of filename extensions and better rendering of B<NULL>, and right after the hackthon multiple fixes regarding the handling of POD commands and formatting codes and the spacing around them in general, substantially redesigning some of the internal interfaces in pod2mdoc.c. During the hackathon, I also started a regression suite for pod2mdoc(1). That work led to the pod2mdoc-0.0.13 release today, on July 19.

The low version number still makes sense, there is much to do still to polish this tool and add missing functionality, in particular heuristics for guessing how various kinds of text ought to be marked up, to make manual postprocessing less painful.

As usual, various other bits and pieces got addressed during the
  • The whatis(1) utility now correctly matches words instead of any
    substrings. This helps man.cgi(8), but is also nice for the
    stand-alone command line version.
  • The security(8) utility no longer complains when /etc/exports
    does not exist - it is now optional. Thanks to Antoine
    Jacoutot (ajacoutot@) for the bug report.
  • Together with Theo (deraadt@), i have cleaned up the format
    of the file /etc/mtree/4.4BSD.dist to make it more readable.
All in all, g2k14 was an exceptionally focused and productive hackathon for me. Having a familiar and very well-organised venue helped a lot (thanks Mitja!). I didn't spend a lot of time in the city this year, but that doesn't matter much because I have seen and enjoyed some of it during s2k11. Well, I did find the time to have a stroll to the Golovec Hill with Rapha@el Graf, which was a very nice conclusion of a great event.

(Comments are closed)

  1. By Amit Kulkarni (amitkulz) on

    Thanks Ingo, now that you mention the man URL, the GUI interface is totally revamped and superb!


Copyright © - Daniel Hartmeier. All rights reserved. Articles and comments are copyright their respective authors, submission implies license to publish on this web site. Contents of the archive prior to as well as images and HTML templates were copied from the fabulous original with Jose's and Jim's kind permission. This journal runs as CGI with httpd(8) on OpenBSD, the source code is BSD licensed. undeadly \Un*dead"ly\, a. Not subject to death; immortal. [Obs.]