OpenBSD Journal
Home : : Add Story : : Archives : : About : : Create Account : : Login :
mandoc - UNIX Manuals
Contributed by jcr on Fri Jun 4 09:32:03 2010 (GMT)
from the manly dept.

If you've ever opened up a raw man page in a text editor, somewhere in the back of your mind you heard the words of Arthur C. Clarke, "Any significantly advanced technology is indistinguishable from magic," or the words of Larry Niven, "Any sufficiently advanced magic is indistinguishable from technology." Either way, you knew you had a lot to learn.

The recent and upcoming mandoc (via mdocml) changes are significant improvements to how manual pages are handled in OpenBSD. This important work not only improves build times, but also improves rendering and flexibility. Kristaps Dzonsons and Ingo Schwarze (schwarze@) were kind enough to tell us about the ongoing work.

NOTE: If you find man page formatting bugs, please do not file a PR (Problem Report) with sendbug. You should report problems directly to Kristaps and Ingo, or better to the mdocml mailing lists:

mdocml is being used by OpenBSD, NetBSD, and DragonflyBSD as well as in the ports collection of FreeBSD, so reporting problems properly upstream helps everyone.

Also, you should check the todo list Ingo Schwarze (schwarze@) has put together so you don't report an already known issue:

Just in case you missed it, Kristaps gave a presentation on mdocml at AsiaBSDCon 2009; Deprecating groff for BSD manual display. You can find the paper and a video of the talk online. Also, you should get familiar with the magic by reading the relevant documentation, both in OpenBSD itself and in the mdocml suite.

A nice description of the mdocml suite is at mdocml.bsd.lv

mdocml is a suite of tools compiling -mdoc, the roff macro package of choice for BSD manual pages, and -man, the predominant historical package for UNIX manuals. The mission of mdocml is to deprecate groff, the GNU roff implementation, for displaying -mdoc pages whilst providing token support for -man.

Why? groff amounts to over 5 MB of source code, most of which is C++ and all of which is GPL. It runs slowly, produces uncertain output, and varies in operation from system to system. mdocml strives to fix this (respectively small, C, ISC-licensed, fast and regular).

The core of mdocml is composed of the libmdoc, libman, and libroff validating compiler libraries. All are simple, fast libraries operating on memory buffers, so they may be used for a variety of front-ends (terminal-based, CGI and so on). The front-end is mandoc, which formats manuals for display.

The mdocml suite is a BSD.lv Project member.

From Kristaps Dzonsons:

I wrote mandoc, so I can provide some historical bits.

mandoc exists because grohtml didn't let me change the colour of `.Sh' section headings. I wanted to HTML-format the manuals for mult and sysjail (cf. http://bsd.lv/) consistent with the style of the surrounding site and was upset when it didn't Just Work. So I wrote a little tool to consume my manuals in -mdoc and directly produce CSS and HTML, so mdocml is short for mdoc2html and CVS dates my first check-ins at 18 months ago.

Anyway, 14 months ago mdocml branched into mdocterm and mdochtml, sometime before AsiaBSDCon-2009, where it was featured in a talk. Soon after it went into OpenBSD.

mdocml begat mandoc, consolidating mdocterm and mdochtml with nroff-compatible arguments (-Txx and -mxx) around 12 months ago. Soon after, mandoc took on -man via libman also about 12 months ago. The only recent feature added was -Txhtml about 3 months ago. The rest is accommodating for roff's strangeisms, both in terms of input and output.

Today, mandoc is in the base system of OpenBSD (linked to the build!), NetBSD, and Dragon Fly BSD as well as in ports for FreeBSD.

mandoc's main code contributors are Ingo Schwarze (schwarze@) and Joerg Sonnenberger (NetBSD). Ingo has performed yeoman labour in fitting mandoc into OpenBSD's build process, submitting patch after patch to get it working properly with OpenBSD manuals, and making it byte-compatible with GNU troff. Joerg has also done considerable work to similar effect, and also contributed the compatibility layer that allows mandoc to work on most any Unix system. Ingo and Joerg are the downstream contacts for OpenBSD and NetBSD. Ulrich Spörlein is the downstream contact for FreeBSD and Sascha Wildner is the downstream for Dragon Fly BSD. Many others, most significantly Jason McIntyre (jmc@), have tested mandoc on all manner of manuals.

I do ask that anybody with a manual please test with mandoc. And if it doesn't work, to cross-check mdoc.samples(7) and mdoc(7) to make sure that the manuals aren't broken before submitting a report. Lastly, if one's manuals are in -man format, for gods sakes re-write them in -mdoc format! (Search for " Fixing on a Standard Language for UNIX Manuals" for the scoop...)

When I made the mistake of filing an OpenBSD PR with sendbug on a man page formatting bug, Ingo was kind enough to send me the following.

From Ingo Schwarze (schwarze@):

In general, when groff copes, mandoc ought to cope, too. Except for very rare exceptions.

We know there are still lots of small problems. It is not difficult to dig up more of these than we can fix quickly by just running automatic comparisons across the tree.

The most helpful bug reports regarding non-fatal mandoc errors currently are those that

  • report formatting errors with important consequences (e.g. layout completely garbled to the point that one cannot figure out the content any more)
  • report one problem at a time
  • say precisely what goes wrong, best with man(7)/mdoc(7) source code and groff and mandoc output

But even those need not go to the OpenBSD bug tracker, or we will quickly put more nits there than people want to see in that place. Just send them to kristaps and me directly.

[topicopenbsd]

<< Interview with OpenBSD developer on Distrowatch | Reply | Flattened | Expanded | ldapd entering the tree >>

Threshold: Help

Related Links
more by jcr


  Re: mandoc - UNIX Manuals (mod 7/43)
by Will Backman (bitgeist) (bitgeist@yahoo.com) on Fri Jun 4 12:58:41 2010 (GMT)
http://bsdtalk.blogspot.com
  I hear this has major benefits for build times on various architectures.
  [ Show thread ] [ Reply to this comment ] [ Mod Up ] [ Mod Down ]

  Re: mandoc - UNIX Manuals (mod 0/46)
by Carson Chittom (carson) (carson@ashatteringbeauty.org) on Fri Jun 4 16:59:51 2010 (GMT)
http://ashatteringbeauty.org
 

This is good as far as it goes, I suppose. But I have to question why we're still using a 40-year-old[1] typesetting system for our man pages, which—as far as I'm aware—isn't used for anything else.[2] I'm not saying I know what the right answer is, but surely there must be something more usable than the immensely arcane syntax I see when I look at the source for a man page: even TeX is better.

Is there some technical reason we still set man pages this way?

[1] Wikipedia says the Unix version of roff dates from around 1970.
[2] Please correct me on this if you can. I'm genuinely curious.

  [ Show thread ] [ Reply to this comment ] [ Mod Up ] [ Mod Down ]

  Re: mandoc - UNIX Manuals (mod 8/38)
by Chris (LizardKing) on Sun Jun 6 22:13:44 2010 (GMT)
http://www.chriswareham.net/
  This is great news for many reasons. One of the more obscure ones, is that native builds of NetBSD Vax choke on parts of the groff codebase. As far as I'm aware, groff is the only major portion of the codebase on any of the BSD's that is written in C++ (toolchain excepted). And as a fully paid up member of the C++ haters club, the switch to a C codebase is much appreciated.
  [ Show thread ] [ Reply to this comment ] [ Mod Up ] [ Mod Down ]

  Re: mandoc - UNIX Manuals (mod -1/19)
by Anonymous Coward (45.116.233.34) on Mon May 23 07:57:22 2016 (GMT)
  Awesome work! That is quite appreciated. I hope you’ll get more success. Electronics
  [ Show thread ] [ Reply to this comment ] [ Mod Up ] [ Mod Down ]

  Re: mandoc - UNIX Manuals (mod 2/18)
by Anonymous Coward (103.10.197.218) on Sat May 28 10:00:13 2016 (GMT)
  Thank you so much for such a great blog. care coaching certification
  [ Show thread ] [ Reply to this comment ] [ Mod Up ] [ Mod Down ]

  Re: mandoc - UNIX Manuals (mod 2/20)
by tom (37.221.160.107) (toms72366@gmail.com) on Wed Jun 1 09:55:12 2016 (GMT)
  Every week-end I used to pay a fast visit this site, because I’d like enjoyment, because this web site conations certainly fussy material. Vivint
  [ Show thread ] [ Reply to this comment ] [ Mod Up ] [ Mod Down ]

  Re: mandoc - UNIX Manuals (mod 1/21)
by tom (37.221.160.118) (toms72366@gmail.com) on Thu Jun 2 06:50:53 2016 (GMT)
  That’s a nice site you people are carrying out there. payday loans no thirdparty
  [ Show thread ] [ Reply to this comment ] [ Mod Up ] [ Mod Down ]

  Re: mandoc - UNIX Manuals (mod 2/20)
by tom (37.221.160.98) (toms72366@gmail.com) on Sat Jun 4 06:35:50 2016 (GMT)
  I continuously continue coming to your website once more simply in case you have posted new contents. advance payday
  [ Show thread ] [ Reply to this comment ] [ Mod Up ] [ Mod Down ]

  Re: mandoc - UNIX Manuals (mod 1/21)
by tom (5.254.65.6) (toms72366@gmail.com) on Wed Jun 15 06:23:50 2016 (GMT)
  I would be supportive on all your articles and blogs as a result of they are simply up to the mark. Entrepreneur Coach
  [ Show thread ] [ Reply to this comment ] [ Mod Up ] [ Mod Down ]

  Re: mandoc - UNIX Manuals (mod 1/19)
by tom (5.254.65.6) (toms72366@gmail.com) on Wed Jun 15 06:24:20 2016 (GMT)
  I would be supportive on all your articles and blogs as a result of they are simply up to the mark. Entrepreneur Coach
  [ Show thread ] [ Reply to this comment ] [ Mod Up ] [ Mod Down ]

  Re: mandoc - UNIX Manuals (mod -1/13)
by mxffiles (218.11.246.179) on Tue Feb 7 07:41:02 2017 (GMT)
  This is a very good post which I really enjoy reading. It is not every day that I have the possibility to see something like this. Software mxf Software mxf converter free download to convert HD camcorder files. ts converter convert ts video files to avi, mp4, wmv, mov mts to avi mp4 mov mkv iMovie, FCP/FCE with mts converter, so to convert mts files for your PC and mobiles. mod converter and convert tod files just free download mod video converter. m2ts
  [ Show thread ] [ Reply to this comment ] [ Mod Up ] [ Mod Down ]

[ Home | Add Story | Archives | Polls | About ]

Copyright © 2004-2008 Daniel Hartmeier. All rights reserved. Articles and comments are copyright their respective authors, submission implies license to publish on this web site. Contents of the archive prior to April 2nd 2004 as well as images and HTML templates were copied from the fabulous original deadly.org with Jose's and Jim's kind permission. Some icons from slashdot.org used with permission from Kathleen. This journal runs as CGI with httpd(8) on OpenBSD, the source code is BSD licensed. Search engine is ht://Dig. undeadly \Un*dead"ly\, a. Not subject to death; immortal. [Obs.]