ingo@ incorporates man into mandoc

I have integrated the traditional man(1) program - yes, the one to display manual pages - into mandoc(1). For apropos(1), whatis(1), and mandoc(1), the unified interface described below has now been enabled in OpenBSD-current since August 26, 2014.

OpenBSD-current still installs and uses the traditional man(1) program by default because the new version changes some of the behaviour of man(1) in some ways, not all of which are intended, and testing and evaluation are incomplete.

If you want to test the new man(1), you can do this:

# mv /usr/bin/man /usr/bin/oman
# ln -s /usr/bin/mandoc /usr/bin/man

Both versions work fine in parallel, so after switching, whenever the new man does anything weird, just re-run your command with oman.

By the way, to get full searching power, don't forget to run

# makewhatis
# echo "MAKEWHATISARGS=' '" >> /etc/weekly.local

if you have 4 Megabyte to spare for a full mandoc database /usr/share/man/mandoc.db. Here is what this is all about:

1. If we ever switch over from the old to the new man(1), we can delete all MLINKS from the base install. That's more than 3000 less inodes in the install tarballs. We could also delete all the .so link files from the Xenocara install. However, matthieu@ decided that's not going to happen in order to not diverge from the upstream buildsystem.
2. We get a unified user interface for man(1), apropos(1), whatis(1), and mandoc(1) - that is, all options currently present in any of them work for all of them where they make sense, hence less to remember. Here is a list:

                                                      OLD     NEW
       -a  show all matching manuals in a pager       man     all
       -c  do not use a pager, just copy to stdout    man     all
       -C: alternate man.conf                         all except mandoc
       -f  whatis(1) = match whole words in names     man     all
       -I: parser input options                       mandoc  all
       -k  apropos(1) = semantic substring+RE search  man     all
       -M: add to MANPATH                             all except mandoc
       -m: override MANPATH                           all except mandoc
       -O: output options                             mandoc  all
       -S: restrict architecture                      all except mandoc
       -s: restrict manual section                    all except mandoc
       -T: output format                              mandoc  all
       -V  display version number and exit            mandoc  all
       -W: warning level                              mandoc  all
       -w  only show filenames of matching manuals    man     all ex. mandoc
   

3. We get a whole bunch of new features basically for free. All of a sudden, stuff like this just works:

       # format the manual you just edited straight to your pager
       mandoc -a man.1
   
       # format all manuals containing "timespec" in a function argument
       # straight to a pager, and then use / to see where it occurs
       apropos -a Fa=timespec
   
       # see whether any of these manuals contain syntax errors
       apropos -acTlint Fa=timespec
   
       # the console is so 1980ies, use graphics (not enabled by default)
       man -Tps gv | gv -
   
       # ... or for the web hipsters (also not enabled by default)
       man -Thtml lynx | lynx -stdin
   
       # where are those blasted manual files?
       whatis -w roff
       /usr/share/man/man7/roff.7
       /usr/local/man/cat7/roff.0
   

4. All this required almost no new code (+350/-170 lines) because basically all the code is already around, I just had to reshuffle it a bit to allow the code flow "first search, then format, then display".

Five months back in Ottawa I presented a slide "possible future directions". This project was not listed. I didn't expect myself that I would do this.

But then, on August 9 this year Paul Onyschuk of Alpine Linux (you know, the first Linux distro that integrated mandoc, in July 2010) asked me:

:: Are there any plans for providing man(1) command also?  This would
:: make mdocml a possible, standalone replacement for groff and man-db
:: combination (typical in Linux distributions).

Almost immediately I returned my standard negative answer, but then I stopped short and realized that almost all the needed code was already there and it cheaply allows doing fancy things without complexity. I had to do the mandoc 1.13.1 and 1.12.4 releases first, so it took two weeks from the idea to the first working implementation... :-)

This also means we can delete MLINKS from the install: When makewhatis(8) runs, it adds names to the "names" table in the mandoc.db(5) databases, noting in the "names.bits" column of that table where the name came from (.DT/.TH header line, .Nm in the NAME section, .Nm in the SYNOPSIS section, file name). It also saves the file names where stuff was found into the "mlinks" table and links both tables together (and to the "keys" table with all the search terms) via the "mpages" table.

So the new man(1) does a four-step process: Look for exact matches (not just matching words like whatis(1)) in the "names.name" column, retrieve a matching file name from the "mlinks" table, pass that filename to the normal mandoc(1) parser+formatter steering function, and optionally spawn your pager.

Authors wouldn't be required to add an MLINK section to makefiles; things listed under NAME will show up correctly. Right now, the new man(1) uses all the types of names when selecting manuals for display. That needs to be tuned:

• The name from the .Dt/.TH is probably not all that useful for this purpose because it lacks the case information.
• The .Nm macros from the NAME section should almost certainly be used.
• The .Nm macros from the SYNOPSIS section should probably not be used for this purpose.
• The file names should probably still be used, just in case someone installs manuals the old way and screws up the name sections.

The following obstacles need to be considered before enabling the new man(1) by default:

• As explained above, the new man(1) may show some additional results, and some of those may be bogus, if NAME sections contain bogus .Nm macros.
• Most man.conf(5) features are not supported. The following types of lines are completely ignored: _build, _default, _subdir, _suffix and section lines. The search order 1, 8, 6, 2, 3, 5, 7, 4, 9 is hardcoded. Instead of _default, the _whatdb lines are used. All subdirectories are hardcoded to {man,cat}N.
• When finding a formatted and an unformatted manual of the same name in the same section, the old man(1) shows the one that was less recently changed. The mandoc man(1) currently always prefers the unformatted version, even if it's older than the formatted version.
• The new man(1) ignores the MACHINE environment variable, and I'm not planning to add support for it.