OpenBSD Journal

Experimental Speedups with UBC

Contributed by jose on from the throroughly-untested dept.

Seen recently on the misc@ mailing list , a UBC patch against the i386 kenrel source was posted by ted: 
I've created my own UBC (unified buffer cache) patch.  I expect it's not
perfect, but seems to be working for me. Instead of using the UBC
branch, I took the original UBC import and rolled it forward to today's
current.  Patch applies against today's cvs co -rHEAD.


pros:
1.  faster. compiling a kernel:
UBC     :  517.178u 41.811s 9:58.91 93.3%  0+0k 121+615io 129pf+0w
current :  522.022u 40.335s 10:15.73 91.3% 0+0k 5877+12957io 729pf+0w
compiling is hardly disk bound.  expect a bigger boost for apps that
have a large disk working set. 2.  OpenBSD is essentially the last OS
that lacks a variable sized disk cache.  This brings us in line.

cons:
1.  I haven't fully tested it, and it's not even close to being synced
with NetBSD yet.  Also some parts, such as NFS, aren't included. 2. 
Even current NetBSD has some problems were the buffer cache can grow out
of control and evict pages from the programs you're using.

The patch is only for i386 right now.  The changes for sparc and other
archs were fairly complicated and I can't test them, and it probably
doesn't matter much for 95% of you.  It also, as noted, doesn't fix up
NFS, which I'm not using and is large and complicated.  It will wait for
a later day.  I patched miscfs/*, msdosfs, and ufs/*.  I don't really
have the means of doing strenuous testing.  I do know that it will fall
over if you run out of memory, but stock OpenBSD has the same problem. 
I can use it for all the same tasks I typically use my system, and it's
prefectly fine, though.


Get it at http://www.zeitbombe.org/ubc.diff.gz

The diff is large.  26000 lines, split 50/50 between UVM and rest of
kernel.  200K compressed. The kernel configuration I used for testing is
in the patch, though you'll almost certainly need to edit it.

DISCLAIMER: I did this mainly for self-interest, and because I want UBC
and I want it now.  :).  Since it's done, I figured I could share, but
don't think that the OpenBSD developers have anything to do with this. 
(Other than the fact that they wrote OpenBSD of course.)  It is not
"officially licensed."

From here, I'm going to fix NFS too, and then start pulling in more of
the NetBSD changes.  If others are interested in working on an
unofficial UBC project, maybe we could put something together.

ted
This patch differs from the U BC branch of the kernel source tree, which Art Grabowski has been slowly working on (among many other things ). You should know that this patch from Ted is highly experimental, could lead to big system instability for you, is for i386 only, is wholly unsupported by the official OpenBSD team, and can lead to permanent hair loss (just kidding about the last one). However, if this sort of thing interests you and you want to learn more about kernel internals, nothing works as a learning motivator like trying to fix what's broken.

(Comments are closed)

Comments

  1. By Marc Espie () espie@openbsd.org on mailto:espie@openbsd.org

    UBC has been briefly committed to OpenBSD before
    3.1, but Art and other people didn't have enough
    time to make it actually work on the platforms
    OpenBSD is supposed to support. There were some
    problems, most of them of a very arcane nature.

    Be aware that UBC will bite very, very hard, in
    case of bugs, at least at first.

    Remember that this deals with memory, and buffers. Which means that, for instance, it's very convenient for storing garbage to your hard-drive.

    I don't want to disparage Ted's work, or anything (and it's likely official UBC *will* return at some point), I just want to make sure people really understand how dangerous such a patch might be if you don't understand the implications...

    Comments

    1. By Janne Johansson () on

      Also, arts UBC did work really nice on my amiga,
      I never had any UBC-related problems on my machine,
      from the day it went in until it went out again.
      It would be better to tweak/fix the UBC branch
      than to make a specific i386-only patch that will
      cover most users machines but wont satisfy the
      project goals by having different stuff on different
      platforms.

    2. By Anonymous Coward () on

      I wonder if fsx, that tool Apple released for testing file system implementations, would be a good debugging tool here.

  2. By Jedi/Sector One () j@pureftpd.org on http://www.pureftpd.org/

    There have been three important committments in NetBSD lately :

    - A major sockets performance improvement :

    http://mail-index.netbsd.org/current-users/2002/07/03/0011.html

    After a first look, these changes seem to be easy to port to OpenBSD.

    - Switch to a signal trampoline provided by libc. To quote : "This will allow marking stack pages as non-executable, which will prevent buffer overflows from executing random code provided by exploits" .

    http://www.netbsd.org/Changes/#libc-signal-trampoline

    Yet another great thing to port to OpenBSD.

    - In NetBSD-current, LFS now really works. LFS (loging filesystem) is faster than FFS (but it still has to take advantage of UBC), and provides instant crash recovery. There are some very old traces of LFS in OpenBSD, but nothing works. NetBSD has also preliminary support for NTFS.

    Comments

    1. By gwyllion () on

      Art is working on a non-executable stack. A lot of changes are being made:
      http://marc.theaimsgroup.com/?l=openbsd-cvs&m=102719323431216&w=2
      http://marc.theaimsgroup.com/?l=openbsd-cvs&m=102675247215742&w=2
      http://marc.theaimsgroup.com/?l=openbsd-cvs&m=102554750016708&w=2
      http://marc.theaimsgroup.com/?l=openbsd-cvs&m=102639632620211&w=2

      Zero-copy for TCP and UDP would be nice as well: http://mail-index.netbsd.org/current-users/2002/05/02/0016.html

    2. By Jedi/Sector One () j@pureftpd.org on http://www.pureftpd.org/

      Point #1 has just been ported to OpenBSD.
      Patch is available here :

      http://www.42-networks.com/obsd_net_speedup/

      Comments

      1. By Anonymous Coward () on

        Any numbers on performance gain?

    3. By AC () on

      ext3 would be great too. I mean not for real use but
      if people have linux on other hd and it upgrades from ext2 to ext3... hmm ... I suppose not very important since I can probably make linux use ext2 even with the newer distributions.

      Comments

      1. By Jedi/Sector One () j@pureftpd.org on http://www.pureftpd.org/

        You can already mount ext3 partitions on OpenBSD, but as ext2 systems.

    4. By Miod Vallat () miod@openbsd.org on mailto:miod@openbsd.org

      Remember that the OpenBSD team is small... So as long as perseant@netbsd's work on LFS does not interest anyone of us, the odds of someone working on this are very, very poor.

      If you really need LFS, then port it, test it, ans submit your work as a patch...

    6. By mirabile () on

      FreeBSD is said to have quite good NTFS support,
      en par with Linux, or at least nearly. NTFS is
      very tricky, though.

Latest Articles

Credits

Copyright © - Daniel Hartmeier. All rights reserved. Articles and comments are copyright their respective authors, submission implies license to publish on this web site. Contents of the archive prior to as well as images and HTML templates were copied from the fabulous original deadly.org with Jose's and Jim's kind permission. This journal runs as CGI with httpd(8) on OpenBSD, the source code is BSD licensed. undeadly \Un*dead"ly\, a. Not subject to death; immortal. [Obs.]