OpenBSD Journal

Malloc improvements in 3.8-beta, testers needed!

Contributed by grey on from the lots of testing needed dept.

Many of you probably already saw this, but thanks to jhoux for writing in about Theo's post to misc@ recently about something that's been on the cards for a long time.

Thanks to Mouring for also pointing out the write up this was given on KernelTrap which may be found here: http://kerneltrap.org/node/5584

And finally, thanks to sthen for hyperlinking the content from Theo's post to relevant links, providing added benefit to undeadly readers who may view the complete post below instead of from the archive:

List: openbsd-misc
Subject: 3.8 beta requests
From: Theo de Raadt
Date: 2005-08-22 23:33:40
Message-ID: 200508222333.j7MNXepH022873 () cvs ! openbsd ! org

We are heading towards making the real 3.8 release soonish. I would like to ask the community to do lots of testing over the next week if they can.

This release will bring a lot of new ideas from us. One of them in particular is somewhat risky. I think it is time to talk about that one, and let people know what is ahead on our road.

Traditionally, Unix malloc(3) has always just "extended the brk", which means extending the traditional Unix process data segment to allocate more memory. malloc(3) would simply extend the data segment, and then calve off little pieces to requesting callers as needed. It also remembered which pieces were which, so that free(3) could do it's job.

The way this was always done in Unix has had a number of consequences, some of which we wanted to get rid of. In particular, malloc & free have not been able to provide strong protection against overflows or other corruption.

Our malloc implementation is a lot more resistant (than Linux) to "heap overflows in the malloc arena", but we wanted to improve things even more.

Starting a few months ago, the following changes were made:

- We made the mmap(2) system call return random memory addresses. As well the kernel ensures that two objects are not mapped next to each other; in effect, this creates unallocated memory which we call a "guard page".

- We have changed malloc(3) to use mmap(2) instead of extending the data segment via brk()

- We also changed free(3) to return memory to the kernel, un-allocating them out of the process.

- As before, objects smaller than a page are allocated within shared pages that malloc(3) maintains. But their allocation is now somewhat randomized as well.

- A number of other similar changes which are too dangerous for normal software or cause too much of a slowdown are available as malloc options as described in the manual page. These are very powerful for debugging buggy applications.

Other results:

- When you free an object that is >= 1 page in size, it is actually returned to the system. Attempting to read or write to it after you free is no longer acceptable. That memory is unmapped. You get a SIGSEGV.

- For a decade and a bit, we have been fixing software for buffer overflows. Now we are finding a lot of software that reads before the start of the buffer, or reads too far off the end of the buffer. You get a SIGSEGV.

To some of you, this will sound like what the Electric Fence toolkit used to be for. But these features are enabled by default. Electric Fence was also very slow. It took nearly 3 years to write these OpenBSD changes since performance was a serious consideration. (Early versions caused a nearly 50% slowdown).

Our changes have tremendous benefits, but until some bugs in external packages are found and fixed, there are some risks as well. Some software making incorrect assumptions will be running into these new security technologies.

I discussed this in talks I have given before: I said that we were afraid to go ahead with guard pages, because a lot of software is just written to such low standards. Applications over-read memory all the time, go 1 byte too far, read 1 byte too early, access memory after free, etc etc etc.

Oh well -- we've decided that we will try to ship with this protection mechanism in any case, and try to solve the problems as we run into them.

Two examples:

Over the last two months, some OpenBSD users noticed that the X server was crashing occasionally. Two bugs have been diagnosed and fixed by us. One was a use-after-free bug in the X shared library linker. The other was a buffer-over-read bug deep down in the very lowest level fb* pixmap compositing routines. The latter bug in particular was very difficult to diagnose and fix, and is about 10 years old. We have found other bugs like this in other external software, and even a few in the base OpenBSD tree (though those were found a while back, even as we started experimenting with the new malloc code).

I would bet money that the X fb* bug has crashed Linux (and other) X servers before. It is just that it was very rare, and noone ever chased it. The new malloc we have just makes code get lucky less often, which lets us get to the source of a bug easier. As a programmer, I appreciate anything which makes bugs easier to reproduce.

We expect that our malloc will find more bugs in software, and this might hurt our user community in the short term. We know that what this new malloc is doing is perfectly legal, but that realistically some open source software is of such low quality that it is just not ready for these things to happen.

We ask our users to help us uncover and fix more of these bugs in applications. Some will even be exploitable. Instead of saying that OpenBSD is busted in this regard, please realize that the software which is crashing is showing how shoddily it was written. Then help us fix it. For everyone.. not just OpenBSD users.

(Comments are closed)


Comments
  1. By Isak Lyberth (83.72.68.240) isak@lyberth.com on www.lyberth.com

    I downloaded the latest snapshot today and burned it on a cd to install on my will be firewall, but i couldn't get it to install. I never made it through the package installation process. From the cds it was failing one package, when i tried to download the packages from the ftp sites it wouldn't show the mirror sites. when i then downloaded it onto another server (from the main ftp site) i could install some packages again, but still it failed before all packages were installed. I tried with the 3.7 and it went trough on first try. Regards Isak

    Comments
    1. By Brad (204.101.180.70) brad at comstyle dot com on

      If you expect anyone to even remotely help you, whether it be on here or through a proper bug report, then you need to give more detail. This post of yours as it is is useless.

      Comments
      1. By Isak Lyberth (83.72.68.240) isak@lyberth.com on www.lyberth.com

        Shows how much i know, i will try to get more information when i try again tomorrow. Yo!

        Comments
        1. By Ghost of devil's past (198.175.14.5) on

          Sounds like it is trying to use /pub/OpenBSD/3.8/xxx/ but the snapshots require that you manually type in /pub/OpenBSD/snapshots/xxx/ instead....

          Comments
          1. By tedu (64.167.149.107) on

            it has always been that way.

            Comments
            1. By George W. Bush (198.175.14.5) on

              Of course. But people are now being encouraged to install snapshots and it may be the first time for some.

  2. By peter (80.108.115.184) on

    While I have to admit I don't understand much of this very technical things I am able to notice an impressive and not natural thing: Just when such a change is announced, the documentation for it is updated already and therefor consistent with the real behaviour of the system! No outdated docs any more - cool! :)

    Comments
    1. By Anonymous Coward (82.236.141.3) on

      OpenBSD man pages are very cool, indeed!
      It is not always obvious, for newbie developers like me, if a function is risky to use or not (and, I am sure, for more experienced developers too; think of the infamous strncpy(3) which has weird semantics and violates the principle of least astonishment). Fortunately, the OpenBSD man pages give hints on what to avoid (e.g. they recently added warnings in the atexit(3) man page).

      Comments
      1. By Anonymous Coward (66.44.1.28) on

        I think strl*() is a good idea as much as the next guy, but calling strncpy "infamous" is a bit of a stretch. I mean, strncpy() does what it does, and it's an improvement over strcpy(). Even strcpy() is not so bad in the very rare cases that you know based on the input that an overflow is impossible.

        Comments
        1. By Anonymous Coward (69.194.239.115) on

          er ... rare? It's perfectly safe if you know the length and malloc. It will just cause some headaches with static analysis tools such as rats or grep.

          /* for example */
          void
          stralloc(char *buf, size_t len)
          {
          char *p = NULL;

          if ( (p = (char *)calloc(len + 1, 1)) == NULL)
          err(EXIT_FAILURE, "calloc");

          (void)strcpy(p, buf);

          free(p);
          }

    2. By almeida (66.31.180.15) on

      If you watch source-changes@, you'll see lots of commits to the man pages with comments like "sync to reality" or "sync with recent changes." It's nice having maintained man pages for everything in the system. The system calls and subroutines sections (2 and 3) are great for development too because they are loaded with examples and warnings.

    3. By Nickus (141.5.11.5) on

      I don't know how often I've been sitting at a Linux box swearing over the low quality of some of their man-pages. That is, if I even find a man-page. With OpenBSD you know that you have all your need installed to be able to use the system - with a Linux box you sit and look for howtos on the net to get going.

      But it looks like the 3.8-beta works fine, at least on my Powerbook G4.

      Comments
      1. By Anonymous Coward (195.224.109.30) on

        and on my Z

  3. By tony (65.31.212.84) tony@lysergik.com on http://lysergik.com

    I'll be installing a snapshot on my 700m tomorrow!

  4. By Random (204.209.209.129) on

    this is one of the many reasons that I think OpenBSD is so amazing. What other operating system can say that they are fixing bugs that are over 10 years old and forcing people to write better software. Thank you OpenBSD for making open source software better.

  5. By Justin (216.17.75.77) on

    So presumably with these changes....
    All things in base work?
    All packages work?
    All ports work?

    Or are the answers to these questions unknown thus the reason for Theo's request for heavy testing?

    Comments
    1. By Anonymous Coward (82.236.141.3) on

      Well, the idea behind these changes is to crash processes when they corrupt their own memory. Before, all cases of memory corruption were not handled by the system so bugs in applications could have stayed "under the radar" for years!

      So it probably affects (buggy) third-party applications much more than it affects base. That is why the OpenBSD developers want to get rid of the most easy bugs in packages before the release.

    2. By Anonymous Coward (195.224.109.30) on

      > All things in base work?

      I would have thought so

      > All packages work?

      Maybe

      > All ports work?

      Probably not

      Comments
      1. By Charles T. (137.240.136.82) on

        I loaded the 3.8 beta on an Intel NetPC
        BL440X (400 Mhz Celeron) 128M RAM...
        The snapshot was dated August 19..
        Installed (August19 snapshot) packages
        (not including dependancies)
        samba
        postfix
        logsentry
        pfstat
        nmap-noX11
        analog (used for fwanalog)
        no issues with any of these..

        this is same setup I have on my 3.6 box (same hardware)

        The only issue was with apcupsd-3.10.18
        in apcupsd.conf I had to change the NISIP
        from 0.0.0.0
        to 127.0.0.1 (the apcupsd did take a significant
        longer time to load over the 3.6, about 20-30 seconds
        vs 2-5 on 3.6)

        hth
        Charles

    3. By sickness (82.61.141.253) on http://www.sickness.it

      i'm trying the snapshots and the only thing I noticed is that links+2 -g crashes a lot :/

      Comments
      1. By SH (82.182.103.172) on

        Perhaps a bug report is appropiate, or just a mail to ports@

        Comments
        1. By djm@ (203.217.30.86) on

          Ideally with a trace from gdb

          Comments
          1. By Anonymous Coward (83.41.116.234) on

            :/

  6. By Anonymous Coward (66.167.105.138) on

    Way to go. This is the right thing to do. Bugs should cause applications to fail sooner rather than later. I've been using primarily Linux for quite a while now, due to only one thing: no modern and production-ready Java environments for OpenBSD. But I really miss the security-first philosophy of OpenBSD, and I think that we are pretty close to having a usable open-source Java implementation (Kaffe and gcj) so I hope to once again have OpenBSD as my desktop and server system.

Latest Articles

Credits

Copyright © - Daniel Hartmeier. All rights reserved. Articles and comments are copyright their respective authors, submission implies license to publish on this web site. Contents of the archive prior to as well as images and HTML templates were copied from the fabulous original deadly.org with Jose's and Jim's kind permission. This journal runs as CGI with httpd(8) on OpenBSD, the source code is BSD licensed. undeadly \Un*dead"ly\, a. Not subject to death; immortal. [Obs.]