OpenBSD Journal

Speeding Up Compiles

Contributed by jose on from the time-make dept.

For developers who do a large amount of building, performance enhancements can make a noticable difference. Aside from more memory and faster machines, there are other things you can do to gain the desired speedups, three of which we note here. The first is the experimental port of ccache , which keeps a hashed cache of objects and recycles ones it has already built. Another is the use of /tmp as an MFS filesystem . MFS is the memory filesystem , which uses your system's memory to construct a filesystem suitable for mounting. Because the compiler creates and destroys files in /tmp, improved IO speeds here can be noticed in build operations. lastly, the use of tunefs(8) , when tweaked with an understanding of filesystems and the characteristics of the typical files which will be there, can also make a big difference.

While each of these can substantially improve the performance of a build machine, it should be noted that the first two also incur high risks. In your quest for improved build performance, you may want to investigate any of these options, or others.

(Comments are closed)

Comments
  1. Comments

    1. By Anonymous Coward () on

      Would that be better?

      Comments

      1. By Anonymous Coward () on

        Whatever gets you off. Although, you might not like the response.

  2. Comments

    1. By Anonymous Coward () on

      Unless every single port in OpenBSD ports tree can be compiled with ccache, ccache will stay experimental. As of right now about 20% of the ports can not be built with ccache.

  3. By RC () on

    the use of /tmp as an MFS filesystem

    Sounds good... Even better, let's make the swap partition in memory too.

    Comments

    1. By Anonymous Coward () on

      Can I install OpenBSD in RAM?

      Comments

      1. By Anonymous Coward () on ftp://ftp.openbsd.org/pub/OpenBSD/snapshots/i386/b

        Can I install OpenBSD in RAM?

        been there done that bought the t-shirt.

        for an easy close approximation, boot into your install floppy/cd, or off of bsd.rd

        Comments

        1. By Anonymous Coward () on

          I was thinking of the whole system.
          If I have 2-4 GB RAM, even all X-stuff and
          ports should fit there.

          Comments

          1. By Anonymous Coward () on

            Yes, but you will lose everything once you reboot.

            Comments

            1. By Anonymous Coward () on

              Can't that be solved with a proper backup procedure. To have the system in RAM would really solve most of our performance problems right now.

              Comments

              1. By pravus () on

                yeah, i can see it now...

                reboot... oop, i forgot i have to re-install... then restore all the updated files...

                seriously though, i think this would only work if you had a hard disk as a mirror of what you have in memory. and even then, every reboot means loading *everything* off of the hard disk into memory. perhaps a better solution is to use NVRAM-based storage? it's expensive, but would be a bit better than what you propose.

                Comments

                1. By Anonymous Coward () on

                  i think this would only work if you had a hard disk as a mirror of what you have in memory. and even then, every reboot means loading *everything* off of the hard disk into memory.

                  There was an article in a recent (two months ago?) sysadmin or linux journal where this guy created a mirror with one side being hd and the other mfs. The benefit was increased reads versus a hd-hd mirror, the drawback was having to resync on each bootup.

                  it would be interesting to create a workstation with a shitload of ram, hd-ram mirror, on boot up the machine resyncs the mirror, once thats done the hd gets taken out of the mirror. If you have no changing data then it might be useful, but due to the impracticalities involved in such a setup it will be more novelty than anything.

          2. By Anonymous Coward () on

            How do you think I had to install Open on my SPARC until I could get a new HD for it... netboot baby! you simply mount / over NFS... of course, you would probably benefit significantly in that scenario by using MFS /tmp

  4. By floh () floh@blafasel.org on mailto:floh@blafasel.org

    ccache shaves about 14mins off a make build on a 1.13ghz pentium with 384mb RAM:
    without ccache:
    4498.00s real 2983.65s user 580.21s system

    with ccache:
    3648.43s real 2059.09s user 458.73s system

    difference of about 14 minutes, as always YMMV

  5. By Anonymous Coward () on

    If you have about 512Mb of RAM or more, you can mount /usr/obj as mfs - this will give you much better "make build" speed improvement than any other methods. Same thing with ports - except that you can compile ports in mfs even if you have less RAM.

    Comments

    1. By Anonymous Coward () on

      yea, but how much will I need in /usr/obj ?
      am a bit worried about the peak usage :/

      Comments

      1. By Anonymous Coward () on

        about 300-400Mb - depends on the architecture.

  6. By Jeffrey N. () on

    increasing the value of this option seems to have some impact on compiles (and other things too). the default is 5% of RAM. i have a machine with 128M and usually change this option to use 20% of RAM instead.

    i did some tests a while back to see any difference. overall, with 5 vs. 10 vs. 20... 20 gave the best times on various tests. i still have the test results somewhere i think...

    regardless, for everyday workstation usage, this does seem to make the system perform better. i know... that is a rather vague statement ;)

Latest Articles

Credits

Copyright © - Daniel Hartmeier. All rights reserved. Articles and comments are copyright their respective authors, submission implies license to publish on this web site. Contents of the archive prior to as well as images and HTML templates were copied from the fabulous original deadly.org with Jose's and Jim's kind permission. This journal runs as CGI with httpd(8) on OpenBSD, the source code is BSD licensed. undeadly \Un*dead"ly\, a. Not subject to death; immortal. [Obs.]