OpenBSD Journal

SMP status report

Contributed by jose on from the no-locks-are-better-than-deadlocks dept.

Simon Paquet writes:
"Jeremy Andrews over from KernelTrap has an interesting synopsis regarding the current state of SMP support in OpenBSD."
SMP on OpenBSD is many peoples' dream. A project like the Spinlocks project seems to be dormant (I recently mailed them to see if they could give any updates). The OpenBSD-smp list is 99% NOT developer traffic but instead wistful thinking. So, give the Kerneltrap piece a good read and exhale, because holding your breath for OpenBSD SMP is not good.

(Comments are closed)

Comments

  1. By Anonymous Coward () on

    The spinlocks project is dead, get over it. Don't waste your time writing mails to them because you won't get any answer. Better check out -rSMP and do the work yourself.

  2. By michael () on

    just a note currently the link in the article is pointing to http://www.deadly.org/www.kerneltrap.org not http://www.kerneltrap.org , just an FYI

  3. By T () corgimax@hotmail.com on mailto:corgimax@hotmail.com

    I was reading a book about performance tuning for sendmail and in the book it says it's better to have two weaker machines to provide email service than one powerful machine. So why expend all this effort on smp and instead focus on clusters or something? Wouldn't the rest of the hardware still be a bottleneck that diminishes the advantages of two cpu's? And the availability of smp motherboards isn't as wide spread as single processors thus we should keep the focus on single processor advances. Just some of my thoughts.

    Comments

    1. By Anonymous Coward () on

      Well said, I think that's a very good point.

      Comments

      1. By zil0g () on

        that would depend on the situation, as with everything else...

    2. By Anonymous Coward () on

      SMP is a necessity for a high-load database server. Not only will it max out CPU, you cannot cluster database servers together (without Oracle - I'm talking more Postgres/My). Your only hope for reasonable performance is to load up one big machine with all the CPU power you can muster.

      SMP is big, it lets OpenBSD play with a lot of real application tasks.

    3. By Teknoenie () teknoenie@site-fx.net on http://www.site-fx.net

      Two words come to mind. Hardware Crypto. It would be nice to have an additional processor that, if not doing anything else, could be dedicated to doing crypto stuff. The added power would certainly be welcome for people running HTTPS web servers, or VPNs. Sendmail however is a different story in that filesystem layout, block sizes, number of queues, etc are what determine its performance and to a lesser degree CPU horsepower. As well having a fallback MX allows for HA mail services.

      Comments

      1. By Anonymous Coward () on

        Why buy a new mobo and second chip when you can just get a PCI hardware crypto card? It's cheaper and the support is already there.

        The word that comes to my mind is HyperThreading. Multiple chips on a die are coming too. SMP is the coming thing whether Theo likes it or not.

        Comments

        1. By grey () on http://www.cansecwest.com

          I'd say that is a much more convincing argument for SMP support, now that even consumer chips (the fast P4's) have HT. Not so much that OpenBSD is going to lose out from not having SMP support sooner - but even the single -cpu-'s people will buy will not get used to their potential, waste makes want (and if you shelled out $ for a cpu which has that, you want your favourite OS to utilize it).

          It's not just Intel either. I'm sure AMD will look towards some form of HT tech if x86-64 generates them enough revenue to keep fighting (here's hoping. Preliminary reports from a friend @ RSA who has been using the Opterons says they're awesome [and he's been using Intaniums for ~3 years without any kind of lasting impression] to quote: "Compiling looks like ls flying by!"). Meanwhile, IBM is making HT/multicore cpu's in their power series and will likely extend that into the PPC970 derivates we're bound to see in Mac hardware by next year. Even HP and Sun I think I recall reading of having multicore+/htish offerings in the future [though Sun is continuing to be a dick to OpenBSD it seems, so I'm not holding my breath for niceness when they push USIV and US5 architectures].

          MP support for various architectures has been on art's list for a long time (it would be nice to see a changelog of that as an aside). It is going to happen, someday. But, this is OpenBSD - light speed is not what they do (or what the users want).

          But look at what you've likely heard before:

          ELF support for x86 is coming soon, now that 3.3 has been sent to get pressed. (weeks from now in all likelihood). As someone else mentioned, this could aid in getting us up to date with gcc (we're still on 2.95). Then in turn, Getting to a newer GCC will enable us to support x86-64 natively (2.95 does not offer support from what I've heard). Where does SMP fit into that? Well, it doesn't - but groundwork is being laid for some more fundamental changes (there's already tons of other stuff going on related just to ELF support, art's been cranking on a different debugger and much more), which will in turn allow for more dramatic events (like a new architecture) to happen. I'm sure that we can expect the same with SMP.

          The information is out there, and it's already pretty clear in what direction certain things are headed. As I see it, ELF and x86-64 support are going to be bigger items than SMP. On the other hand, after things like that are in... I can't really see anything [for _now_] that would be a bigger neat new cpu-oriented feature to work towards (well, maybe supporting PPC970's for future apples if that's the direction taken).

          Another way to break it down is to look at a wishlist of sorts (note this is maybe more mine, not a developer wishlist) and then rank things as far as likelihood:

          ELF for x86: likely (imminent actually, private work has been cranking along for a while)

          Newer gcc: likely, though this might cause some headaches with all the propolice work we've done (although propolice has already worked with gcc 3.2 from other reports, we'll see how many bumps there are).

          x86-64 support: (Just look at Theo's comments here, even he's excited about this architecture) extremely likely - but certain things need to happen first.

          PPC970 support: well, once Apple releases hardware, we'll see if that's what happens and when - time frame unknown (next year?).

          SMP support: someday - I think everything else might take precidence.

          OpenCM: pretty unlikely (it's a pig by all accounts), it will likely need to have major improvements on its own before we see it.

          TenDRA: wouldn't it be nice to dump as much GNU as possible? Unfortunately, others tell me that I am hitting the crack pipe too hard (and they're very likely right in this case). It would still be neat, but this is ages out if ever - it needs to be functionally equivalent to gcc for all our architectures, and we'd have to deal with propolice and tendra intregration, and on and on.

          There's a -lot- more out there too, but those are just some bigger things that have been popular on deadly in the past half year. You'll notice that some are recurring themes (like SMP) whereas others are pretty new (x86-64). I'm hoping that at Theo's talk at CanSecWest (SIGN UP AT www.cansecwest.com ) next week he will take a pretty long look, not just at the cool stuff from 3.3 that -current users have been appreciating, not just at things that will be nailed by 3.4, but even beyond. Of course, with any kind of future looking (even if you're the head of the project) you can't really state anything with certainty, or give hard and fast dates, but I wonder what else might be up his (and others') sleeve.

          Comments

          1. By Anonymous Coward () on

            Missed a big one. ACPI. Required for x86-64, and the Itaniums. It's a big win for many systems (APM and configure PCI busses).

          2. By Anonymous Coward () on

            It is called ``SMT'', NOT ``HT''. Hyperthreading is just some ugly marketing buzz name for Intels SMT-implementation in one ugly ISA.
            SMT is the correct definition for that technology.

            Also, the POWER4 from IBM does not feature SMT but CMP. It has multiples cores on one die, and multiple dies on one module (MCM).

            SMT was implemented in some of IBM's *Star CPUs (Northstar etc.), which were a POWER3 implementation IIRC.

            Comments

            1. By grey () on

              My apologies for using the vendor specific naming convention, I think the meaning was conveyed.

              More information (and likely more accurate) on future CPU roadmaps can be found here:
              http://www.realworldtech.com/page.cfm?AID=RWT012603224711

              And I do realize that currently IBM has only had multicore implementations in their power series, but if you refer to the article below you'll see that IBM is also working on SMT -and- multicore support in future Power hardware releases. Albeit, I don't think we'll see -that- in a consumer machine that OpenBSD would be likely to support anytime soon (at all?). Article is here:

              http://www.siliconstrategies.com/story/OEG20030224S0052

              With regards to the other poster mentioning lack of ACPI support, sorry I'm not omniscient, and I can't (and didn't) list every neat feature that could or should be added. Just a question though, does x86-64 really -require- ACPI?

            2. By Anonymous Coward () on

              IBM calls it H(ardware)M(ulti)T(hreading), this is a user selectable feature of a core, and multiple cores are in single module for long time

          3. By Alejandro Belluscio () baldusi@guesswho.com on mailto:baldusi@guesswho.com

            From what I recall from the mailing list the move from gcc 2.95 to 3.X was "over my dead body". But they do need it for the X86-64.

      2. By grey () on http://www.openbsd.org/crypto.html

        http://www.openbsd.org/crypto.html

        Depending on one's needs and what one buys - various supported hardware cryptographic accellerators are already A. working and B. more economical for this task.

        But, they're also dependant on the cipher. While AES chips have been trickling into the market for a while now - I doubt we'll ever see blowfish or twofish implemented in hardware (by the same token, *fish is pretty fast in software, then again AES had that as a design goal as well), so possibly SMP support as a generic crypto offload would be effective. But really, crypto support I think won't really benefit much from SMP, when a crypto board/chip could suffice more economically in many cases. SMP is not a panacea for throughput - however, it is a more generic way of making things speed up in hardware, when a specific hardware offering might not be available.

      3. By Henning () henning@openbsd.org on mailto:henning@openbsd.org

        dedicating a $$$$$ 2nd processor to do crypto is certainy braindead with a 92$ PCI card available doing the crypto stuff at least as fast an already beeing supported. man hifn.

    4. By Anonymous Coward () on

      Why clusters? Why SMP? - smtp is built with redundancy in mind. Better focus your effort on DNS setup, to use more webservers for purpose of www fun.

  4. By Anonymous Coward () on

    Is there a need, or simply a desire for SMP? In nearly all the articles I've ever read about where people are using Obsd, it will do just fine with as a single processor system in place. But maybe if SMP were supported, i'd see more articles about Obsd being used as an app or database server. Which is the cause, and which is the effect? hrmm...

    Comments

    1. By Nuintari () on

      Well, I bought a pair of nice IBM rackmounts to serve as DNS servers for my company, the guy threw in extra cpu's and ram for 50 bucks, so I took it, despite the fact that I knew they would be running OBSD. Someday, it would be nice to have dual p3 500's, but I don't really need it, just like I don't really need the gig of ram they each sport.

      But I want it! Gimme! Gimme! Gimme!

      Comments

      1. By Anonymous Coward () on

        Those processors will sure come in handy for running DNS. Yeah, right. Maybe if you get hit by a DDOS.

        Comments

        1. By Nuintari () on

          Well, they also do mx relays, and one has my cvs on it, I didn't pay the extra fifty for the cpu's, I bought it for the extra chunck of ram he also offered as part of the deal. The extra cpu time would sit idle unless I felt like compiling something. Like I said, I didn't need it, and I bought it for the ram.

          Comments

          1. By Beop () on

            I don't forsee any future for a Uni-CPU OS.

            Comments

            1. By Anonymous Coward () on

              There are lots of Uni-CPU machines, especially in Server segment

            2. By Nuintari () on

              Well, my own example is is one place where smp is just not needed. DNS isn't terriably cpu intensive, one will do just fine for all but the most ridiculous of servers. It is more of a memory eater than anything, and that is only for huge zone files. Mail server's aren't all that cpu intensive either, their bottleneck is disk I/O and network.

              Dynamic web servers can be cpu intense, but to get them cpu intnse, your probably either a really bad coder, writing everything in interpreted languages and making your scripts way too long, and/or your saturating the network, thereby, cpu time is sitting idle waiting for stuff to go through.

              Servers don't need much speed, they need bus bandwidth, network bandwidth, and memory, and of course, disk space.

              Now, workstations, for serious crunchwork, yeah, I would want a sun e450 for my workstation, quad cpu nightmare. But I still don't need it. ALmost no one ever puts dual and quag systems to good use.

              Sorry, not spellchecking that.

              Comments

              1. By Beop () on

                Even Low/mid end servers in the marketplace are smp these days, dudes.

                Comments

                1. By Anonymous Coward () on

                  Optionally dude

            3. By Stefan () on


              ... and I forsee the time of reckoning is upon us.*

              ... well, if I'm not mistaken it is one
              of the advantages of OpenBSD that it runs on
              rather modest hardware - and a lot of people really
              appreciate that - don't have to run to the store
              when another kind of "multi-matrix-psionic-enhanced"-whatever
              CPU comes out.

              cheers
              stefan

              * sorry, couldn't resist.

  5. By Anonymous Coward () on

    ...use Linux (or FreeBSD). I am not trolling here. I use and support (CD's, shirts, posters) OpenBSD, but SMP is not a goal of Theo's. Linux has a lot of support from hardware companies and others for SMP and FreeBSD is always an option. It is simply a matter of the right tool for the right job. I'm not going to run the Myrinet cluster in my basement on OpenBSD and I'm not going to run my firewall on anything but OpenBSD. Theo isn't trying to make OpenBSD everything to everybody and it is damn good at what it does do.

  6. By Cyrus Yunker () cyrus@80d.org on http://80d.org

    After reading the following, I was struck by an idea.


    From: Niklas Hallqvist
    To: smp
    Subject: Status
    Date: Fri, 21 Mar 2003 14:37:24 +0100
    ...
    * All the working code there is, is in the SMP branch. It is only
    i386-specific parts, no machine-independent parts have been worked
    upon. The code manages to detect and spinup a 2nd processor, but
    not let it do any useful work.
    ...

    I'm guessing that most developers see this project as much to large of a bite to swallow, and nobody really knows where to start, since the code affects so many subsystems. People realize the work is tedious and are worried about breaking things, exposing the system to security flaws, etc.

    Rather than attacking the whole project (blank page syndrome), start by getting the other processor fired up and give it little jobs, integrating into only specific parts of the system that people can manage.

    I'd suggest, for starters, just using additional processors to offload crypto work, manage filesystems, or do packet mangling. Rather than trying to integrate spinlocks and get another processor into every bit of work that's going on in a typical system, just make it availible for specific jobs, and speedup those.

    Ideas:
    Speedup encrypted filesystem operations
    ditto for encrypted swap
    let mod_gzip use the processor for web traffic
    key generation
    bzip2 module that could use the additional CPU

    Perhaps some sort of module system could be built, and the additional processor management code could be designed to manage only those plugins. A separate "thread namespace" could exist on the additional processor, almost completely separated from the regular system kernel and userland processes. System and userland processes could communicate with these modules via some sort of communication layer, allowing work to be offloaded to the, perhaps at times, very unbusy silicon.

    It's not full use of the system potential, but you'd have your OpenBSD security, and you'd be farther ahead than you are now.

    I'm not an OpenBSD developer and would be happy to hear about the faults with this idea. It's just something that popped into my head, and I hadn't seen it elsewhere, so I thought I'd put it out there to you all.

    Let the other work (fully MP enabling OpenBSD) be ported over as people become interested and want to spend the time going through the tedious work.

    Comments

    1. By Christian Gruber () cgruber@israfil.net on http://www.israfil.net

      Let's try to get things off of Art's TODO list and other such things. Let's do some of the internal structures re-work that is on the core developers' roadmap so that the internals are cleaner. Before you try to make something reentrant and/or heavily concurrent, you need to clean it up. The worst thing for a piece of software is to make it concurrent when it's messy. You will almost always slow things down, as you get into hard-to-predict, and hard-to-find deadlocks and hidden subtle mis-behavior.

  7. By Firdarrig () on

    ..."IDS". Intrusion Detection Systems are a natural match for OpenBSD: you want something to run snort on that you'll never have to worry about being compromised. Problem is, all that packet-handling takes processor power, the more so when you're running a manager for several snort engines, dumping their input into a database and running correlation against it. The more horsepower you can throw at a problem like that, the better.
    The idea about offloading smaller tasks to a second processor is neat, though, and kind of a unique angle: you can do that on Solaris using pbind, but it has to be manually initiated by the admin rather than automatically done by the OS. That could be super-useful on an IDS sensor node: dedicate one processor to receiving and queuing packets (including reassembly) and the other to matching them against a signature database and sending the results upstream to a manager. Hmm...

    Comments

    1. By Anonymous Coward () on

      SMP is about symmetric multiprocessing, this is execution threads you are asking for, and they cen be scheduled to different cpu-s (ideally)

  8. By Anonymous Coward () on

    So what does this mean on this page http://www.openbsd.org/plus.html

    "Sync the SMP branch to 3.3."

    SMP to come ?

    Comments

    1. By zil0g () on

      Not really, it just means they'l sync the SMP tree with the rest of the system, right now (or last time I looked at it) it was "OpenBSD 3.0-Current" and building the 3.2 userland on it was a bit of a pita =)

  9. By Ash'aman () on

    This issue keeps coming up and people keep whinging about SMP in OpenBSD.

    I'd much rather developers spent time doing all the great things that they've done like systrace, pf, propolice integration, privilege separation, chroot apache, spamd, minimising setuid binaries, hardware crypto support, than any of this smp garbage. It's so much work for so little reward.

    I'd much rather developers spent time on such things as improving the scsi subsystem, hardware support for onstream tape drives & some fibre channel adapters, porting to new sparc and amd systems and working on things like UFS2, documentation, and licensing.

    If OpenBSD had a team even half the size of say the team working on the linux kernel with the same injections of cash that Linux gets from IBM, Oracle, RedHat, etc., then I think that expectations of SMP in OpenBSD would be reasonable, but the fact is they don't. Even with DARPA funding, the priorities of the project should come first. That's why I use it, and that's why SMP wonks should use another OS if it's so important to them.




  10. By Iain () on mailto:ikyte(at)yahoo(dot)com

    First I am waiting for SMP so I can deploy OpenBSD as my developer OS and not need to use Linux/Windows so my second CPU can get a work out. But really I do say going SMP will be an entire Kernel Re-Write.

    To do this we need to continue to assemble the re-write team. Process and scheduling will need to be done. First is any one with the info on documents, white papers, thesis, ect., should assemble them on the SMP document page.

    Next we should do a Kernel Map. This will help isolate the locations to put the locks and add ons needed. On the side it will help with Code Walk Throughs and getting new Kernel Developers started.

    After the above the team could plan the strategy, who does what, when to test. When a prelinary version is ready we move to the larger test groups. After which it is then submitted for Code Review.

    If the core team accepts we then stamp OpenBSD version X.Y now has SMP.

    Question, Do we want to support one or two kernels. That is one kernel with SMP for Intel or two, one with SMP and one without?

    Iain

    Comments

    1. By Anonymous Coward () on

      <em>I am waiting for SMP so I can deploy OpenBSD as my developer OS and not need to use Linux/Windows so my second CPU can get a work out.</em> <br> <br> <p >Switch to FreeBSD 5. The SMP works. It is BSD so the system will be familiar to you.

  11. By Kyle Korndoerfer () on

    I run OpenBSD for the simple reasons that it is very secure with little effort, clean, and easy to update/maintain. I run a simple little web-server for myself and my wife to keep the rest of the family informed (pictures, news, etc.) as well as for learning/working with web technologies (DB's, PHP, XML, SOAP, etc.) and programming.

    I would like to see SMP for a simple reason: I can replace my aging P1-233MMX machine for free with a 2x200 Pentium Pro server from my employeer that has run it's course and is being decomishened. I would prefer to run OpenBSD for the reasons stated above and would prefer to have both CPU's being used.

    I know the responses will be to just setup another box, but space and money are both issues. OpenBSD runs great on older hardware that is cheap to find and use.

Latest Articles

Credits

Copyright © - Daniel Hartmeier. All rights reserved. Articles and comments are copyright their respective authors, submission implies license to publish on this web site. Contents of the archive prior to as well as images and HTML templates were copied from the fabulous original deadly.org with Jose's and Jim's kind permission. This journal runs as CGI with httpd(8) on OpenBSD, the source code is BSD licensed. undeadly \Un*dead"ly\, a. Not subject to death; immortal. [Obs.]