OpenBSD Journal

Progress on SMP

Contributed by jose on from the vroom-vroom dept.

jtorin writes: "Niklas Hallqvist writes in: http://marc.theaimsgroup.com/?l=openbsd-smp&m=107752035705159&w=2


Hi,

As some of you have seen there is once again SMP activity, this time I have good hopes
of getting to a point where the branch will be usable before I once again will be hit
my real life demands :-)  Genua is sponsoring me for this work, and it is very
appreciated.

The SMP branch now works for some i386 MP systems (the config to use is GENERIC.MP),
if booting multiuser and staying up for a few hours can be considered working :-)
There are a bunch of machine specific problems (e.g. motherboards with non-standard
interrupt routing), at least one more serious data corruption problem (meaning, do
not write to important files, if you have no way of recreating them) and there
is still missing all userland support (like ps and top support).

That said, the report Marco provided is for a build of the normal GENERIC kernel
without SMP support, of course those issues need to be fixed too, although it's a
low priority.  I don't even believe such a kernel will work, I have not tested it
at all.  I also doubt the MP kernel will work on UP systems, but you are free to test.
Don't expect me to react fast to report or questions on this, I rather keep
a low communicative profile, and instead work at a full pace, and I believe my
funder expects me to as well :-)

Happy hacking,
Niklas


(Comments are closed)


Comments
  1. By Alejandro Belluscio () baldusi@hotmail.com on mailto:baldusi@hotmail.com

    I wonder why they don't set up an account that's easy to donate to, but that's specific to certain projects. Call it a tipjar or a bounty. I'm sure LDAP, SMP and Heimendal and 802.11X framework would find a lot of supportes. And I mean supporting with $$$. I know I would.
    I think this would attract more donors. I know I make a donation of money because of the Hackathlon case and some WiFi hardware because I actually wanted to use it.

    Comments
    1. By Anonymous Coward () on

      I would help donate too, not that I have much to offer but I'd do my best.

      I think this is great! What ever happened to spinlocks though?

      Comments
      1. By Anonymous Coward () on

        maybe they got a girlfriend or something

        Comments
        1. By Anonymous Coward () on

          ROFL!

        2. By Peter Hessler () spambox@theapt.org on http://www.theapt.org

          Niklas is Married with Children

          Comments
          1. By henning () on

            he's married with children...? lasy time i checked he was swedish and not belgian...

      2. By Anonymous Coward () on

        Not that this was the point here, but I would personally enjoy being able to donate via the Amazon Honor System: http://s1.amazon.com/exec/varzea/subst/fx/help/payor-faq.html/ref=zm_pb_h_05/058-5482927-8158660

        At least Freeciv uses it...

      3. By schnick () on

        I tried to track down the spinlocks guys here in Waterloo and was unsuccessful. I'm guessing they failed their project ;)

        Comments
        1. By Dan () spam@datasquid.net on mailto:spam@datasquid.net

          actually, it was quite successful. i was in their class. it wasn't full SMP support they worked on, however.

    2. By Chad Loder () on

      I wonder why they don't set up an account that's easy to donate to, but that's specific to certain projects.

      If you [all] want a feature implemented and you're willing to pay for it, then you should pool your money and hire a qualified developer to do it. If you can't pool enough money to hire a qualified developer, then what is a "tip jar" with those insufficient funds going to accomplish?

      There are only a handful of project members who can do meaningful work on, say, SMP, and they all have day jobs. Having a pittance in a tip jar is not going to give them more hours in the day.

      If you allow users to segment their donations into separate jars that can only be used for their favorite features, the result will be 50 jars, each filled with insufficient funds to accomplish anything. The proliferation of special jars would probably fragment donations to the point where the general purpose fund gets depleted -- and basic expenses like bandwidth, equipment, travel, etc. could not be paid for.

      Comments
      1. By Henrik Holmboe () on

        Obviously you havent thought about the fact that many potential donors dont actually have any contact? Even if they are after the same feature, doesnt necesarilly mean that they can pool together.

        And a tipjar can raise enough money over time (from smaller donors, perhaps).

        Perhaps the tipjar function should allow the donor to specify multiple targets for the donation, so if it isnt used for feature X it would wind up being used for other expenses.

        Just a thought.

        .h

        Comments
        1. By Chad Loder () on

          I disagree. This "tip jar" suggestion comes up every 6 months or so. A lot of people come out of the woodwork and talk about how they will donate or WOULD donate if only such a system existed. Then after a few hours they forget all about it and they never donate a penny.

          People who are actually going to donate will do it (or have done it) regardless of their favorite projects. Those who never intend to donate will simply talk about it until something else grabs their attention.

          Comments
          1. By Alejandro Belluscio () baldusi@hotmail.com on mailto:baldusi@hotmail.com

            But what I said is not contrary to your vision. But coming to think about my personal behaviour I thought that having a "candy" and some "call to arms" is a good strategy. Let's say, some linux Guy reverse engineers the Broadcom WiFi drivers. You could post a mail saying that you need such and such hardware and some amount of money (you can't add time but you can compensate for extra hour time). I guess an interesting amount of the people that are compulsory donors (which you implied are the mayority) would go for it. Then you do the SMP showcase of advancement and ask for something different. And so on. It's just a proposal of marketing. That's how fund raising usally works.
            I sure think it would be great if some really big donor (like billg who uses the OBSD utilities in its Unix Services :-) would make a donation big enough to make an endowment fund. You could also try to put some fraction of donations in there. That way you'd get a base "income" for the project. It's just standard foundation/non-profit organization.

            Comments
            1. By Anonymous Coward () on

              Are you living under a rock? Thats exactly what's happening. If any developer feels the need for hardware, its listed on http://www.openbsd.org/want.html

              And about "call to arms", there was one for amd64 hardware just this week. Those things happen.

              And the base income - what do you think the two releases every year are for, exept throwing out great software? Its for selling CD sets!

              Comments
              1. By Alejandro Belluscio () baldusi@hotmail.com on mailto:baldusi@hotmail.com

                You didn't get it, do yo?
                Yep they do ask. But an easy more specific way of contributing should be great. I've been buying the CDs since 2.7 and donating hardware and money whenever I can (not many since Argentina devaluated :-( .) But I don't consider myself the public that has to be added. I'm already a donor. This is for those that are impulsive. Besides, I've got an offer of a dual Pentium II BX platform turned down. So it's not like this guys actually make this to get this stuff.
                Besides, there's a huge difference between stating in a page your needs (that's for people like us) and going out to raise resources (that's for the mob.)
                And regarding the base income, there's a big difference. One thing is to hope that the sales help you. Sales may go well, may just let you break even, or may sunk. A good portfolio will have much less variance. And might even allow some developers to actually "work" for OpenBSD full time. Forever.

    3. By jtorin () spmkll on myrealbox point com on mailto:spmkll on myrealbox point com

      I have also been thinking about "bounties", but as others has pointed out in this thread, it might fragment the available resources to the point there isn't enough to cover the daily business (bandwidth, electricity, etc).

      I do however belive that if you clearly (and in short!) states what you want to prioritize in the OpenBSD project, the team can't do more than read your statement. They are definately *not* obliged to follow your plea, but they will get an idea of the general opinon. And as it is backed up with money, it is actually worth something (as opposed to the constant whining on misc@openbsd.org/deadly.org about the lack of feature X).

      Yes, I'm aware that the people that makes up the OpenBSD team pretty much choose whatever they want to work on, and how much. But still, it's the money from CD sales and donations that funds the game. Sure, the moment the money leaves my account it is the hands of the OpenBSD project and they can do whatever they damn wish with it. But remember, if they want more they have to atleast listen to the hands that feeds them.

      I have donated money before and I will do it again.

    4. By Anonymous Coward () on

      BSD IS DYING

  2. By van Rihn () on

    It works for some Motherboards?!
    Which boards?

    I've a INTEL SHG2.. I will try it ,)

    Comments
    1. By Isak Lyberth () isak@lyberth.dk on www.lyberth.dk

      Please report back on how it went.

  3. By j0rd () mits_rox@OHNOS.hotmail.com on http://j0rd.ath.cx/

    Glad to hear work is being done on SMP systems. I think opening obsd to SMP will increase userbase. It's the main reason i don't run obsd on my servers/workstations (besides lack of native java of course (which is comming))

    I'd be happy to test it and give feedback once something is working, and get gentoo off my main workstation at home.

    Comments
    1. By Chad Loder () on

      I'd be happy to test it and give feedback once something is working, and get gentoo off my main workstation at home. How does that help anyone?

      Comments
      1. By Anonymous Coward () on

        He is going to remove Gentoo and install OpenBSD/SMP on his main workstation (presumably an SMP one) to test it out. And then he'll give feedback.

    2. By Anonymous Coward () on

      Kurt Millar made a native port of the Java SDK (1.3.1) available some time ago for OBSD 3.4. So far I've only found a few niggles with it (mostly with applets not working right). For the most part native java is already here (just a few versions behind).

  4. By Nick B. () on

    I've got a couple of old AST Premmia GX boxes here with dual P133's. Hell yes I know they are old and they run with a single CPU but I can't find anything thats runs MP on them apart from Winnt 4!
    I'll gladly test a MP kernel.

    Comments
    1. By Anonymous Coward () on

      I had an old DEC dual p90's that were the exact same way, and I wouldn't get my hopes up. A lot of those older PI multiproc machine use really weird setups for their SMP that is completely undocumented, 'proprietary,' and only works with NT4 (presumably because they gave MS special drivers to ship with it to enable the second processor).

  5. By SoupIsGood Food () soupisgood_food at reallyfast dawt info on mailto:soupisgood_food at reallyfast dawt info

    SMP is a problem that has been solved by open-source software projects many, many, many times before. Linux, NetBSD, FreeBSD, Mach/Darwin, and even Solaris all offer solid, documented code examples of how to get SMP working on multiple platforms. The fact that OpenBSD can't get SMP up and running on x86, never mind Sparc, Alpha and PowerPC, after years of development work, is something of an embarrassment. It's getting to the point where servers and workstations are more commonly configured with two processors than one, so the lack of SMP support is a sore point. I've just deployed Gentoo on a new (used) Ultra2, because OpenBSD can't handle the extra proccessor.

    Something is clearly broken with the approach the developer(s) are currently taking... wheel re-invention is strongly suspected.

    SoupIsGood Food

    Comments
    1. By Anonymous Coward () on

      SMP isn't top priority in OpenBSD, it's Security and I for one am still very grateful at that...

      We know they can do SMP but before you rant, you might want to check http://www.openbsd.org/goals.html for yourself.

      But hey, thanks for volunteering to do some SMP work, glad to see another person come forth and offer a hand...

      When will you be starting? :-)

      Regards.

    2. By Anonymous Coward () on

      There have _not_ been years of active development. Read what Niklas is saying: the work has stalled for quite some time. Active work has started again only recently.

      As stated on many places, it is a question of setting priorities, avaliable resources etc etc.

      Comments
      1. By Jeff () on

        Here here...
        Obviously, Niklas is a fellow with other priorities that must be dealt with. I seem to recall that he might have a family or something like that..?

        I don't personally care much about SMP.
        Some people cannot afford fancy computer equipment ;)

        Anyway, this is all beside the point.
        Shouldn't we all just say, "Thank you Sir Niklas for helping the OpenBSD project. We all appreciate your work!"

    3. By Anonymous Coward () on

      I think Solaris is the only one with solid SMP support

      oh look at that ... it's not open source

      Comments
      1. By philipp () pb@ on mailto:pb@

        Clueless?

        HP-UX? AIX? IRIX?

        Those are not solid? Yeah, right.

        Go back under your rock.

        Comments
        1. By Luiz Gustavo () on


          Perhaps the same people who claims JFS is rock solid five nines file system...
          At least I know how to use fsdb when needed.
          (;

        2. By Anonymous Coward () on

          HP-UX: closed source
          AIX: closed source
          IRIX: closed source

          His point was that all stable/mature SMP OS's are closed source and thus we can't see how it was done. Perhaps you can name a stable/mature open source implementation?

          This stuff is hard, especially when you don't have control over the hardware, unlike Sun, HP, IBM, SGI, etc.

          Comments
          1. By henning () on

            amazing how much bullshit is talked here.
            it is not like SMP is a two files device driver you can port over from some other OS...

        3. By Anonymous Coward () on

          HPUX is not rock solid from my experience. In fact it is a crock of shit.

          How do I know?

          I administer 2x 4-way HP N-class servers running Oracle.

          The hardware and HPUX is a shit crock from hell. I mean the FC interface spams the SAN and even HP don't know how to stop it.

          Comments
          1. By ciph3r () on

            HP-UX is known for it's poor support in its Tachyon FC controllers. We have a master server and a media server backup with them and they suffer every day.

            It give us errors and it's not optimal but has benn solid until now.

        4. By Anonymous Coward () on

          > HP-UX? AIX? IRIX?
          >
          > Those are not solid? Yeah, right.

          Those were not any of the OSes listed... Solaris was the only closed-source OS listed.

    4. By pravus () on

      what is embarrassing is your lack of research on this topic. you say you just moved to Gentoo? good riddance.

      Comments
      1. By Anthony () on

        He'll be back. Gentoo is too annoying. "Stable" gets compile errors for weeks at a time sometimes...

    5. By Anthony () on

      It's not the SMP servers and workstations that concern me. The big problem is Intel HT CPUs. Even mid range computers now need an SMP OS to take full advantage of the hardware. Other companies are moving towards multi-core or multi-threaded designs as well, so it seems it's something we'll have to deal with sooner or later.

      The development process is not broken. In fact, NetBSD's SMP support is serving as a guide. OpenBSD forked from NetBSD, so they're very closely related.

      The lack of SMP support is due to it being a low priority. That priority will increase over time as SMP support becomes more important in an OS.

      Lack of SMP support isn't the only reason to avoid OpenBSD. There's plenty of other things it's bad at. I makes for a convincing argument to use the OS best for the job, doesn't it?

    6. By Chad Loder () on

      :)

    7. By tedu () on

      whiner without a clue about smp development process strongly suspected.

    8. By Ben () mouring@eviladmin.org on mailto:mouring@eviladmin.org

      It does not matter how much SMP code exists. If you don't have a MP safe kernel you'll be having race conditions all over. Taking code and making it MP( or Thread safe for that matter) is always dangerous and never an easy task.

      Retraining programmers to consider the MP/Thread cases also takes time. "Is this function re-entrant?" becomes the question of the day.

      Some code can be borrowed from NetBSD, but in the end it boils down to fixing what is in the tree.

      Comments
      1. By ciph3r () on

        NetBSD SMP code it's right now "pre-Big-Lock removal".

        Maybe we can see what has been done in FreeBSD or even better, adapt the great job Matt Dillon is doing in DragonFlyBSD once it'll get stable.

        Comments
        1. By Ben Lindstrom () mouring@eviladmin.org on mailto:mouring@eviladmin.org

          Still requires manpower and understanding how to weed out race conditions.

          Comments
          1. By ciph3r () on

            Of course. I was only pointing to Open Source (and BSD licensed) implementations out there :)

    9. By Anonymous Coward () on

      BSD IS DYING

      Comments
      1. By krh () on

        Thank you. It's rare that I see a post that makes me laugh that hard.

      2. By Anonymous Coward () on

        Oh great, a /. troll rolled in on his training wheels...

      3. By Anthony () on

        Guess he didn't read the part about posting IP addresses.

    10. By Marc Espie () espie@openbsd.org on mailto:espie@openbsd.org

      Nice Troll.

      You don't know anything about how the project worksm nor about SMP development.

  6. By Michael van der Westhuizen () on

    Niklas is working on getting us to a state where we have working SMP (even if it is with biglock only) - from there on out we can go through the same lifecycle that the other BSDs have: i.e. make subsystems MP (or HT?) safe one at a time, using finer grained locks.

    Yet all but a few comments I've read here are negative.... why?

    Why not just offer some (at least verbal) support for the huge job he's undertaking.

    (a side note to the guy who spoke about maybe lifting code from NetBSD or FreeBSD: nice try, it really doesn't work like that... try it).

    Comments
    1. By Anonymous Coward () on

      I think most of them were positive with a few negative. I'm very grateful of OpenBSD itself and any SMP work is a huge bonus!

      My many thanks to all those who make OpenBSD what it is and will be and my many thanks to Niklas personally for his time and effort!

      Regards.

    2. By Anonymous Coward () on

      I'll throw in a positive comment. This would be great. Servers tend to be SMP these days. However, I get the impression that it is very difficult to create a good SMP system, and it takes years of tuning and work to get it working optimally. Linux is much closer now with 2.6. It may take OpenBSD a while, but it's great that it's starting and that means more stuff to look forward to in future releases.

Latest Articles

Credits

Copyright © - Daniel Hartmeier. All rights reserved. Articles and comments are copyright their respective authors, submission implies license to publish on this web site. Contents of the archive prior to as well as images and HTML templates were copied from the fabulous original deadly.org with Jose's and Jim's kind permission. This journal runs as CGI with httpd(8) on OpenBSD, the source code is BSD licensed. undeadly \Un*dead"ly\, a. Not subject to death; immortal. [Obs.]