OpenBSD Journal

Rthreads Hackathon Part the Epilogue

Contributed by tbert on from the once-more-with-feeling dept.

Ted Unangst(tedu@), the original author of rthreads (you can watch his DCBSDCon09 presentation on youtube), recently took the time to answer some of our questions.

OpenBSD Journal: You initially began working on rthreads due to performance issues in the existing uthreads implementation. Why did the userland-only threading library last so long?

tedu@: It was almost good enough? I think also the amount of work involved in switching to kernel threads, which even though we've tried to keep it simple is still a lot, slowed things down. For many programs, rthreads will be an incremental improvement at best, and it's a lot of work. I'll get to this more in a bit, but lots of programs just use threads as a state machine abstraction, and uthreads were well suited to that task.

OpenBSD Journal: There's been discussion by the OpenBSD developers that, especially in the realm of networked daemons, threading is an inferior solution to asynchronous solutions. Where do you come down in that debate?

tedu@: If you're pushing traffic in and out, without much processing, I would agree. If you're spending time on the CPU, however, threads will let you use more CPUs. And if you're doing disk IO, you can't do eventing, because the file system layer just doesn't (in OpenBSD) support async ops.

There's also some debate about the preference for threads or multiprocess. I think they amount to about the same thing. And if you've designed your program nicely, you won't even know if your message passing interface is a pipe between processes or a shared memory queue between threads. You could even swap one backend for another. Threads are a little easier to work with for heterogenous tasks and small scale (think desktop), while multiprocess readily becomes multimachine if you want to go all web scale.

I would also like to say that if you go the async/evented route, you really have to know what you're doing. There are blocking APIs *everywhere* that you must call with caution, if ever. I think the people writing web services with things like node.js are rediscovering this, over and over. Just as with threading, it's easy to write code that mostly seems to work, until it doesn't.

OpenBSD Journal: There has seemingly been a rise in the use of threads over the past decade or so; is that an illusion, or is it real? And, if real, is it something to be embraced, or merely endured?

tedu@: Rise of multicore? As a user, if the program works, I don't care about the internals. To the extent shitty programs use threads, I don't think the shit is a direct consequence of threading.

OpenBSD Journal: Is there a logical follow-on to the work being done on finishing rthreads? Is there anything that is now possible which wasn't before?

tedu@: I think there is going to be some performance work necessary. Other than things working better, I'm not sure anything is more "possible". Maybe java becomes more viable for some people.

In terms of some lessons learned: When I gave the original rthreads presentation way back in 2005, part of the talk focused on the fact that we were going with a 1:1 threading model because it was simpler. Developers from both NetBSD and FreeBSD came up to me and explained how we would regret that choice, and the M:N models they were both building were so much better. Well, today they have either entirely scrapped (NetBSD) or mostly sidelined (FreeBSD) the M:N threading systems in favor of 1:1 models.

I can't laugh too hard, because they still shipped first and I had to be carried across the finish line on the backs of others, but it's worth reflecting on the OpenBSD development philosophy. Try to solve the real problem you have, not problems you imagine someone else having. Also, I think the simple model is precisely what's allowed other people to step in and finish rthreads. M:N with scheduler activations is just so hairy, it doesn't attract developers to fix bugs, it attracts developers to rewrite everything as 1:1.

OpenBSD Journal: What did you think of The Avengers?

tedu@: ok, so the hulk in the air was out of control, but then down on the ground he's like, it's all cool? How did that happen? They made it seem like he faked the whole out of control thing, but in that case why'd he go tearing the shit out of the place?

Truly, a question for the ages.

(Comments are closed)


Comments
  1. By Marc Espie (espie) espie@nerim.net on

    I've been using "real" threads ten years ago to do fast matrix computations on a sun.
    Basically, anything that has to deal with a lot of data and do several computations on them more or less simultaneously can benefit from threads, if coded carefully.

    This includes video encoding... which is probably the only application left (besides games) that can make use of more processor power (as opposed to more memory bandwidth, which is another issue entirely).

    Of course, with the cloud, everyone will focus on webscale and similar shit. But hey, all the world is NOT network applications only. OpenBSD on the desktop WANTS efficient vlc/mencoder/ffmpeg. Heck, even gstreamer. All that can benefit from efficient threads.

    Comments
    1. By tbert (tbert) on

      Yes, but what did you think of The Avengers?

      Comments
      1. By Gilles Chehade (gilles) on https://www.poolp.org/~gilles/

        > Yes, but what did you think of The Avengers?
        >

        It is because of people like you that Hollywood comes up with movies such as "Cowboys VS Aliens" ... YES, YOU.

        Comments
        1. By tbert (tbert) on

          > > Yes, but what did you think of The Avengers?
          > >
          >
          > It is because of people like you that Hollywood comes up with movies such as "Cowboys VS Aliens" ... YES, YOU.

          Stop making eye contact.

        2. By Miod Vallat (miod) on

          > It is because of people like you that Hollywood comes up with movies such as "Cowboys VS Aliens" ... YES, YOU.

          It's a shame the movie has almost nothing in common with the comic book, which was much better. (It used to be available on Drunk Duck, but after it was made into a movie the pages were removed and the sequel, still in comic book format, was interrupted. What a shame)

      2. By Marc Espie (espie) on

        > Yes, but what did you think of The Avengers?
        >

        First, I enjoyed being a filthy european and getting it a week before the stupid americans, for once. So it's almost old history for us by now, while you still enjoy the buzz.

        Apart from that, well, I've been a fan of Joss Whedon (as a screen writer) ever since BtVS, so it delivered in spades. It was very cool to see so much crammed on the screen without a feeling they tried to cram that much (compare to watchmen).

        So what if there were blatant inconsistencies ? I would still be hard-pressed to say which character I enjoyed more, between the Black Widow, Loki, and Iron Man. Yep the others are still good, but these 3 totally steal the movie for me.

        Comments
        1. By tbert (tbert) on

          > First, I enjoyed being a filthy european and getting it a week before the stupid americans, for once. So it's almost old history for us by now, while you still enjoy the buzz.

          Mr. Espie, you forget I've gone native: Min brannmann lukter grevling!

          Comments
          1. By Gilles Chehade (gilles) on https://www.poolp.org/~gilles/

            > > First, I enjoyed being a filthy european and getting it a week before the stupid americans, for once. So it's almost old history for us by now, while you still enjoy the buzz.
            >
            > Mr. Espie, you forget I've gone native: Min brannmann lukter grevling!

            You're still an american to us, don't worry ;-)

            Comments
            1. By tbert (tbert) on

              > > > First, I enjoyed being a filthy european and getting it a week before the stupid americans, for once. So it's almost old history for us by now, while you still enjoy the buzz.
              > >
              > > Mr. Espie, you forget I've gone native: Min brannmann lukter grevling!
              >
              > You're still an american to us, don't worry ;-)

              But all non-Parisians are Americans to you.

              Comments
              1. By Marc Espie (espie) on

                > > > > First, I enjoyed being a filthy european and getting it a week before the stupid americans, for once. So it's almost old history for us by now, while you still enjoy the buzz.
                > > >
                > > > Mr. Espie, you forget I've gone native: Min brannmann lukter grevling!
                > >
                > > You're still an american to us, don't worry ;-)
                >
                > But all non-Parisians are Americans to you.

                We have noticed your recent attempts at rudeness.

                As if you thought "artificial rudeness" would qualify YOU as a frenchman.

                Ah! it's more difficult than that.

                Do you even eat smelly cheese ?

                Comments
                1. By tbert (tbert) on

                  > We have noticed your recent attempts at rudeness.
                  >
                  > As if you thought "artificial rudeness" would qualify YOU as a frenchman.
                  >
                  > Ah! it's more difficult than that.
                  >
                  > Do you even eat smelly cheese ?

                  1) My rudeness is 100% authentic, you cheese-eating surrender monkey.

                  2) I never claimed to be a Frenchman, only a member of the society that stole your women for several centuries.

                  3) I prefer smelly fish.

                  Comments

Credits

Copyright © - Daniel Hartmeier. All rights reserved. Articles and comments are copyright their respective authors, submission implies license to publish on this web site. Contents of the archive prior to as well as images and HTML templates were copied from the fabulous original deadly.org with Jose's and Jim's kind permission. This journal runs as CGI with httpd(8) on OpenBSD, the source code is BSD licensed. undeadly \Un*dead"ly\, a. Not subject to death; immortal. [Obs.]