OpenBSD Journal

i386 goes ELF

Contributed by jose on from the milestone dept.

tony was one of many to write us about this:
"The i386 branch recently went ELF, complementing the W^X work.

If you wish to upgrade, you must do so by installing a binary snapshot.. upgrading from source won't work.

You can find the original email here . For a _very_ detailed description of W^X on i386, check out the follow-up email ."

Having done the a.out to ELF upgrade on a SPARC, yes, it's just easiest to reinstall from scratch. All sorts of things will stop working otherwise, and it's easiest to just do a wipe and reinstall. The snapshots are up and making the rounds of the mirrors, so if you feel daring, download and test stuff out.

(Comments are closed)


Comments
  1. By Anonymous Coward () on

    Looks like I'm gonna buy 3.4 then! Good job guys.

  2. By Dries Schellekens () on http://marc.theaimsgroup.com/?l=openbsd-misc&m=105

    Theo also answer how the recent OpenBSD changes (ProPolice, W^X, non-exec stack) compare to PaX. This was discussed in the comments about Theo's CanSecWest presentation: http://64.90.164.50/commentShow.php3?sid=20030413010250&pid=467

    Comments
    1. By Dries Schellekens () on

      Sorry URL was to long: http://marc.theaimsgroup.com/?l=openbsd-misc&m=105053858319166&w=2

      Comments
      1. By Anonymous Coward () on

        Perhaps you should read this, before repeating Theo's false comments as fact:
        http://marc.theaimsgroup.com/?l=openbsd-misc&m=105058898328497&w=2

        Comments
        1. By Dries Schellekens () on

          I'm waiting for Theo to reply on this. And yes I read this (but after I posted it here).

          Comments
          1. By Anonymous Coward () on

            Good idea. Put your independent thinking on hold and wait for Theo the hero to tell you what to think. You would have believed him if a reply wasn't posted. You'll believe him when he posts a reply to this as well.

            Comments
            1. By Dries Schellekens () on

              Sorry, last time I check PaX only had PAGEEXEC and Theo's claims are correct (low TLB software loading). I just printed out the new PaX documentation and I'm reading it right now. Apparently I read http://pageexec.virtualave.net/docs/pageexec.old.txt, which is heavily out dated at the moment.

              Comments
              1. By Anonymous Coward () on

                So you checked a year ago. I checked OpenBSD a year ago and it had tons of holes. By your logic, those holes still exist, and I should continue to talk about them in public as fact. Theo used his selective memory to attempt to discredit PaX. Theo would have better spent the time it took to write the lengthy mail in doing a minimal amount of research. Considering Theo's ego, this was perhaps too much to expect.

                Comments
                1. By Anonymous Coward () on

                  It sounds to me like OpenBSD is upsetting you because you think that it's stealing your ideas. Why not say so? Why not say, "I believe that PaX originated many of the ideas that OpenBSD has now reimplemented and deserves credit for inspiring OpenBSD." This business of attacking OpenBSD, especially Theo because he is its most prominent member, is juvenile. If you have a complaint, say it!

                  Comments
                  1. By Dries Schellekens () on

                    Theo claims the W^X idea was not inspired by PaX:
                    Considering we'd never heard of PaX before we did W^X.

                    You seem to think OpenBSD just copied PaX as your website states:
                    PaX features are part of OpenBSD (MagicPoint presentation)
                    non-executable stack pages based on the segmentation logic implemented by the Openwall Project
                    non-executable stack and heap pages based on the segmentation logic implemented in RSX
                    non-executable stack and heap pages based on the segmentation logic implemented in kNoX

                    You make a clear distinguish between OpenBSD and other non-exec stack/heap implementations!

                    Comments
                    1. By PaX Team () on

                      > Considering we'd never heard of PaX before we did W^X.

                      This is a lie. I've posted one proof on the mailing list, and in the meantime i got word that he had been made aware of PaX at HAL2001 as well (that's August 2001). If you want to prove me wrong (i'm not accusing, just suspecting), please make your private development mailing list archives available to the public, you should have nothing to hide.

                      > You seem to think OpenBSD just copied PaX as
                      > your website states:

                      What's wrong with this (you realize i talk about PaX *features*, not PaX per se)? Didn't PaX have non-exec page semantics 2 years before OpenBSD? Didn't PaX have randomization that OpenBSD is now considering (not to mention the strange coincidence of you guys having stack randomization one month after it was released in PaX, after HAL2001 too)? Again, making your development process open to the public would allow to clear up any confusions one may have.

                      As for the distinction between OpenBSD and the others: they predate PaX (as far as the segmentation based non-exec feature is concerned), you're not suggesting to put claims on them, are you?

                      Comments
                      1. By Dries Schellekens () on

                        > > Considering we'd never heard of PaX before we did W^X.

                        > This is a lie. I've posted one proof on the mailing list,

                        To me http://marc.theaimsgroup.com/?l=openbsd-tech&m=97416533704634&w=2
                        nothing. OK, you can assume someone he heard the name. At that moment OpenBSD developers totally weren't interested at implementing something similar. So I doubt they looked hard at it (perhaps just briefly). What did PaX do at that moment? Non-exec stack and heap on i386, or more features (like randomization)?

                        > Didn't PaX have non-exec page semantics 2 years before OpenBSD?

                        Didn't Solar Designer implement a non-executable stack on Linux several years before PaX? Doesn't Solaris have noexec_user_stack many years before PaX? The PaX team didn't invent this.

                        > Didn't PaX have randomization that OpenBSD is now considering (not to mention the strange coincidence of you guys having stack randomization one month after it was released in PaX, after HAL2001 too)?

                        So art@'s stackgap_random is inspired by PaX (if PaX had this before Augustus 16, 2001; I'm not able to verify this). Perhaps this was the only new thing (invented by the PaX team), that could be easily implemented in OpenBSD. At this moment OpenBSD developers didn't think non exec stack was a good idea. I doubt you had randomization of everything, non exec paging based on segmentations, mprotect restrictions, ... at that moment.

                        I believe Theo when he says when they started thinking about W^X, they didn't look at PaX; they had forgotten the name and didn't bother to look at the progress you made since HAL2001. Ofcourse this is pure speculation of my part.

                        > Again, making your development process open to the public would allow to clear up any confusions one may have.

                        You guys are good at this too. No CVS, no mailing list archives, no changelog, ...

                        Comments
                        1. By Anonymous Coward () on

                          > What did PaX do at that moment? Non-exec stack
                          > and heap on i386, or more features (like
                          > randomization)?

                          What we call PAGEEXEC/MPROTECT. The randomization came late July 2001.

                          > The PaX team didn't invent this.

                          You're shooting at the wrong target. Noone talks about invention here (for all i know, little in PaX can actually be said to be 'first' of its kind, maybe full ASLR and KERNEXEC of late, not that it mattered though). What we are talking about is where OpenBSD got the inspiration, you guys should come clean on this (you don't want to seriously sell the idea that a security conscious OS development crew doesn't follow the scene at large...).

                          > So art@'s stackgap_random is inspired by PaX

                          We don't know, but you see, if OpenBSD didn't consider Intrusion Protection/Prevention mechanisms at that time (you said so), how come this one came out of the blue all of a sudden? How come that after almost two years it still hasn't evolved into full randomization? I say it's fishy but i'll happily accept your version of the story as soon as you provide it (from a credible source, not 'my word against your word' style).

                          > Perhaps this was the only new thing (invented
                          > by the PaX team)

                          Wrong, as i said above, few things can be attributed to us as 'invention', nor does it matter. As far as i could track it back, the stack randomization idea popped up sometime 1997 (but i'm sure it's older), there's even one US patent issued on it (it has prior art too from the same year).

                          > At this moment OpenBSD developers didn't think
                          > non exec stack was a good idea.

                          And they still don't and they're right. Nor has PaX ever been a non-exec stack implementation. So what point did you try to make here?

                          > I doubt you had randomization of everything,

                          Wrong, when ASLR was first released, it already provided *full* randomization of the address space, nothing changed since.

                          > non exec paging based on segmentations,

                          Correct, it came in September 2002.

                          > mprotect restrictions, ... at that moment.

                          Wrong, it was included in November 2000.

                          > Ofcourse this is pure speculation of my part.

                          If what you speculate on is true then it sheds some bad light on your developers as they failed to follow the security scene (being security conscious would have one believe that it's an important thing to you).

                          > You guys are good at this too. No CVS,

                          http://www.grsecurity.net/cvs.php
                          http://cvsweb.grsecurity.net/index.cgi/

                          > no mailing list archives,

                          http://www.grsecurity.net/mailinglist.php

                          > no changelog

                          You can generate it from the CVS.

                          Now, where did you say your archives were again?

                          Comments
                          1. By Dries Schellekens () on

                            I'm not a OpenBSD developer, just a happy user. As I said, I just speculated about what happened. As I'm no developer, I'm not able to answer your questions.

                            If you want the exact timeline of events, ask for it on the official OpenBSD mailing list; Theo doesn't read deadly.org (some developers do). Or talk to him on some conference...

                            Thanks for pointing me to the CVS. I didn't know PaX was developed in the grsecurity tree. It's not stated on your main website: http://pageexec.virtualave.net/

                            Comments
                            1. By Anonymous Coward () on

                              He did ask. Read the mailing list.

                            2. By PaX Team () on

                              > ask for it on the official OpenBSD mailing list

                              http://marc.theaimsgroup.com/?l=openbsd-misc&m=105060386219963&w=2

                              It's left unanswered (no private mails either).

                              > I didn't know PaX was developed in the grsecurity tree.

                              That's because it is not. grsecurity graciously provides CVS hosting to PaX, but they've got their own trees with their own PaX in there (there was some integration work to do).

                              > It's not stated on your main website

                              This is for reasons beyond my control, i can't explain it in public, sorry. If and when the situation changes (or i lose my patience), i'll announce it on our site as well. The CVS has been and will stay open to the public regardless.

                          2. By Dries Schellekens () on

                            > We don't know, but you see, if OpenBSD didn't consider Intrusion Protection/Prevention mechanisms at that time (you said so), how come this one came out of the blue all of a sudden?

                            I don't know, perhaps because of the remote exploit for OpenSSH.

                            > How come that after almost two years it still hasn't evolved into full randomization?

                            Because the evolution started only recently.
                            Non exec pages happened in the summer of 2002. The W^X stuff got in the tree end 2002/begin 2003. This week W^X became possible on i386 and macppc still has to come. Complete randomization (ASLR) is on the future work list, and to Theo this seems a natural evolution from W^X.

                            The addition of stackgap_random was just an isolated event. Much like frantzen's stackghost code for sparc (http://stackghost.cerias.purdue.edu/).

                            I think the work started only recently (summer of 2002) and is slowly evolving to something comparable to PaX, but with other design choices, mainly because of different interpretations of POSIX standards (BTW you haven't commented on miod@'s mail on misc@).

                            > > At this moment OpenBSD developers didn't think non exec stack was a good idea.
                            >
                            > And they still don't and they're right. Nor has PaX ever been a non-exec stack implementation. So what point did you try to make here?

                            I meant that at that moment OpenBSD developers didn't intend to implement a non exec pages (stack+heap).

                            > > non exec paging based on segmentations,
                            >
                            > Correct, it came in September 2002.

                            At this moment OpenBSD had already designed there one non exec stack ugly hack on i386.

                            Comments
                            1. By PaX Team () on

                              > I don't know, perhaps because of the remote exploit for OpenSSH.

                              That was in June 2002, HAL2001 and OpenBSD stack randomization were in August 2001.

                              > The addition of stackgap_random was just an isolated event.

                              That doesn't answer the simple question i asked: where did it come from (you'll see from my mailing post that i'm working on some 'history' docs, hence it'd be nice to know more)? You still haven't pointed us to the real source: your private development communications that would clear things up.

                              > Complete randomization (ASLR) is on the future
                              > work list, and to Theo this seems a natural
                              > evolution from W^X.

                              You are wrong on this, randomization is 'orthogonal' to non-exec pages, that is, they can be independently developed and deployed. Your history notes are interesting (even if i knew it already) but irrelevant to my question (why not evolve into full ASLR during all this time if stack randomization was deemed important enough to get into the tree so long ago).

                              > (BTW you haven't commented on miod@'s mail on misc@).

                              If you mean his last one, i did comment on it, but in private. If you followed and understood that thread of discussion, it should be clear that POSIX (in the OpenBSD interpretation) mandates EACCES if PROT_EXEC was requested in that situation whereas OpenBSD (and the others too for that matter) fail only for PROT_READ (this is in the code snippet i had posted beforehand). So the conclusion to any reasonable person is crystal clear: OpenBSD violates POSIX. Since Theo went apparently incommunicado (spelling?) i saw no reason to bother him or the others with this, if noone cared so far, unlikely they would all of a sudden. I hope they learned a lesson though: don't talk trash about PaX when it's not true and they have a problem themselves.

                              Comments
                              1. By Dries Schellekens () on

                                > > The addition of stackgap_random was just an isolated event.
                                >
                                > That doesn't answer the simple question i asked: where did it come from? You still haven't pointed us to the real source: your private development communications that would clear things up.

                                Send an email to art@ and you will know.

                                > > Complete randomization (ASLR) is on the future work list, and to Theo this seems a natural evolution from W^X.
                                >
                                > You are wrong on this, randomization is 'orthogonal' to non-exec pages, that is, they can be independently developed and deployed.

                                I'm under impression that the planned randomization ( Randomized mapping for shared objects, Try to find a way to map the main program randomly, Randomized mmap ) requires ELF (as did W^X).

                                Comments
                                1. By PaX Team () on

                                  > Send an email to art@ and you will know.

                                  Maybe i didn't express myself clear enough: i'm interested in full public archives, not what an individual is willing to share, that is, it's not only for myself but for all. It's a pity the OpenBSD community goes so easily by this closed development process - i don't. If art or anyone else has anything to say on history, they'd better make it available to everyone. Besides, i made my request already clear on the misc@ list.

                                  > I'm under impression that the planned
                                  > randomization (Randomized mapping for shared
                                  > objects, Try to find a way to map the main
                                  > program randomly, Randomized mmap) requires ELF
                                  > (as did W^X).

                                  I thought a.out supported runtime relocation which is all you need for randomization. Other than that, even if in OpenBSD you need ELF support for both features, my claim stands that they can be implemented independently of each other (once you have the common infrastructure - ELF - in place). As a sidenote, PaX has had a.out non-exec pages since the beginning on i386 (although not really maintained or even tested as noone uses a.out on Linux), only OpenBSD's approach needs ELF.

              2. By Dries Schellekens () on

                I'm still reading the documentation. So you need a userland tool (chpax) to make existing programs (like XFree86, wine, java vm) work? Isn't this what mprotect() with PROT_WRITE|PROT_EXEC is for? Where does the POSIX standard says you have to hack the ELF header to be able to run your binary?

                BTW how are non-exec pages implemented on ppc? It's not documented yet (because it have only been added recently)?

                Comments
                1. By Anonymous Coward () on

                  Where does POSIX standard say OpenBSD has to use an arbitrary mquery call for the dynamic linker?

                  You answer my question.

                  Comments
                  1. By Dries Schellekens () on

                    So implementing non-exec pages is difficult. PaX has to some flags in the ELF header for some applications, while OpenBSD has to change its linker. *Personally* the latter seems better.

                    Now my question: how are non-exec pages implemented on ppc?

                    Comments
                    1. By Anonymous Coward () on

                      False.

                      This has nothing to do with the linker. Programs doing fixed mmaps will need to use this interface. PaX requires one second to change the flags on a binary. OpenBSD requires source modification. So much for binary compatibility.

                      Comments
                      1. By Dries Schellekens () on

                        Yes, OpenBSD requires programs not to be broken (properly using mprotect() when needing pages that are both PROT_WRITE and PROT_EXEC).

                        You still haven't answered my question.

                2. By PaX Team () on

                  The implementation of PaX features is documented on i386 only (you'll realize when you read the main doc, it says so). As for ppc: right now we have the equivalent of SEGMEXEC on it, that is, the address space is divided into two halves (not surprisingly the barrier falls on a 256MB boundary) and the lower half is non-executable and VMA mirroring ensures synchronicity. Due to some position independent code here and there in userland, there's some extra emulation needed, that is, until we can get to the userland part of the project where this can be fixed up and the ugly hacks removed from the kernel part. Next we'll try the equivalent of PAGEEXEC as well. If you have more questions, it's probably better to ask them directly in email.

            2. By tedu () on

              i'll just put my independent thinking on hold while you tell me what to think.

              while i'm switching over to PaX, which is clearly better if an anonymous coward told me so, any suggestions for lunch? i can't decide. you tell me what to do.

              Comments
              1. By Anonymous Coward () on

                Thai food. It's a good day to taunt death!

          2. By Anonymous Coward () on

            http://marc.theaimsgroup.com/?l=openbsd-misc&m=105059243101777&w=2

            There's your hero in action. You should feel proud.

            Comments
            1. By Dries Schellekens () on

              I'm sorry my hero didn't give you a polite answer.

              He's main problem seems to be that you add extra restrictions to mprotect. In this way you're required to hack the ELF header to run programs like XFree86, wine, ...

              Comments
              1. By PaX Team () on

                you're wrong. said applications are not managing writable/executable memory properly. in more detail:

                XFree86: they have their own ET_REL ELF file loading mechanism (it escapes me why they can't just use ET_DYN libraries which are actually meant to be loaded/executed) which assumes calloc() and the like gives them executable memory. i don't know about OpenBSD but on linux glibc doesn't even ask for PROT_EXEC in those functions. the solution is however more suprising than you'd think: link the X server statically and all problems are solved. as a bonus you'll have reduced the in-memory footprint of XFree86 as well.

                wine: it has its own file format loader for PE files (something like ELF in the UNIX world, but less developed IMHO) and uses mmap() and the like. it can be fixed by several ways, once the kernel work in PaX is finished, we'll get to it hopefully.

                real casualties are Java and others that really need to generate code at runtime, they can be fixed in the source code or just disable PaX on them and you don't need to hack the ELF header if you put the proper ACL on said files (in grsecurity).

                PS: after having read the thread, i hope you realize that Theo's point is wrong, PaX doesn't break POSIX.

    2. By tedu () on

      openbsd has had pax since release 2.0! man pax

      Comments
      1. By Dries Schellekens () on

        $ man PaX
        No manual entry for PaX

        I guess UNIX is case sensitive :-P

    3. By tedu () on

      openbsd has had pax since release 2.0! man pax

    4. By Jedi/Sector One () j@pureftpd.org on http://www.pureftpd.org/

      I really don't see the point in the flamed discussions between PaX author and Theo.

      Obviously, Linux had stack address randomization and non-executable pages before OpenBSD.

      So what? Did someone violate a patent?

      We are talking about free software here.

      Free software is not a war between projects, it's a collaborative work.

      If OpenBSD rips off nice ideas, and even code from other operating systems, this is great. I would love to see every feature of GrSecurity implemented in OpenBSD.

      If RedHat ships a Linux kernel with PF, this would be great also. Both for RedHat and for OpenBSD (more users, more bugs will be fixed, more ideas will be suggested, etc) .

      OpenBSD already closely watches every change in NetBSD and FreeBSD in order to port relevant ones. It's ok, it's great, it brings a lot of fixes and additions.

      As I see, all the flamed craptalk between PaX and Theo is about credits. Even if OpenBSD developpers never heard about PaX (and don't read deadly.org either, because we already talked about PaX before), how much does it cost to give a little credit? GrSecurity already gives credit to OpenBSD for the IPID generation.

      The OpenBSD developpers have some experiences that GrSecurity/PaX developpers don't have. But the reverse is also true. There are good ideas to take from both sides, code to share, and this is a good way to make both projects better. But after the discussions, it's clear that both projects will ignore each other, and possibly go in the same direction, implementing the same thing a bit differently. This happen all the time in a commercial world, but it's a bit pity for OSS projects.

      Comments
      1. By PaX Team () on

        You misunderstood the problem. As i already explained it to Dries in private, the credit issue has (at least) two sides. One is when you take other people's ideas/code/etc into your own work and another when you merely acknowledge their existence. I don't care that much about the former as i do about the latter. Trash talking or outright lies won't fly with me and i won't hesitate to make my point clear. Theo was wrong on both PaX violating POSIX/mprotect() semantics and OpenBSD not doing the same on mmap().

        > But after the discussions, it's clear that both
        > projects will ignore each other, and possibly
        > go in the same direction, implementing the same
        > thing a bit differently.

        What made you arrive at this conclusion? Theo != OpenBSD and from other private correspondance with other developers it's obviously that we're not ignored (and i don't know where you got that we ignored them, or plan to). Also you have to understand that kernel development is very specific to a given kernel, sharing/cooperation can rarely go beyond that of ideas/high-level concepts for technical reasons. What will happen to userland changes is an interesting question though, we'll see what can be shared/re-used (reminds me that propolice is already in Trusted Debian).

  3. By zil0g () on

    Ah say gatdang!

    =)

  4. By Anonymous Coward () on

    But is not going to for 3.3 right? :(

    Comments
    1. By Anonymous Coward () on

      No.

  5. By Anonymous Coward () on

    MicroBSD had ELF support years bef...

    Ah, foo. All thhe fun's been taken out of that one...

    Comments
    1. By mirabile () mirabile@bsdcow.net on http://MirBSD.BSDadvocacy.org/

      Well, in fact MirBSD (not MicroBSD tho) had ELF
      two hours before, or so...

      Okay, I started with the code from drahn@ and worked
      with him on resolving some issues, and I took other
      code from NetBSD...

      But damn, WineX still does not work. A port that
      builds is sent at ports@ some days ago.

      Comments
      1. By Anonymous Coward () on

        Don't get the point. MicroBSD had support for ELF way before OpenBSD ever did..

  6. By Anonymous Coward () on

    Which architectures are still non-ELF now?

    Comments
    1. By tedu () on

      mac68k and vax i think.

  7. By Anonymous Coward () on

    AWESOME!
    What's ELF? Why not Dark ELF?

    Comments
    1. By Anonymous Coward () on

      Buzzword is UPGRADE, not the fishy elf

  8. By Anonymous Coward () on

    so OpenBSD has ELF on i386? Welcome to the 20th century!

Credits

Copyright © - Daniel Hartmeier. All rights reserved. Articles and comments are copyright their respective authors, submission implies license to publish on this web site. Contents of the archive prior to as well as images and HTML templates were copied from the fabulous original deadly.org with Jose's and Jim's kind permission. This journal runs as CGI with httpd(8) on OpenBSD, the source code is BSD licensed. undeadly \Un*dead"ly\, a. Not subject to death; immortal. [Obs.]