Randomization within malloc()

I'm not sure I understand how this works. Does this randomize the virtual memory address that
the allocation gets mapped into?

It seems the code, instead of providing the "next" block of allocation, now gives any suitable block "near" the optimal allocation.

My guess is that the space that is skipped by the randomizer remains unmapped by the process.

Given that most programs don't come anywhere near using the entire address space, this seems like a super-keen thing to do, but I imagine it would reduce the useable address space for programs that might, like mmap'ing many large files.

Please enlighten me...

gone and done it again
if "it" means copying yet another feature of PaX.
way to go!

just mmap, not malloc. that part hasn't been committed yet. :)

What is the speed impact on this? If a program allocates large chunks of dynamic memory, wouldn't there be a significant speed impact when reading from memory? Just wondering :)

tedu claims in this thread:

"except pax doesn't do anonymous mmaps, which are kinda important."

Maybe you missed this part, when you were reading the PaX documentation to see what you could copy next and gain some fame for:
http://pageexec.virtualave.net/docs/randmmap.txt

The goal of RANDMMAP is to introduce randomness into memory regions handled
by the do_mmap() kernel interface. This includes all file and anonymous
mappings, such as the main executable, libraries, the brk() and mmap()
managed heaps.

please apologize for your malicious lie. thank you.

Here's a question for the openbsd developers who were involved in the randomization "development" on OpenBSD.

Do you know how infoleak bugs are exploited?

Do you know your between-mapping randomization is a waste of code?

Do you realize that by modifying the mmap base, all subsequent mappings are randomized (yes, including anonymous mappings tedu, you retard)?

Why do you make a big deal out of your between-mapping randomization when it serves no purpose? Is it simply to make yourself look like less of a rip-off from PaX by copying the features that actually do things (I noticed you used the same 16-bit mmap base randomization PaX uses) and then adding useless bloat?

Very interested in your response, geniuses!

hey, is there a simple mechanism in order to disable all and any performance-reducing security-increasing mechanisms for low-risk lower-cpu machines? i know there's likely to be some kind of sysctl things, but i was wondering if there's a guide on how to disable everything.

The only problems I've ever seen have been from pax trolls, so why not just disable comments on stories that have anything to do with anything pax does?

When I first began reading the comments to this thread I began thinking that one of my favorite web sites was turning into another "troll fest of useless comments" like slashdot. It was nice when near the end "Anonymous Coward" and "tedu" began to discuss things in a more civil manner. It's since peaked my interest as to what each package (PaX/W|X) offers.

Damn IF we steal code from PaX and It's derived from SysV work...
Besides I really dont care about PaX after all, probally when they get their crap in main linux kernel BRANCH we can talk again.
Btw please save the whales

i'll respond to a few points raised in one post lest it gets lost in the noise...

1. "OpenBSD developers don't give a fuck what features PaX provides! They just don't look at PaX (not at their source, nor at their documentation)! They come up with these features themselves. Why do you trolls keep having a hard time believing this?!"

let me begin with this one, as it probably highlights the core of the problem. contrary to what the claim would have you believe, said developers do care about PaX but not for the reason you think - i'll spell it out now because it seems to have been lurking in the background all this time but noone voiced it. consider why OpenBSD exists and why it is being used: mostly it's because of their various security related claims. i won't go into the validity of those claims, but i'll point out that apparently whatever they had achieved by mid-2002 (the time when certain 'high-profile' bugs surfaced) made them look for more security, stuff that's called intrusion prevention techniques in the commercial world (Entercept and others). whatever the origin of their techniques, they had clearly realized since that theirs was not the only one, and not even the best one for that matter - this means that their main claim to existence was in danger, OpenBSD is (and was) not the 'best' (security-wise, here it means 'resistance to exploits' which they care about the most, at least the infamous claim on their homepage makes one believe so). in my opinion it is for this reason that they chose to avoid any mention (let alone comparison) of PaX ever since, OpenBSD would simply not stand up, and coming from the developers' mouth would make them look bad (or so they seem to think).

you also make a claim about the origin of their features without showing any proof - the few links you provided are just the same tiring rehash of the question i had asked (and never got a response to) since the beginning: where's your development process documented? apparently you don't know any better, why don't you just quit this silly war of the words until you do?

2. "Please could you in favour for Random Memory Allocation Protection Method show some research papers on this sumbject for the community?"

http://seclab.cs.sunysb.edu/addr_obfs/docs/ao.pdf
http://www.crhc.uiuc.edu/~junxu/Papers/TechReport_TRR_UILU-ENG-03-2207.pdf
http://www.usenix.org/events/sec03/tech/cowan.html

not sure if you have access to USENIX, but you can try google/the authors' homepages.

3. "256MB is a reasonable amount of VM space to burn on fragmentation"

if your sshd maps 8 libraries, that's a 1GB (on average) of reduction in the largest mapping possible, if it's of any indication of a typical app, some of them might not be happy about it (the same apps you were referring to when you pointed out the halved address space under SEGMEXEC and Theo did the same a while ago in private mail - one wonders how/why he let this change fly now...). the very first version of ASLR in PaX did the same (per mapping randomization) by the way but it was found to be not worth it. if you think about it, there are two enemies of randomization: information leaking and attacks where absolute addresses don't have to be known. for the former, per library randomization is pointless because you never know (cannot guarantee) what parts of the address space can leak, so why bother? for the latter, it is pointless again for libraries because you cannot have overflows that cross ELF file mappings (there will be a non-mapped region or a r-x mapping between two rw-, unless you expect overflows from static data to .bss in a given library, but then you cannot randomize the gaps between such). it may matter for anonymous mappings provided they are used for all malloc() requests, something that's rarely the case (i don't know about OpenBSD, but on linux/glibc by default only requests above 128k are satisfied via mmap()). you said you were going to use mmap() for malloc() requests, i assume you meant those only that are above the page size, otherwise you'd be wasting lots of memory (something like ElectricFence does). in this case putting a random gap between them is pointless again, just have a non-mapped page and your overflows are stopped cold. in short, i see no reason for per mapping randomization.

4. "calling people liars when they haven't told a lie isn't very nice."

let's hope you keep reminding Theo about it as well.

5. "So of all these projects only OpenBSD blindly copied PaX and the others didn't? These Windows implementations of the PaX ideas are independent and when OpenBSD claims they developed this independantly, you keep bitching on mailing lists and websites."

a few words on the projects linked to on the PaX site. the windows versions have been developed based on the PaX documentation (note that contrary to a claim made here, i didn't develop either of them myself, although i did talk to the guys about various problems that popped up), they both acknowledge it on their respective sites - something you cannot really say about OpenBSD (i'd like to note that when Theo said on bugtraq the other day that he hadn't read anything where PaX was announced/talked about in the past... well, he was not saying the truth, namely he had been reading bugtraq itself where PaX showed up a few times... go figure). you guys keep saying that you had developed all this independently, yet when asked for evidence, silence is your response. will you too go silent now?

by the way, the description of Exec-Shield is there to point out its non-executable page features, not the randomization ones (otherwise i should have put up many more links to other randomization projects that popped up over time, they'll be mentioned/analyzed in a future document though).

6. "what happens when an attacker overflows A? he hits B. so if you know that writing 172 bytes past A will corrupt B in such a way as to lead to an exploit, no amount of adjusting the base helps."

i think what you're getting at is a dead-end, it has been done in the past and is called ElectricFence and it's unusable in production (although really great for debugging).

7. "how does PaX prevent overflowing A from overwriting data in mapping B?"

PaX doesn't prevent any kind of overflow, it is simply not the goal (or hasn't been so far, this may very well change in the future). the goal is to prevent exploit techniques from working, read the documentation for details. your particular attack scenario allows for what i called the 3rd type of exploit technique (data modification) for which PaX explicitly provides no protection yet (again, it's clearly spelled out in the docs).

8. "you can only do this if you disable mprotect. that's not an option."

not sure where you got this from, but PaX does not disable mprotect(), what it does is that it prevents certain page protection combinations which is perfectly valid according to POSIX.

9. "the per-page ness was a lot more complicated than W^X. you can get pretty close using just the hardware without having to reimplement stuff in software."

i assume here you're talking about i386 and vma mirroring, as on the other archs it takes next to no code to get per page non-exec protection. have you got some comparison as to how much effort/code it took in OpenBSD to accomodate userland to your solution (ld.so i think)? also, i don't understand what you were referring to by 'reimplement stuff in software'.

10. "Sorry, OpenBSD doesn't provide proc/pid/maps and /proc/pid/stat like PaX."

actually, PaX doesn't provide them either as it is the /proc filesystem in linux itself that does and if you care to read the docs, you will find out why i didn't touch this code (hint: PaX is part of bigger systems that may and do want to do it their way, so why interfere).

11. "have you considered that we might be trying to switch to mmap-based malloc?"

this is an interesting statement, earlier this year Theo told me the following in private mail:

---------------------------------------------
> how is the brk() managed heap mapped/handled/used in OpenBSD? would
> malloc() not use it for 'small' allocations (like it does in linux)?

brk is not used.
---------------------------------------------

so what gives?

12. "maybe the application your attacking doesn't have any bad format strings?"

this maybe of some interest: http://marc.theaimsgroup.com/?l=bugtraq&m=105941103709264&w=2

Dear tedu,

Thanks for your contribution. Sorry that a flamefest ensued.

--Michael

I use openbsd because it has been, over the long haul, the most secure OS out there that runs on my hardware and gives me the features I want. I'm just a user--my view is worthless beyond that.

I don't care if PaX had idea(x) 3.74 seconds before OpenBSD did. The fact remains that OpenBSD did independant development, unless PaX has a patent on the process or there is copyright violation going on, I don't care who thought the thought first. As pretty much everyone agrees (except maybe SCO's lawyers) the developement was seperate and there isn't a copyright violation.

Neither the GNU or BSD license requires you to give credit for the IDEA, only the implementation. If the implementation was done seperately there is no legal basis to demand anyone give anyone else credit.

Personally, I tend to buy the disperate-origions/ same-conclusion argument. Smart people (which most of us here are, save a few of the obvious trolls) tend to have good ideas. Good ideas all tend to be in the same direction. It's quite possible that all these good ideas happened around the same time. Maybe it's something in the water.

But, in the end, it's all about me. I decide what I want to use and why. OpenBSD or PaX can both have great ideas and great implementations but if neither meet my needs I won't use it. If I --NEED-- a feature in PaX I'll go build myself a Linux system and load PaX up. If OpenBSD continues to provide a consistantly good security record (a few misses here and there is still far better than anything else out there) then I'm going to continue using it and GNU/Linux/PaX won't get my $$$.

The unstated goal of the BSD camp is to make all software BETTER. The clearly stated goal of the GNU camp is to make all software FREE. I for one see these two goals pretty much in line with each other. Asking for credit for an IDEA is pretty anti-GNU from my vantage, it smells an awfully lot like a patent. If OpenBSD wants to take an IDEA like DNS or SSH or W^X and write a BETTER implementation of it (without violating copyright law/licenses) then why are you complaining? If PaX likes an IDEA in OpenBSD's setup they can take it too.

It's purely academic if PaX or OpenBSD's approaches are technically better. I'd love to read a full breakdown of the two. I'd like it even more, for my own selfish reasons as a user, if the best bits of both were integrated together--which is the general direction of OpenBSD anyways.

http://www.netsys.com/openbsd-misc/1999/12/msg00873.html

Read that hilarious thread and then think that Theo claims to have developed the entire idea of W^X shortly after. One day it's really really hard, and the next he's magically solved every problem by himself? That's pretty incredible stuff, folks!