l2k15: guenther@ on hacking libc symbols

Mmmm, fresh burek and other pastries for breakfast, with plums from the farmer's market to slurp down while hacking. It was a great trip...

I spent most of l2k15 banging on libc with Theo. We finally started explicitly specifying the exported symbols late last month and managed to chop about 200 symbols from the export list before the hackathon. During the hackathon we made further progress on two fronts:

• we stopped exporting another 100 symbols or so by hiding internal APIs and, in some cases, deleting routines or moving them into particular applications. Theo did most of this.
• we stopped permitting overriding of many internal calls in libc. I did most of this.

To explain that latter a bit more: ELF symbols resolution rules state that symbols defined in an application override those in a shared library. For libc, that's almost never what you want. For example, in ISO C the name 'read' is not reserved, so an application should be able to define its own read() function. If it then calls stdio to read data, the stdio library should *not* use the application's read(), but rather should continue to use the real read syscall.

To do that, we now define internal aliases for many of the functions in libc and redirect internal calls to use those. This solves the overriding problem for those functions and also reduces overhead: to make the overriding work, the compiler and linker had those calls going indirectly through the PLT (Procedure Linkage Table) and/or the GOT (Global Offset Table). Doing the lazy symbol resolution process for those meant that ld.so was involved on the first call and some page of memory had to be updated. By skipping that, the internal calls can go direct to the real function and no extra memory changes are necessary, which improve performance and security. Win win! This has been the standard practice on many other OSes for a while, so it's been good that we're finally catching up on this.

There's still a ways to go on this process. Some symbols *do* need to be overriden still: libpthread needs to override errno and many syscalls, while some applications (*cough* emacs *cough*) expect to override the malloc family of functions. I'm still deciding how we want to handle the former: merging the libpthread wrappers into libc would be cleanest, but that requires setting up and using the TCB (thread control block) and we don't have fast TCB handling on mips64, arm, alpha, or vax. alpha just needs a couple kernel hooks to be fast for this, arm has a coprocess register defined for this that may require emulation (slow?), while mips64 and vax will require an *ABI change*. Blech. Maybe we'll go with the callback table approach until those can be worked out...

Other than that I served as a sounding board for some of beck@ and jsing@'s libtls cert checking API discussions, and reviewed various diffs by others, including some hairy kernel sleep API usage by dlg@, and got some other random diffs in including the last of the kbind changes: all dynamic archs use kbind for updating lazy resolutions, yay!

All in all, had a great time, enjoying the location, the weather, the hacking, and some interesting presentations at the security conference that our host (Tonimir Kisasondi, many thanks!) was running locally immediately after the hackathon.

Thanks for a lot of awesome work, Philip (and Theo)! And thanks for the report!

We're hoping to publish more reports from this hackathon, where 17 developers did their thing to various parts of our favorite operating system.