n2k15: bluhm@ on MP networking (out from under biglock)

Next up in the ongoing series of n2k15 reports is Alexander Bluhm (bluhm@), who writes:

This hackathon I was sitting between mpi@ and sashan@ to work on multiprocessor networking.

The next step, we are currently working on, is to remove the big kernel lock from forwarding and routing. mpi@ has been doing this for a long time, but some corner cases were still left. I have written a regression test for handling ARP packets to show that all cases including proxy ARP are still working.

Another thing that may happen with lock-free routing is that the interface is destroyed on one CPU while another CPU is working with a route to that interface. We finally got this resolved. The code that destroys the interface has to wait until all routes don't use this interface anymore. I moved the sleep before the destruction of the interface is started, so that the routes can always operate on a completely valid interface structure.

Currently pf is not MP safe on OpenBSD, but we have such an implementation on Solaris. To get this imported, we want to start with locks around the entry points to pf. Apart from the obvious ones in ip_input() and ip_output(), the TCP and UDP stack is holding pointers to pf states to speed up socket lookups and to make divert-to and divert-reply reliable. I have moved the pointer manipulation into functions within pf. This is a more obvious entry point and can be protected by locks and reference counters.

Thanks for the report and this work, Alexander! We're really raring to go with this in the tree.