ldapd entering the tree

Back in November, Martin Hedenfalk posted to the tech@ mailinglist with an announcement for his ldap daemon. A few months have passed in which Martin has improved his code and then imported it into base, connecting it to the build only a few days later.

Undeadly spoke with Martin (martinh@ now) about ldapd(8) and plans for the future. Please read on for the interview:

OpenBSD Journal: What can you tell us about yourself ?

martinh@:I'm Martin Hedenfalk, I'm 33 years old and live in Stockholm, Sweden. I began using OpenBSD around 3.7 when I did a VPN appliance based on IPsec on OpenBSD. I was impressed by the consistency of the system; the manual pages actually represented what was running in the system, and things Just Worked.

OpenBSD Journal: Why did you start working on your ldap daemon ?

martinh@:I started working on an ldap daemon because I wanted a simple address book. I wasn't too impressed by OpenLDAP, so I decided to try to write my own. It was also an ideal project to try to implement an append-only btree database that I had been thinking about for some time.

In essence it seemed like an interesting problem. I had also used OpenLDAP at work, with quite a large directory (>2 million entries), and wanted to see if I could improve on the LDIF import time.

OpenBSD Journal: You recently got an account and committed your work to the OpenBSD repository. It's linked to the builds now too. What can you tell us about this process - how did you get your account etc. ? How was your experience working with some of the other OpenBSD developers ?

martinh@:I initially posted about my work on the tech@ mailing list in november last year. I got some positive feedback from developers and continued working on it. Recently I was talking to marco@ about my btree implementation, and was asked if I wanted to join. And gilles@ offered to be my mentor.

But I haven't met anyone in person yet. I really look forward to seeing many of them at c2k10.

OpenBSD Journal: Any relation to ypldap(8) ?

martinh@:ldapd can of course be used as the server in a ypldap setup, but other than that there is no additional integration.

OpenBSD Journal: What is the current state of ldapd ? What works, what doesn't work ?

martinh@:It's still in an early stage, and it's not recommended for production use yet.

The basic stuff works, adding and modifiying entries, searching, as well as binding using both simple authentication and SASL PLAIN with your regular BSD credentials. Connections can be secured with TLS or SSL, and there is a simple access control scheme, although that needs more work to be really useful.

Support for schema definitions is lacking. Currently ldapd doesn't know about different syntaxes and matching rules. All attributes are treated as case-insensitive strings, which works quite well in most situations, but isn't conforming to standards and is, well, wrong. Expect this to change soon.

OpenBSD Journal: How does your work compare to the venerable OpenLDAP ? What about compatibility ?

martinh@:ldapd takes a more minimalist approach than OpenLDAP. There will not be support for multiple backends, for example. Another goal is to provide reasonable and working defaults as far as possible.

The LDAP RFCs (and there are _lots_ of them), are pretty serious about establishing standardized behaviour, so compatibility with OpenLDAP and other LDAP implementations is good, at least protocol-wise. You should be able to use any LDAP directory browser or client utility to talk to ldapd.

The differences are in non-standardized areas such as access control and replication, where each implementation does its own thing. Compatibility here is not a goal, but I've looked at using RFC 4533 (the OpenLDAP syncrepl protocol) as replication method. But nothing is decided yet.

OpenBSD Journal: How does your on-disk database work ?

martinh@:The btree database is a modified prefix B+tree using append-only writing. It's basically a regular B+tree with prefix key compression, but without the leaf chaining. The difference is that modified pages are appended to the file rather than being updated in-place.

This means that each change creates a new tree on the disk, so any existing cursors traversing the database will always see a consistent snapshot. So it's a lock-free database; readers will never be blocked, but writes must be serialized.

Appending takes advantage of efficient sequential write speed in hard drives. This comes at a cost though: the database files grow quickly. To reduce the file size it must regularly be rewritten, or compacted.

This was inspired by the Apache CouchDB project, but is by no means a new idea. Log-structured filesystems have been doing this for a long time.

OpenBSD Journal: What about redundancy / failover situations ?

martinh@:Replication will be implemented, but that has to wait until the basic functionality is done.

OpenBSD Journal: Do you have any other future plans for ldapd ?

martinh@:Most probably, but I don't know what they are yet :-)

OpenBSD Journal: Any final remarks ?

martinh@:LDAP isn't only useful for authentication and address books. I'm using ldapd as backend for a couple of web applications, using perl and Net::LDAP, and it works really well.

There is this hype about NoSQL databases, but LDAP has been here all the time. The one thing that makes LDAP difficult is it's love for meta data, in the form of schemas. If all you want to do is store data as key-value pairs, you can configure ldapd to ignore schema checking and treat all keys as case-insensitive strings. This is done with the "relax schema" keyword in the config file.

Finally I'd like to thank everyone involved in OpenBSD for a great OS. It certainly makes hacking fun.

OpenBSD Journal: Thank you, Martin, for your time in answering these questions and thanks for working on ldapd.