OpenBSD Journal
Contributed by jj on from the dawn of the living tedu dept.
Log message:Nginx fans will of course find it in ports, and people that just want to run a simple web server should read up on the new httpd(8).
remove nginx from the base system in favor of OpenBSD's own httpd(8)
(Comments are closed)
Copyright © - Daniel Hartmeier. All rights reserved. Articles and comments are copyright their respective authors, submission implies license to publish on this web site. Contents of the archive prior to as well as images and HTML templates were copied from the fabulous original deadly.org with Jose's and Jim's kind permission. This journal runs as CGI with httpd(8) on OpenBSD, the source code is BSD licensed. undeadly \Un*dead"ly\, a. Not subject to death; immortal. [Obs.]
By jeanot (10.241.102.218) on
Comments
By Elvis (203.206.211.188) on
From http://www.openbsd.org/faq/upgrade55.html:
"OpenBSD's version of Apache 1.3 will be removed from base in favor of nginx(8)."
The plot thickens...
By He-Man (138.190.32.7) on
By Ska (182.250.240.85) on
It supports fastcgi so it isn't barebones but can it stream html5 video?
Comments
By Ska (182.250.240.85) on
> It supports fastcgi so it isn't barebones but can it stream html5 video?
Before I get inevitably downvoted, I'll let you know that I realize that I messed the section number up. Also what the hell are tech@ and misc@ useful for? I got to watch all sorts of flame wars and clueless threads but even now I can search for httpd and only get veiled references(hur hur your still on nginx)
By Anonymous Coward (93.38.92.79) on
Comments
By a|r (63.153.244.16) on
> Well, it sounds reasonable to me. There is no reason to ship the base system with a full web server when it's so easy to get the one you prefer directly from ports/packages. One more step towards having a cleaner base!
Comments
By Anonymous Coward (217.140.96.21) on
Comments
By Karol (89.65.133.49) on
By Nick Holland (NickHolland) on http://www.openbsd.org/faq/
A reasonable question.
An answer: Think of the base OpenBSD system as a "fork kit" -- if you want to build a new OS based on OpenBSD, everything you need is in the base system and source code: compilers, editors, change management system, diagnostics, fetching and distribution tools. A web server is a distribution tool.
Given the base system and source code tar balls (or a CD set), you could be locked on a desert island with no Internet connectivity and continue work on the project.
As someone who runs a number of servers, it's nice to have basic services "just there", available at the flip of a software switch -- mail, dns, web, ftp, ssh, etc. Need to document a service on a system? Drop a file in /var/www/htdocs, activate the web server, and ta-da. Next update, the updates to that web server are done automatically.
Another answer: because the developers want it there. :-)
Comments
By Anonymous Coward (91.154.65.231) on
As a user, my perception is that this all leads to more secure and robust software. Something I feel I can rely on. I'm thankful for the developers who do this.
Of course, I'm not just blindly trusting the software. Having something simple, small, and clean in base means happy hours reading code under /usr/src. I don't enjoy reading the (usually large) sources of third party packages nearly as much.
By Anonymous Coward (62.159.86.18) on
I think this decision is bad, but that's just imho.
Comments
By Amit Kulkarni (170.49.177.108) on
>
> I think this decision is bad, but that's just imho.
If you carefully read the commit messages, it will tell you the story. Nginx is getting too complicated to audit and understand. This is a direct fallout of Heartbleed. First OpenSSL was forked, then Kerberos was removed from base, then some more audits happened, and now the new httpd was written based on the privilege separated relayd code. Features will come slowly over time. I value security over bloat.
Comments
By Simon (130.185.136.244) on
> If you carefully read the commit messages, it will tell you the story. Nginx is getting too complicated to audit and understand. This is a direct fallout of Heartbleed. First OpenSSL was forked, then Kerberos was removed from base, then some more audits happened, and now the new httpd was written based on the privilege separated relayd code. Features will come slowly over time. I value security over bloat.
I really loved having Nginx in base, but I can't say I don't understand the decision. If httpd had a feature similar to Nginx proxy_pass I would be pretty much set.
Comments
By Anonymous Coward (204.80.187.9) on
> > If you carefully read the commit messages, it will tell you the story. Nginx is getting too complicated to audit and understand. This is a direct fallout of Heartbleed. First OpenSSL was forked, then Kerberos was removed from base, then some more audits happened, and now the new httpd was written based on the privilege separated relayd code. Features will come slowly over time. I value security over bloat.
>
> I really loved having Nginx in base, but I can't say I don't understand the decision. If httpd had a feature similar to Nginx proxy_pass I would be pretty much set.
relayd might be there.
By Denis (2001:7a8:b5ad::10:10) on
Couldn't relayd(8) act as a substitute ?
Comments
By Anonymous Coward (85.81.131.17) on
I didn't actually think of that, I just assumed that relayd where not able to look at the URL path.
Mostly I would want domain.com/webapp1 go to localhost:8081 and domain.com/webapp2 go to localhost:8082. It's not really clear from the documentation if that's possible.
Also it's a little confusing that relayd is on port 80, directing http traffic, if you just run a small server with a single web application or two and a few static files.
By Renaud Allard (renaud) on
> > If you carefully read the commit messages, it will tell you the story. Nginx is getting too complicated to audit and understand. This is a direct fallout of Heartbleed. First OpenSSL was forked, then Kerberos was removed from base, then some more audits happened, and now the new httpd was written based on the privilege separated relayd code. Features will come slowly over time. I value security over bloat.
>
> I really loved having Nginx in base, but I can't say I don't understand the decision. If httpd had a feature similar to Nginx proxy_pass I would be pretty much set.
What I think is missing to httpd to fill 99% of peoples needs are:
- proxy_pass equivalent (should be easy as relayd can do it)
- basic auth (to protect some pages that are not intended for everyone)
- basic ACLs (like deny access to /config)
- some rewrite rules (like redirecting http://mail.domain.tld to httpS://mail.domain.tld or run things like wordpress,...)
Comments
By Anonymous Coward (79.238.55.158) on
> > > If you carefully read the commit messages, it will tell you the story. Nginx is getting too complicated to audit and understand. This is a direct fallout of Heartbleed. First OpenSSL was forked, then Kerberos was removed from base, then some more audits happened, and now the new httpd was written based on the privilege separated relayd code. Features will come slowly over time. I value security over bloat.
> >
> > I really loved having Nginx in base, but I can't say I don't understand the decision. If httpd had a feature similar to Nginx proxy_pass I would be pretty much set.
>
> What I think is missing to httpd to fill 99% of peoples needs are:
> - proxy_pass equivalent (should be easy as relayd can do it)
> - basic auth (to protect some pages that are not intended for everyone)
> - basic ACLs (like deny access to /config)
> - some rewrite rules (like redirecting http://mail.domain.tld to httpS://mail.domain.tld or run things like wordpress,...)
You basically just described relayed + httpd. Not everything needs to implement every function you need; think in terms of systems, not single daemons :)
By reyk (79.194.193.186) on
> - proxy_pass equivalent (should be easy as relayd can do it)
> - basic auth (to protect some pages that are not intended for everyone)
> - basic ACLs (like deny access to /config)
> - some rewrite rules (like redirecting http://mail.domain.tld to httpS://mail.domain.tld or run things like wordpress,...)
I do not think that 99% of people need proxy_pass, especially not in a base httpd. But httpd is a companion of relayd which is a wonderful alternative. Support for basic auth and ACLs will be added later as they're expected from a "basic web server".
There is some whining about rewrite rules and people are demanding support for regular expressions. But this is not going to happen. There are other ways to do this already but if you really need super advanced regular expressions in your web server config you can a) either fix it in the fastcgi backend (which should be totally possible in 2014) or b) use another webserver from ports - nginx is still there.
Reyk
Comments
By Anonymous Coward (162.229.162.103) on
Just to clarify, when you say "this is not going to happen", are you referring to any rewrite rules, of just rewrite rules with super advanced regular expressions?
Thanks.
By Renaud Allard (renaud) on
> > - proxy_pass equivalent (should be easy as relayd can do it)
> > - basic auth (to protect some pages that are not intended for everyone)
> > - basic ACLs (like deny access to /config)
> > - some rewrite rules (like redirecting http://mail.domain.tld to httpS://mail.domain.tld or run things like wordpress,...)
>
> I do not think that 99% of people need proxy_pass, especially not in a base httpd. But httpd is a companion of relayd which is a wonderful alternative. Support for basic auth and ACLs will be added later as they're expected from a "basic web server".
>
I agree, many people won't probably need a proxy_pass like feature. However, if you run a java/python/perl server on 127.0.0.1 with a dedicated user, proxy_pass can be used to make it ssl compliant or use the standard port 80 with a virtual name. Given that relayd has this kind of feature and httpd is based on it, it might be a good idea to provide at least a simple proxy_pass feature for simple cases.
By Oko (24.112.236.113) on
> > If you carefully read the commit messages, it will tell you the story. Nginx is getting too complicated to audit and understand. This is a direct fallout of Heartbleed. First OpenSSL was forked, then Kerberos was removed from base, then some more audits happened, and now the new httpd was written based on the privilege separated relayd code. Features will come slowly over time. I value security over bloat.
>
> I really loved having Nginx in base, but I can't say I don't understand the decision. If httpd had a feature similar to Nginx proxy_pass I would be pretty much set.
proxy_pass like feature was added to relayd circa two months ago. Check out misc for Reyk post.
Comments
By Anonymous Coward (79.238.55.158) on
> > > If you carefully read the commit messages, it will tell you the story. Nginx is getting too complicated to audit and understand. This is a direct fallout of Heartbleed. First OpenSSL was forked, then Kerberos was removed from base, then some more audits happened, and now the new httpd was written based on the privilege separated relayd code. Features will come slowly over time. I value security over bloat.
> >
> > I really loved having Nginx in base, but I can't say I don't understand the decision. If httpd had a feature similar to Nginx proxy_pass I would be pretty much set.
>
> proxy_pass like feature was added to relayd circa two months ago. Check out misc for Reyk post.
I believe this is the email to which you're referring[1]:
"I removed all the "protocol nodes" code from relayd and started to
reimplement it as a new filtering subsystem. The resulting
configuration language uses last-matching pf-like rules starting with
the "pass", "block" or "match" keywords. If you know how to use
OpenBSD's pf, you will quickly know how to use the filter rules;
otherwise it is a bit of a learning curve."
[1]http://marc.info/?l=openbsd-misc&m=140492841707084&w=2
By Craig Skinner (78.33.153.148) on
Superb:
the goal is to provide an HTTP server that:
a) provides minimal features
b) serves static files
c) provides FastCGI support
d) follows common coding practices of OpenBSD
I'm thoroughly looking forward to deploying ANOTHER excellent OpenBSD developed daemon towards the end of the year!
Static sites only - of course.
Comments
By Anonymous Coward (63.153.244.16) on
> > people that just want to run a simple web server should read up on the new httpd(8).
>
> Superb:
>
> the goal is to provide an HTTP server that:
> a) provides minimal features
> b) serves static files
> c) provides FastCGI support
> d) follows common coding practices of OpenBSD
>
> I'm thoroughly looking forward to deploying ANOTHER excellent OpenBSD developed daemon towards the end of the year!
>
> Static sites only - of course.
Comments
By Anonymous Coward (2601:6:51c0:e1:9597:caaf:1335:fa85) on
I suspect the developers want to have the ability to provide the www.openbsd.org website using the base distribution and not requiring the use of ports/packages.
By Anonymous Coward (81.200.189.1) on
Excuse me? Could you please support this statement with facts?
"We keep hearing"... That does not sound very serious.
Plus, as an aside, this is precisely why nginx has been removed from base: OpenBSD's httpd is probably a much smaller and cleaner code base.
By reyk (79.194.193.186) on
You don't seem to get the point of it. nginx was external software that happened to sit in our source tree for a while. It got patched to have a rudimentary privilege dropping and chroot, and to fit a bit better into OpenBSD. But most of the code base was "read-only" for us and even reading and understanding the code was hard. The amount of code constantly grew. That doesn't mean that nginx is bad, it just means that some code is so über-optimized and written in a different dialect of C that we had a hard time reviewing and understanding it with the same standards that we have elsewhere in base. We did communicate with the maintainers - but sometimes they simply didn't understand us demanding better privsep/privdrop or disabling SPDY. And we don't really like some things in nginx: it uses custom memory allocators (for performance reasons) and it is wrapping or replacing standard C library functions all over the place. This could eliminate some of our built-in security mechanisms. All in all, nginx is fine in ports.
But httpd was written in and by OpenBSD and it shares a lot of code and efforts with other daemons, including relayd, snmpd, iked and even the "cousins" bgpd, ospfd, smtpd and others. The FastCGI implementation was contributed by florian@ based on his slowcgi code. We have an amazing lineup of tools in base, but they are not individual software projects -they share code, principles, and concepts. They are not "alien" to us and we have multiple people who "know" or understand the code base. I get useful contributions, tests, bug reports and direct feedback from many OpenBSD developers and our user community. It is a big difference. Moving httpd to ports would eliminate most of these benefits.
>The developers have enough code to deal with in the base system as it is, as we keep hearing about security bugs in the system years old.
You make it sound like a conspiracy. Software has bugs, bugs have to be fixed. We have a big and very sane code base but nobody in OpenBSD ever claimed that we have a bug-free operating system. But we're constantly and proactively working on improving the quality, fixing the bugs, and advancing the exploit mitigation techniques.
Reyk
Comments
By Sebastian (91.65.157.11) on
>
> You don't seem to get the point of it. nginx was external software that happened to sit in our source tree for a while. It got patched to have a rudimentary privilege dropping and chroot, and to fit a bit better into OpenBSD. But most of the code base was "read-only" for us and even reading and understanding the code was hard. The amount of code constantly grew. That doesn't mean that nginx is bad, it just means that some code is so über-optimized and written in a different dialect of C that we had a hard time reviewing and understanding it with the same standards that we have elsewhere in base. We did communicate with the maintainers - but sometimes they simply didn't understand us demanding better privsep/privdrop or disabling SPDY. And we don't really like some things in nginx: it uses custom memory allocators (for performance reasons) and it is wrapping or replacing standard C library functions all over the place. This could eliminate some of our built-in security mechanisms. All in all, nginx is fine in ports.
>
> But httpd was written in and by OpenBSD and it shares a lot of code and efforts with other daemons, including relayd, snmpd, iked and even the "cousins" bgpd, ospfd, smtpd and others. The FastCGI implementation was contributed by florian@ based on his slowcgi code. We have an amazing lineup of tools in base, but they are not individual software projects -they share code, principles, and concepts. They are not "alien" to us and we have multiple people who "know" or understand the code base. I get useful contributions, tests, bug reports and direct feedback from many OpenBSD developers and our user community. It is a big difference. Moving httpd to ports would eliminate most of these benefits.
>
> >The developers have enough code to deal with in the base system as it is, as we keep hearing about security bugs in the system years old.
>
> You make it sound like a conspiracy. Software has bugs, bugs have to be fixed. We have a big and very sane code base but nobody in OpenBSD ever claimed that we have a bug-free operating system. But we're constantly and proactively working on improving the quality, fixing the bugs, and advancing the exploit mitigation techniques.
>
> Reyk
>
Reyk don't get me wrong but will httpd support basic technologies like ServerSideIncludes? Also I have trouble to understand how to cleanly convert some stuff I do with (currently) nginx to httpd.
Example 1:
How to prevent to display the default "vHost" if no Hostname was specified in the request. Nginx does:
server {
server_name _;
return 444;
}
Do I need to use relayd combined with httpd?!
How to get rid of some nerving scanners or specify an error-page like with:
rewrite ^/(.*php)$ http://www.exampledomain.de/$1 permanent;
error_page 404 /index.html;
This redirects all PHP-Requests for a specific domain (scanners..) to a different domain and marks the requested files as permantly moved.
With all due respect but I assume that some basic RegEx support or the ability to simply send nothing (code 444 from nginx) is something many people do need (ok Code 444 is maybe not true , it depends if people do know about it..). Nobody exspects a full featured nginx-replacement wich allows us to rebuild YouTube from scratch. But what are the technical goals for the next release related to httpd (in 5.7)?
Is there a roadmap or is the httpd considered finished?
Kind regards,
Sebastian
Comments
By reyk (91.9.220.173) on
>
No ServerSideIncludes.
But you can do legacy SSI in the FastCGI backend, for example using the Perl SSI modules.
> How to prevent to display the default "vHost" if no Hostname was specified in the request. Nginx does:
>
> server {
> server_name _;
> return 444;
> }
>
Something like "return" makes sense. But I think the magic 444 code is a nginx-ism and sounds like a bad conceptual workaround.
> With all due respect but I assume that some basic RegEx support or the ability to simply send nothing (code 444 from nginx) is something many people do need (ok Code 444 is maybe not true , it depends if people do know about it..). Nobody exspects a full featured nginx-replacement wich allows us to rebuild YouTube from scratch. But what are the technical goals for the next release related to httpd (in 5.7)?
>
"I won't buy this car if it doesn't have at least two cup holders". OK, no problem, you could just continue using nginx.
httpd has its use cases that will make many people happy. But I never claimed to replace nginx in general, httpd just replaced nginx in OpenBSD base.
> Is there a roadmap or is the httpd considered finished?
>
httpd is not finished.
The roadmap is to improve it, implement features that make sense for us and to keep httpd simple. I get good feedback, contributions and requests from other developers and the user community, but I also have to reject features and diffs regularly.
Reyk
Comments
By Anonymous Coward (91.65.157.11) on
> >
>
> No ServerSideIncludes.
>
> But you can do legacy SSI in the FastCGI backend, for example using the Perl SSI modules.
>
> > How to prevent to display the default "vHost" if no Hostname was specified in the request. Nginx does:
> >
> > server {
> > server_name _;
> > return 444;
> > }
> >
>
> Something like "return" makes sense. But I think the magic 444 code is a nginx-ism and sounds like a bad conceptual workaround.
>
> > With all due respect but I assume that some basic RegEx support or the ability to simply send nothing (code 444 from nginx) is something many people do need (ok Code 444 is maybe not true , it depends if people do know about it..). Nobody exspects a full featured nginx-replacement wich allows us to rebuild YouTube from scratch. But what are the technical goals for the next release related to httpd (in 5.7)?
> >
>
> "I won't buy this car if it doesn't have at least two cup holders". OK, no problem, you could just continue using nginx.
>
> httpd has its use cases that will make many people happy. But I never claimed to replace nginx in general, httpd just replaced nginx in OpenBSD base.
>
> > Is there a roadmap or is the httpd considered finished?
> >
>
> httpd is not finished.
>
> The roadmap is to improve it, implement features that make sense for us and to keep httpd simple. I get good feedback, contributions and requests from other developers and the user community, but I also have to reject features and diffs regularly.
>
> Reyk
>
Thank you for the inview!
But do you consider to implement rewrite rules or would you recomment to use relayd for this (see my example to redirect visitors who want a *.php to a different server)?
Also about the SSI situation: Would it be possible to include an SSI-CGI from thttpd into the base-system (to keep it maintained)?
http://www.acme.com/software/thttpd/ssi_man.html
The License is descriped as BSD.
Kind regards,
Sebastian
Comments
By Anonymous Coward (91.65.157.11) on
The Code 444 just tells nginx to send nothing back. No error or other data.
Kind regards,
Sebastian
By Jorden Verwer (37.0.95.173) on
1. Stick with Apache 1.3, installing it from ports
2. Switch to Apache 2, also from ports
3. Switch to a different HTTP daemon and use slowcgi
4. Switch to a different HTTP daemon and do a partial rewrite away from CGI
Up until now I was planning to choose option 3 and go with SCGI on nginx, but now I'm not so sure anymore. Now that nginx isn't part of base anymore (well, it is for 5.6, but that'll only buy me six months, which is totally not worth the effort) it is essentially back on par with Apache in terms of out-of-the-boxness. I could opt for httpd instead and go with FastCGI, but that's a far more complicated protocol. I really don't know what to do, especially because I'm not sure about httpd's goals. Using slowcgi is a possibility, but if at all possible I'd like to avoid having another layer in between.
Therefore, I'd like to know:
1. If CGI support is planned for httpd
2. If SCGI support is planned for httpd
3. If other alternatives to FastCGI are planned for httpd
4. If I can rely on httpd by the time of, say, OpenBSD 5.8
Although it shouldn't matter much, the application is written in C. There are actually some advantages to rewriting it to use SCGI or FastCGI, because it means I can improve the application's performance by caching slow-changing data. However, a recent hardware upgrade has shown that while the current implementation may be inefficient, its performance is acceptable to users. So avoiding the need for a rewrite would be moderately good news. On the other hand, having to implement FastCGI instead of SCGI would be bad news. I really don't like pulling in another external library, but the prospect of having to implement FastCGI myself does not sound very appealing to me.
This project is basically something between a hobby and volunteer work, so I can't spend infinite amounts of time on it, but if the work is enjoyable I'm okay with performing a serious partial rewrite.
What would be the best option for me: Apache, nginx or httpd?
Comments
By Anonymous Coward (91.9.220.173) on
>
> 1. Stick with Apache 1.3, installing it from ports
> 2. Switch to Apache 2, also from ports
> 3. Switch to a different HTTP daemon and use slowcgi
> 4. Switch to a different HTTP daemon and do a partial rewrite away from CGI
>
> Up until now I was planning to choose option 3 and go with SCGI on nginx, but now I'm not so sure anymore. Now that nginx isn't part of base anymore (well, it is for 5.6, but that'll only buy me six months, which is totally not worth the effort) it is essentially back on par with Apache in terms of out-of-the-boxness. I could opt for httpd instead and go with FastCGI, but that's a far more complicated protocol. I really don't know what to do, especially because I'm not sure about httpd's goals. Using slowcgi is a possibility, but if at all possible I'd like to avoid having another layer in between.
>
> Therefore, I'd like to know:
> 1. If CGI support is planned for httpd
> 2. If SCGI support is planned for httpd
> 3. If other alternatives to FastCGI are planned for httpd
> 4. If I can rely on httpd by the time of, say, OpenBSD 5.8
>
> Although it shouldn't matter much, the application is written in C. There are actually some advantages to rewriting it to use SCGI or FastCGI, because it means I can improve the application's performance by caching slow-changing data. However, a recent hardware upgrade has shown that while the current implementation may be inefficient, its performance is acceptable to users. So avoiding the need for a rewrite would be moderately good news. On the other hand, having to implement FastCGI instead of SCGI would be bad news. I really don't like pulling in another external library, but the prospect of having to implement FastCGI myself does not sound very appealing to me.
>
> This project is basically something between a hobby and volunteer work, so I can't spend infinite amounts of time on it, but if the work is enjoyable I'm okay with performing a serious partial rewrite.
>
> What would be the best option for me: Apache, nginx or httpd?
The slowcgi(8) daemon in base acts as a translator between FastCGI and old-school CGI scripts. #3 is still a valid option, if you just want to use base openbsd
By Puffy_eat_Python (193.200.119.132) on
Comments
By Eugene Yunak (178.136.235.180) e.yunak@gmail.com on www.yunak.eu
Yeah. But it looks like it would never be accepted, so I won't bother drafting a diff.
By Nik (95.158.188.141) on
Comments
By journeysquid (Tor) on http://www.bsdnow.tv/
Both nginx and httpd will be in 5.6, nginx will be removed thereafter.
By Bliss (125.237.32.170) on
peace
W