OpenBSD Journal

Apache Reverse Proxy on OpenBSD

Contributed by Dengue on from the lest-I-forget-how-to-do-this-the-next-time dept.

Recently, I've been working on creating a community wireless network near my home in beautiful near-south Lincoln, NE. (Yeah, you laugh.) I'm planning on using Tomcat to host some web applications for users of this network, but I the only operating system I'm comfortable exposing to untrusted networks is OpenBSD. So, after a read of the Tomcat docs, I decided to use OpenBSD's chrooted Apache to proxy requests to the Tomcat server, which I can keep tucked nicely away in a private DMZ.

What follows are the steps I had to take to get this particular setup to work.

Out of the box (so to speak) OpenBSD provides mod_proxy as an apache DSO. I want to run this baby chrooted and as cleanly as possible, so we need to make some changes to support this.

  1. Rebuild Apache with static support for mod_proxy
    • Edit src/usr.sbin/httpd/Makefile.bsd-wrapper to include support for a static mod_proxy
      • Make sure "--enable-module=proxy" exists
      • Remove line "--enable-shared=proxy" to force a static module.
    • Rebuild Apache
      • rm config.status (this will force apache to reconfigure and pick up the changes you've just made)
      • make -f Makefile.bsd-wrapper clean
      • make -f Makefile.bsd-wrapper obj
      • make -f Makefile.bsd-wrapper depend
      • make -f Makefile.bsd-wrapper
      • /usr/src/usr.sbin/httpd/obj/src/httpd -l should report mod_proxy.c in output of statically compiled modules.
      • make -f Makefile.bsd-wrapper install
  2. Configure Apache as a reverse proxy
    • Edit httpd.conf
      • <IfModule mod_proxy.c>
                ProxyRequests   Off
                CacheRoot       "/var/www/proxy"
                ProxyPass        /jsp-examples http://remus_hme0:8081/jsp-examples
                ProxyPassReverse /jsp-examples http://remus_hme0:8081/jsp-examples
        </IfModule>
        
        This tells Apache to proxy all requests for HOSTNAME/jsp-examples to the host remus_hme0:8081/jsp-examples. Also, by using the "CacheRoot" directive, the output will be cached for greater speed and less load. Make sure the CacheRoot is owned by user www.
    • Create /var/www/etc/hosts
      • Since apache is chrooted, we need to create a hosts file for it to resolve the name of the application server. Using an ip address in httpd.conf for the app server failed with "Bad file descriptor" errors. YMMV
  3. Configure on the app server a proxy connector for Tomcat.
    • Edit server.xml. Here's an abbreviated example of a connector:
      • <Connector port="8081"
         proxyName="www.yourserver.net"
         proxyPort="80"/>
        
  4. Verify apache configuration and restart.
What we wind up with when we are done is the standard chrooted Apache instance running on OpenBSD except that now, it proxies all requests for the URI /jsp-examples to a host named remus_hme0 on port 8081. The Tomcat instance on remus knows (because of the connector we created) to handle the request and return the results to the reverse proxy. Doing this, we can keep our app server safely away from inbound traffic except for the traffic being proxied to it. Be careful however, since the proxied traffic can still reach the app server, it is not immune to attacks. Data driven attacks using the HTTP protocol will still reach it. You need to write web applications with security in mind.

(Comments are closed)


Comments
  1. By Christopher Kruslicky () on

    I'm curious why the port 8081 proxying step is done if tomcat is running on a seperate host.

    I've used the proxy to send different host/domain requests to another machine running seperate chroot'd apache installs per site. It works out rather nice, and it seems to work fine using IP:port instead of hostname in the httpd.conf of the proxy. The only differences I can think of are that I do not compile the proxy module in statically, and I specified each proxy mapping in a seperate virtual host directive "ProxyPass/Reverse" rule.

    Comments
    1. By dengue () on

      2 hosts, 1 dual-homed. Traffic arrives on the pub_if of hostA. Apache proxies it via the private_if residing on a LAN segment with hostB which only has a private_if and knows no routes outside of the LAN it is on. hostA doesn't forward packets from hostB. As for Apache magic, were you running the Proxy chroot'd with a DSO? Can you post an example? I'm always looking for a better way to do things.

      Comments
      1. By Christopher Kruslicky () anub-deadly.200301@livenudefrogs.com on mailto:anub-deadly.200301@livenudefrogs.com

        Yes using the DSO. What I did was have a gateway, hostA, and setup the Apache Proxy there. That host also runs static content from the same apache chroot. The internal machine, hostB, has three seperate Apache chroots, each running on a different (tcp) port. Some httpd.conf examples modified so the "tags" aren't interpreted:

        hostA:
        LoadModule proxy_module /usr/lib/apache/modules/libproxy.so

        [IfModule mod_proxy.c]
        ProxyRequests Off

        [Directory proxy:*]
        Order deny,allow
        Deny from all
        [/Directory]

        [Directory proxy:http://internal1.example.net/]
        Order deny,allow
        Allow from all
        [/Directory]

        [Directory proxy:http://internal2.example.net/]
        Order deny,allow
        Allow from all
        [/Directory]

        ProxyVia Off
        [/IfModule]

        [VirtualHost 192.168.1.1]
        ServerName internal1.example.net
        ProxyPass / http://10.1.1.1:81/
        ProxyPassReverse / http://10.1.1.1:81/
        CustomLog logs/access_log.int1 combined
        [Location /]
        Order allow,deny
        Allow from all
        [/Location]
        [/VirtualHost]
        [VirtualHost 192.168.1.1]
        ServerName internal2.example.net
        ProxyPass / http://10.1.1.1:82/
        ProxyPassReverse / http://10.1.1.1:82/
        CustomLog logs/access_log.int2 combined
        [Location /]
        Order allow,deny
        Allow from all
        [/Location]
        [/VirtualHost]


        hostB is fairly straightforward in that there's a few directories like /var/www/int1 and /var/www/int2 - each of these is a stripped down version of what comes in /var/www by default (minus documentation) plus a /var/www/int1/usr directory to hold a copy of everything needed by perl and some things in /usr/lib that apache needs. A for loop in rc.local loops through and starts each one, the individual httpd.conf files specify listen port and chroot dir to match its location:

        if [ "X$wwwdomains" != X"NO" -a -x /usr/sbin/httpd ]; then
        for conffile in $wwwdomains; do
        echo -n "HTTPD: starting $conffile ..."
        /usr/sbin/httpd -f /var/www/$conffile/conf/httpd.conf
        echo " done."
        done
        fi

        and rc.conf has a line specifying each directory name with a live chroot:
        httpd_flags=NO
        wwwdomains="int1 int2"


        The drawback I find with this approach is that the access_log on hostB is not so useful because of the requests coming from hostA. That's why each domain gets it own customlog on hostA.

        I'm not sure this way is really any better, I just hadn't come across the issue you mentioned with needing hostnames in the config =) It just dawned on me, another thing that might affect that is having reverselookups off?

  2. By Anonymous Coward () on

    You can always use squid for that, performance wise seems a better choice.
    Or just use apache with mod_jk...
    Btw for tomcat you can stick with blackdown 1.3.1 jvm with sunwjit and jikes.

  3. By Menno Duursma () on

    I'd probably setup Pond for this kind of thing:
    http://www.apsis.ch/pound/

    Comments
    1. By Anonymous Coward () on

      Unless you deserve to cache your data to avoid tomcat "shortcomings".

      Comments
      1. By sthen () on

        Caching can be an advantage or a disadvantage depending on the situation (-:

        Pound has an advantage over general-purpose proxies: it is pretty good at detecting failed backend servers (if no data is received within a set time, it can send the request to another server without informing the client).

        Another nice use for Pound is SSL offloading (I think Apache proxy can do this too, but I don't think Squid or Oops do this).

        Here's a nice trick if you'd like resilience and load balancing: use CARP (or functional equivalent) on the servers, place a pf box before them on the network, and load-balance between the resilient IP addresses with an address pool in a rdr rule. This same trick works nicely for other protocols too, and I think it's quite an improvement over listing multiple A records (which is of course impossible where an IP address must be specified, e.g. DNS resolvers).

        Comments
        1. By Anonymous Coward () on

          squid can do that or you can also use ssltunnel.

  4. By Lennie () leen@wirehub.nl on mailto:leen@wirehub.nl

    You might wanna use <a href=http://www.modsecurity.org/>mod_security</a> ? <br> <br> or an other similair module.

  5. By J2 () testuser99@hotmail.com on mailto:testuser99@hotmail.com

    OpenBSD in Lincoln? And I thought I was the only one... Nice!

  6. By John Wright () on

    I've got OpenBSD 3.3 loading the distributed apache DSOs without having them in the chrooted directory.

    I guess apache chroots after parsing the LoadModules.

    Comments
    1. By Allan (24.68.59.30) on

      > I've got OpenBSD 3.3 loading the distributed apache DSOs without having them in the chrooted directory.
      >
      >
      >
      > I guess apache chroots after parsing the LoadModules.

      On my 4.0 system, I noted that "apachectl restart" does not yield running instances of httpd with LoadModule for proxy_module enabled. However, if I start Apache from the command line (root prompt) "httpd -DSSL" the proxy module loads and Apache starts and functions as expected.

      The module loads without chrooting the module path. I tested this several times without special thoroughness.

      Is there possibly some reason apachectl has an allergy to LoadModules for a chrooted httpd?

    2. By Anonymous Coward (80.124.186.106) on

      > I've got OpenBSD 3.3 loading the distributed apache DSOs without having them in the chrooted directory.
      >
      >
      >
      > I guess apache chroots after parsing the LoadModules.
      >
      >

      It seems not running with OpenBSD 4.4...

      Still investigating.

Credits

Copyright © - Daniel Hartmeier. All rights reserved. Articles and comments are copyright their respective authors, submission implies license to publish on this web site. Contents of the archive prior to as well as images and HTML templates were copied from the fabulous original deadly.org with Jose's and Jim's kind permission. This journal runs as CGI with httpd(8) on OpenBSD, the source code is BSD licensed. undeadly \Un*dead"ly\, a. Not subject to death; immortal. [Obs.]