Contributed by jose on from the big-network dept.
In particular, I'm curious about the feasibility of having a system dedicated to compiling source/ports/etc and then installing on my production systems. Or does everyone just CVS source updates and recompile on all their systems? My thought here would be to reduce the number of systems that need access through the firewall for installing ports and grabbing the latest CVS.
I'd be interested to hear the approaches others use for this."
(Comments are closed)
By Jim () on
Thoughts.?.?.
Comments
By Brett Eisenberg () on
Comments
By Vincent () on http://www.openbsd.org/cgi-bin/cvsweb/src/etc/Make
By Anonymous Coward () on
By Anonymous Coward () on
However the sources where just fine - there must have been a problem with the distcc configuration or a bug/problem with distcc ..
It probably didn't succeed becaues I didn't try extremly hard - but it aint easy.
And that is very sad because it would have been very cool to combine the force of all my boxes to compile OpenBSD =) - but I'll survive om my 1.4 Ghz workstation
By Marc Espie () espie@openbsd.org on mailto:espie@openbsd.org
But building the whole ports tree on a box, and reusing the resulting packages on several machines is supported.
In current, you can grab packages through scp, and more improvements are going to happen before 3.5.
Comments
By Jim () on
I found the cross-tools et al in src/Makefile that led me astray. Ah well.
By Jordan Dubuc () on
If it's a kernel patch I'm going to be applying, I of course go back and re-apply any previous kernel patches to the source, along with the new one(s). Compile, make install (or copy new kernel over old), then remove source from machines.
Our BSD firewall server doesn't have the compiler tools on it normally, so in order to do patching I first have to mount a "tools CD" with all the necessary binaries.
By tom () tom@replic8.net on mailto:tom@replic8.net
if your machines can be managed remotely (serial console for instance) netbooting is very handy to upgrade as you don't have to run around hassling with medias.
upgrading packages can be done in a similiar way. keeping them on the distribution server and creating some scripts to wrap around pkg_add (...) can do the job using local cron jobs or centralised using public key authentication with openssh.
there are many more approaches... just think :)
Comments
By Anonymous Coward () on
Comments
By tom () tom@replic8.net on mailto:tom@replic8.net
i'm using intel based nics (fxp) pxe.
By Michael Sullenszino () no spam at sullenszino org on mailto:no spam at sullenszino org
I originally did this as a whim with no intenet of supporting more than 3 or 4 servers that were on site. But it has worked so well in practice (i.e. even replacing libc and /bin and /sbin has been a breeze) that I am now supporting off site boxen with fixed IPs and no problems to date.
As for the firewall, the updating script has a routine that checks for the current mountd ports used and updates the firewall to open those up for a short period of time to the IP's that will be using the master server for a 'make install'. Then you need to keep a list of all the directories that have been patched with a note as to whether you need the Makefile.bsd-wrapper, put that in a perl hash and away you go. This also helps in rolling out a new server. When I bring out a new device I can just patch it to current from the master server...
I would appreciate any input with people who have more NFS experience than as to any pitfalls that this might bring about that I have not run into yet.
Comments
By Anonymous Coward () on
Comments
By Michael Sullenszino () no spam at sullenszino com on mailto:no spam at sullenszino com
3 things to note:
-----------------
1) the CVS master has ssh dsa keys on all servers, including the firewall allowing for remote passwordless root access. The security implications should be clear, you want this box behind a firewall and locked up!
2) I put this script at the beginning of the script that goes out and updates the server and then put it at the end with the "off" argument to close off the firewall again.
3) The firewall has this as part of its pf.conf
--------------------------
# Current portmapped mountd ports
nfs_tcp="937"
nfs_udp="642"
nfs_server="192.168.0.22"
table { 192.168.0.1/32, ...}
[more stuff here]
# Port Forward traffic to CVS master
rdr on $ext_if proto udp from any to $external_addr port $nfs_udp -> $nfs_server port $nfs_udp
rdr on $ext_if proto tcp from any to $external_addr port $nfs_tcp -> $nfs_server port $nfs_tcp
rdr on $ext_if proto tcp from any to $external_addr port 2049 -> $nfs_server port 2049
rdr on $ext_if proto udp from any to $external_addr port 111 -> $nfs_server port 111
....
# Pass in rdr nfs traffic
pass in on $ext_if proto udp from to $nfs_server/32 port { 111, $nfs_udp } keep state
pass in on $ext_if proto tcp from to $nfs_server/32 port { 2049, $nfs_tcp } flags S/FSRPAU keep state
pass in on $ext_if proto tcp from to $nfs_server/32 port > 500 flags S/FSRPAU keep state
pass in on $ext_if proto udp from to $nfs_server/32 port > 500 keep state
---------------------------------------------
Here is the perl script I use:
----------------------------------------------
#!/usr/bin/perl
# Script to pull the current nfs portmapped ports for udp and tcp,
# then suck down the current firewall pf.conf from firewall. It
# then updates the nfs_udp and nfs_tcp ports gleaned from rpcinfo -p
# Finally, it scp the temp file back up to firewall and executes pfctl to initialize the new firewall.
# Usage: $0 [off]
# If the off argument is given, the script shuts down the port forwarding
$firewall = "firewall.yourdomain.com"; # or ip address
# Get current pf.conf from firewall
@pfconf = `ssh root@$firewall cat /etc/pf.conf`;
# Create temp pf.conf in /root/tmp/
$pf_tmp = "/root/tmp/pf.conf";
open(PF_FH, ">$pf_tmp");
# See if this is a request to turn on or off nfs on the firewall
if ($ARGV[0] eq "off") {
foreach $line (@pfconf) {
chomp($line);
if ($line =~ /^pass.+nfs_server/) {
$line=~ s/^pass/#pass/;
}
print PF_FH "$linen";
}
} else {
# Otherwise get current mountd ports and reload firewall
# Get current NFS/mountd tcp and udp ports
$rpcinfo = `rpcinfo -p`;
@rpcinfo = split(/n+/, $rpcinfo);
foreach $line (@rpcinfo) {
if ($line =~ /udp.+mountd/) {
chomp($line);
$line =~ s/.+([t|u])wws+(d+).+/$2/;
$udpport = $line;
}
if ($line =~ /tcp.+mountd/) {
chomp($line);
$line =~ s/.+([t|u])wws+(d+).+/$2/;
$tcpport = $line;
}
}
# update and print out new pf.conf
foreach $line (@pfconf) {
chomp($line);
# replace tcp port with current
if ($line =~ /^nfs_tcp/) {
$line = "nfs_tcp="$tcpport"";
}
# replace udp port with current
if ($line =~ /^nfs_udp/) {
$line = "nfs_udp="$udpport"";
}
# uncomment any commented out nfs_server lines
if ($line =~ /^#pass.+nfs_server/) {
$line =~ s/^#pass/pass/;
}
print PF_FH "$linen";
}
close(PF_FH);
}
# Send the new file back to firewall
system("scp $pf_tmp root@$firewall:/etc && ssh root@$firewall pfctl -f /etc/pf.conf");
By Matt Van Mater () on
I use it to automatically generate a new release every once in a while. I also and added a few things such as:
-upload new release to local ftp
-grab release from ftp and unpack to / which upgrades my system to stable (after a reboot)
-apply patches and recompile (so you don't have to cvs up to stable)
This isn't useful between versions of obsd (you have to update /etc by hand and things like that) but at least it makes it easy to keep all my systems up to date while only having to cvs and compile one time.
I use it at home on my servers, as well as at work on a handful of boxes.
Comments
By Anonymous Coward () on
Comments
By Sleepy () on http://www.maximumunix.org
we are a linux dev box.but I run OpenBSD on the firewalls and all production(DNS,Mail,etc) servers.
mainly because of the ports collection , security and consistency.I don't install compxx.tgz on the boxes, I do have one machine where I do the untar->configure->compile->install
cycle.I have hacked around SlackBuild scripts(yeah I know, but slack is the best of that breed) and it creates packages for Me,this have been working well for me in the last couple of years.
when it comes to kernel, I usually have 2 kernels
one for the mainstream hardware, and one specific(I got few soekris around for internal usage)
take a look at slackbuild. it would be nice to discover if something similar(or better) exists for OpenBSD.
Comments
By Anonymous Coward () on
By Matt Van Mater () on
By Anonymous Coward () on
This has the juicy bits:
http://www.deadly.org/article.php3?sid=20030319035045
By andre () on http://rootshell.be/~andre
> (you have to update /etc by hand and things
> like that)
You might want to have a look at updiff.sh (http://rootshell.be/~andre/software/updiff.sh).
There's a README file at http://rootshell.be/~andre/software/README.updiff
By Jascha () jascha@localareasecurity.com on mailto:jascha@localareasecurity.com
Comments
By Anonymous Coward () on
http://www.benzedrine.cx/pf.html
The presentation has some comparisons:
http://www.benzedrine.cx/sucon/
By Dom De Vitto () on
By Willb () on
We don't know what the patch will be, but the port is just a fake...it will then go out and download the real patch as if it were a dependency.
When patch 16 comes out, we do a make 16. Ports then fetches the patch source from a repository, compliles, installs, etc.
If only I knew anything about the ports system.
By Christopher () on
Comments
By Peter Hessler () spambox@theapt.org on http://www.theapt.org
Restoring the server (should be. I haven't tested it yet) hsould be as simple as: fetch cvs info, install all of the ports listed, install the config files, and reboot. I'm very careful to edit the cvs version of my config files, rather than the "live" version, to make sure everything is checked in.
By Anonymous Coward () on
"I agree. I even took it a step further. I have a "gold server" where I build releases. Once the "make release" process is complete, I rsync the new install sets to our ftp server. All openbsd servers that I manage have a perl script installed that I wrote that then query /etc/obsd_release, which contains the list of install sets that were used to build the current system i.e. base33.tgz, etc33.tgz, bsd. The perl script fetches all the install sets relevant to the current system, installs, then runs mergemaster to ensure /etc and /var are up to date. I have been doing this for over a year for at least 10 openbsd boxes and it took me ~10 minutes to get them all up to date. I just need a faster "gold server" cause it takes quite a while to build the new install sets. :)"
---
"i usually have some system behind the firewall, generally a faster system with disk to spare, and use it to build releases. see "man release" for that procedure.
then, you transfer the updated .tgz files you need and unpack them, except for etcXX.tgz. if you see that there are changes in /etc to be made, the thing to do is unpack etcXX.tgz in /tmp or some other place, and merge the changes."
Comments
By Anonymous Coward () on
Comments
By James () james@unixninja.us on http://www.astrocreep.net
Send me an email if you are interested.
Comments
By James () james@unixninja.us on mailto:james@unixninja.us
https://www.astrocreep.net/scripts/obsdsysup-0.1.tar.gz
Comments
By James () james@unixninja.us on mailto:james@unixninja.us
https://www.astrocreep.net/scripts/obsdsysup-0.1.tar.gz.asc
Comments
By Anonymous Coward () on
By Jeff () jtharp@smalldark.net on mailto:jtharp@smalldark.net
I definitely agree with the idea of a single master server to use for distributing updates. It also looks like 'make release' may be helpful. Add in a little PERL or shell script glue and some SSH magic and I can probably get what I was looking for.
I didn't mention in the original article, but the perfect niche I've found for OpenBSD is as a reverse proxy for exposing internal webservices to our DMZ apps. This works great because the default install includes all I need: Apache, mod_proxy, mod_ssl, and pf. This allows a few layers of security between the DMZ app and our internal enterprise systems. I've talked about using a similar reverse proxy solution as a layer of security in front of our external websites as well. There was a good article on doing this and including the new mod_security. Just not sure if the performance impact would be bad or not.
By Anonymous Coward () on
By Anonymous Coward () on
https://www.astrocreep.net/scripts/obsdsysup-0.1.tar.gz