Backing Up

Contributed by jose on 2002-09-20 from the CYA dept.

Back in grad school, my laptop died with my thesis on it. I hadn't saved it anywhere, so you can imagine that was a pretty scary moment. Since then, I really learned to keep backups of even my personal stuff. A few resources which round out an earlier story are out to help you figure out how to use some more archiving tools.

Using the pax tool was one of the latest BSD pieces on the OnLamp site.

A brief rsync tutorial also gives some example scripts for backing up your system. As a reference, my brief "backup" command is just a small shell script:

#!/bin/ksh
export RSYNC_RSH=/usr/bin/ssh
rsync -avz ~/* --exclude music/ jose@192.168.0.11:~/rsync/tank/

Keep your data safe!

(Comments are closed)

Comments

By dash () on 2002-09-20 04:22

This is my personal script.
##################################
#!/bin/sh
today=$(date +%m%d)
OPT="--delete --stats"

cd /home/rsync_log/

echo "Backup start" | mail -s "Backupstart" dash at foru.sarang.net

rsync -avz 211.239.150.31::test /Backup/test $OPT 2>&1 > /home/rsync_log/test_$today_log
.
.
.

echo "Backup Done" | mail -s "Backupdone" dash@foru.sarang.net
####################################
of course, backup server opened rsync port.
Comments
1. By Brett Jones () on 2002-09-20 04:51
  
  Just for fun.... lets start sending dash "Backup start" and "Backup Done" emails at random time throughout the day.
  Comments
  1. By dash () on 2002-09-20 06:12
    
    Yes, I want monitoring All Server. ;-P
    my mistake....I worried about that..
By Pascal Lalonde () plalonde@overnet.qc.ca on 2002-09-20 05:54 mailto:plalonde@overnet.qc.ca

I do my backups this way:
/usr/local/bin/gtar zcvf /var/cd_image/backup.tar.gz -T /etc/bkinclude -X /etc/bkexclude >> $BKLOG 2>&1

The contents of bkinclude are:
/etc/
/home/
/root/
/cvs/
/var/named/
/var/www/
/var/backups/
/var/log/
/usr/local/

and bkexclude:/etc/
/home/mp3share
/home/pub

The cd_image directory is later picked by cdrecord, on a CD-RW. (This is my "home server", my personnal data is in /home).

I don't know if this is a good way to do backups, but until now it worked very well. But I'd gladly accept suggestions.
Comments
1. By Anonymous Coward () on 2002-09-20 06:19
  
  you dont backup crontabs or mail or even your kernel.
  the problem with not backing up everything is forgetting what to include, so specify only what to exclude, not what to include.
  Comments
  1. By Pascal Lalonde () plalonde@overnet.qc.ca on 2002-09-20 07:06 mailto:plalonde@overnet.qc.ca
    
    The problem is, I only have 650-700megs of backup space. As I said, this is my personnal server (at home), so I can't afford a SCSI tape or some other big backup media for that.
    
    I don't mind having to spend 2 days rebuilding it. I just want to be able to get all my personnal data back, and configuration data. My backup already takes 340 megs, gzipped. And it could grow very fast. My mail resides under /home, Maildir format. For crontabs, I guess I should add /var/cron/tabs. Maybe I could be more "general" in what I include, and exclude what I don't need. For example, include /var, but exclude /var/cd_image.
By Anonymous Coward () on 2002-09-20 06:02

rsync -avz ~/* --exclude music/ jose@192.168.0.11:~/rsync/tank/
hrm, the only thing i really *cant* reproduce is the music i have on my servers....
By RC () on 2002-09-20 08:25

Am I the only one that uses plain, old-fashioned, dump and restore?
Comments
1. By Anonymous Coward () on 2002-09-20 09:26
  
  nope it's the best method backing up whole filesystems. I do so too
  greets Dennis
  Comments
  1. By vincent () vincent at igc dot ethz dot ch on 2002-09-20 12:04 mailto:vincent at igc dot ethz dot ch
    
    could you elaborate?
2. By Anonymous Coward () on 2002-09-20 09:41
  
  I am using it too. I've tweaked an incremental dump strategy that I've found somewhere on the net to a local disk directory, the files are then fetched to multiple other hosts.
  
  Cheers, Matthias
  Comments
  1. By Anonymous Coward () on 2002-09-21 01:05
    
    Could you post a link to that? Or email a description to helge at olsentech dot net
    
    Thanks in advance :-)
    
    Comments
    
    By Anonymous Coward () fuc952d at tninet.se on 2002-09-20 19:34 mailto:fuc952d at tninet.se
    
    http://www.backupcentral.com/hostdump.html
By danny () danny@partisan.de on 2002-09-20 12:36 mailto:danny@partisan.de

 #!/bin/bash ORACLE_HOME=/usr/local/oracle PATH=/bin:/usr/bin:$ORACLE_HOME/bin SERVER_ROOT=/server-root DATE=`date +%Y%m%d` DAY=`date +%a` MDAY=`date +%d` SERVER="dev pre docs svr presse" dev="documents cgi-bin perl sql bin c" pre="documents cgi-bin" docs="documents cgi-bin" svr="documents" presse="documents" if [ $DAY = Fri ] then touch /server-root/.backup for server in $SERVER do eval DIR=$$server cd $SERVER_ROOT/$server tar czf - $DIR | ssh -C -q backup@backupserver "cat > /var/backup/full/svr_$DATE_$server.tar.gz" done exp username/password@ORADB file=/tmp/expdat.dmp full=y gzip /tmp/expdat.dmp scp -Cq /tmp/expdat.dmp.gz backup@backupserver:/var/backup/inc/svrdb_full_$DATE.dmp.gz rm /tmp/expdat.dmp.gz else for server in $SERVER do eval DIR=$$server cd $SERVER_ROOT/$server tar czf - `find $DIR -type f -newer /server-root/.backup` | ssh -C -q backup@backupserver "cat > /var/backup/inc/svr_inc_$DATE_$server.tar.gz" done touch /server-root/.backup exp username/password@ORADB file=/tmp/expdat.dmp owner=svr1 gzip /tmp/expdat.dmp scp -Cq /tmp/expdat.dmp.gz backup@backupserver:/var/backup/inc/svrdb_svr1_$DATE.dmp.gz rm /tmp/expdat.dmp.gz fi 
Comments
1. By danny () danny@partisan.de on 2002-09-20 12:39 mailto:danny@partisan.de
 
 sorry, was my 1st try....
By Jim () on 2002-09-20 14:03

Anyone get rsync&&ssh running on Windoze? My goal is to use rsync to pull changes from a windoze client via ssh to my openbsd backup machine. In the meantime I thought about making the openbsd machine a smb server and mount the share on the windoze machine, but I don't really want to go there.
Comments
1. By Shane () on 2002-09-20 14:08
  
  I've run rsync and ssh on Windows 2000. I used an OpenSSH port by Network Simplicity (http://www.networksimplicity.com/openssh/). I forget where I got rsync for Windows, but you should be able to Google for it. The setup was pretty easy, if I remember. I could then rsync stuff through ssh from my OpenBSD box to my Windows box. It was pretty useful.
  Comments
  1. By deekayen () on 2002-09-20 14:42 http://openbsddiary.org/
    
    cygwin has rsync
    
    Comments
    
    By Shane () on 2002-09-21 00:33
    
    Yes, but the OpenSSH port by Network Simplicity is not designed to run with Cygwin. It uses the Cygwin dll's, but the two clash (so I'm told) if you try to install both. You can get rsync for Windows without installing Cygwin.
2. By Anonymous Coward () on 2002-09-21 15:13 http://www.cis.upenn.edu/~bcpierce/unison/
  
  Thanks to ocaml, it runs on Linux, Window$, MacOS X and, of course, OpenBSD.
By Coward () on 2002-09-20 14:18
By Coward () on 2002-09-20 14:18
By Anonymous Coward () on 2002-09-20 15:03 http://www.cis.upenn.edu/~bcpierce/unison/

A great tool for synchronizing data between PCs is unison :

http://www.cis.upenn.edu/~bcpierce/unison/
Comments
1. By William () on 2002-09-20 19:36
  
  I was curious if you actually built this on OpenBSD. It looks interesting, but I have not been able to build it on -current.
  Comments
  1. By Anonymous Coward () on 2002-09-21 09:27 http://www.cis.upenn.edu/~bcpierce/unison/
    
    I had no problem compiling it in -stable. What I have done :
    
    1 - download latest ocaml compiler : http://caml.inria.fr/
    2 - install ocaml
    3 - download unison source file
    4 - compiling it
    
    Comments
    
    By William () on 2002-09-21 09:46
    
    I installed ocaml 3.06 from ports and followed the same procedure from there, but no deal. I guess I'll mess with it some more.
    
    Comments
    
    By William () on 2002-09-21 10:05
    
    Never mind. I just realized that I needed to compile with gmake since I have not replaced make with gmake.
By Jedi/Sector One () j@pureftpd.org on 2002-09-20 16:06 http://www.pureftpd.org/

When you got plenty of sub-directories, rsync crawls like hell, takes all the available cpu+memory+swap, and crash. Even backuping 40gigs with rsync doesn't seem possible.

I switched to ssync, a lightweight tool to sync two (locally mounted) directories. It works extremely well and fast for backups.
By Kyle Amon () amonk@backwatcher.com on 2002-11-01 17:44 http://www.backwatcher.com/

Some might like to check out DataCrypt-0.10.tar.gz
at...

http://www.backwatcher.org/index.php?page=software

It is a perl program (with a couple config files) for making efficient, secure backups of large numbers of remote unix machines with rsync over ssh.

It could be made to backup winblows machines too, but since I have no need for that, I never bothered with it. It's GPLd, so add that if you like. :-)

BTW, there is very little documentation in that tarball at present, but then if you need more than that, you probably need to backup winblows machines easily anyway. :)

Later,

-- Kyle

Latest Articles

Sat, Nov 30
- 18:42 Initial list of 21 EuroBSDcon 2024 videos released (0)
- 18:39 Rafael Sadowski on dpb(1) (1)
Fri, Nov 29
- 09:31 -current now has more flexible performance policy (0)
Mon, Nov 25
- 07:13 unwind: wildcard in blacklist (0)
Fri, Nov 22
- 07:33 Game of Trees 0.106 released (0)
Fri, Nov 15
- 06:51 Game of Trees 0.105 released (0)
Wed, Oct 23
- 07:12 Game of Trees 0.104 released (0)
Tue, Oct 15
- 08:46 LibreSSL 4.0.0 Released (0)
Mon, Oct 14
- 07:05 sshd(8) splitting continues (1)

Credits

Copyright © 2004-2008 Daniel Hartmeier. All rights reserved. Articles and comments are copyright their respective authors, submission implies license to publish on this web site. Contents of the archive prior to April 2nd 2004 as well as images and HTML templates were copied from the fabulous original deadly.org with Jose's and Jim's kind permission. This journal runs as CGI with httpd(8) on OpenBSD, the source code is BSD licensed. undeadly \Un*dead"ly\, a. Not subject to death; immortal. [Obs.]