back up scripts

[bhoff]

This is what I hope will be a long run on scripting for back-ups. Feel free to post or take something that someone else does and add some functionality and share it with the group.

My scripts and windows installers will be fourth coming..

[oddball]

Here's my backup script:

Code:

#!/bin/bash

tar -X exclude.txt -czvf $(date +%d.%m.%Y).tar.gz ~/ ; sudo tar -czvf $(date +%d.%m.%Y)_logs.tar.gz /var/log/

The date stuff is to give the resulting tarballs the current date, in the form of dd.mm.yyyy.

The exclude.txt is a listing of stuff I don't want backed up, and it currently contains:

Code:

~/tmp/
*.tar.gz
*.iso
~/.thumbnails
~/.Trash

[bhoff]

right now I am using rsync and ssh to make this work for our windows clients I am using cygwin on windows to give me the basic Linux commands.
If you have not looked at this you must check it out it is the most awesome tool out there for bringing windows into Linux environment.
I have bash,vim,ssh, and rsync and all the core utilities right from a C: prompt very handy.
As for my installer to setup every thing and install what I want and the scripts I am using Nullsoft Scriptable Install System. This allows me to package every thing up into a nice
installable that I can send to remote clients and install without being there.

This works really well from across the room or across the country.
I use the windows automated task manager to run the scripts and of course cron on Linux.

This is the basic overview. I will have the scripts coming as I clean them up.

And the things I am working on is checking who did not backup and email me a pass/fail list.
to have the client try a secondary server if the primary is down and also a file that the client can drop a file into and it creates a shortcut to that file for the script to add that to the backups.

Right now I am using this between 2 Linux scp servers and it runs every 15 minutes with great success..

So let me know if you have any ideas.

[bhoff]

For those following this Check out deltacopy for windows and mac it is a gui front end for cygwin&rsync that allows for easy setup for backups from one machine to another and even works with ssh.
Also has nice feature to allow restore from remote computer and email notification on success and fails of backups.
Also everyone should be aware of cygwin this utility lets you run linux commands in Windows very handy for those of us that work in both fields.

[oddball]

here's my current backup scripts:

Code:

#!/bin/bash
backup_logs;backup_root;rpm_backup;backup_main

Each command is another script.

backup_logs:

Code:

#!/bin/bash
sudo tar -czvf /crap/$(date +%d.%m.%Y)_logs.tar.gz /var/log/

backup_root:

Code:

#!/bin/bash
sudo tar -X /crap/exclude_root.txt -czvf /crap/$(date +%d.%m.%Y)_root.tar.gz /bin /boot /etc /lib /opt /root /sbin /srv /usr /var; sudo -k

rpm_backup:

Code:

#!/bin/bash
rpm -qa >> ~/packages_$(date +%d.%m.%Y).txt

backup_main:

Code:

#!/bin/bash
tar -X /crap/exclude.txt -czvf /crap/$(date +%d.%m.%Y).tar.gz ~/ /usr/share/fortune/ /usr/share/icons/ /usr/share/wallpapers/ /etc/zypp/repos.d/; split -b 4400M --verbose -d /crap/$(date +%d.%m.%Y).tar.gz /crap/$(date +%d.%m.%Y)_part

exclude_root.txt contains:

Code:

/var/cache
/var/lock
/var/run
/var/tmp

exclude.txt contains:

Code:

~/tmp/
*.tar.gz
*.iso
~/.thumbnails
~/.Trash

[green_dragon37]

How about mine:  It creates ISO images and burns them to a cd-rw sitting in the drive, I'm not sure how well it works any more though, I haven't used it in a few years and just pulled it out of what I thought was non-existence, while cleaning up my drive!

Code:

#!/bin/bash
#
# This is a _full_ backup script.  It doesn't care about
# file creation times, or anything except the diretories
# you tell it to back up. :)
#

# Customize these to match your site setup
MKISOFS="/usr/bin/mkisofs"
MKISOFSOPTS=" -o"

CDRECORD="/usr/bin/cdrecord"
CDRECORDOPTS="dev=0,0,0 speed=12 -tao"
CDRECORDOPTS_BLANK="dev=0,0,0 speed=12 blank=fast"

TAR="/bin/tar"
TAROPTS=" --create --bzip2 --file"

# Even more config settings
backup_dir="/tmp/backups".`date +%Y%m%d`

# OK, no more changes :p
opts=$*

# Version Information
echo "`basename $0` Version 0.10a -- Mar 2004"

# First, a help option
for opt in $opts; do
  if [ "$opt" = "-u" ]; then
    echo "`basename $0` [-h] [-e] [-w] DIR1 [DIR2...]"
    exit 0
  elif [ "$opt" = "-h" ]; then
    echo "`basename $0` -- Useful for backing up to CD media
    -u Usage information
    -w Write to CD media
    -e Erase CDRW (implies \"-w\")
    -h Show this help"
    exit 0
  elif [ "$opt" = "-e" ]; then
    opts=${opts#-e}
    erase=1
    write=1 # Erase implies write
    #echo $opts
  elif [ "$opt" = "-w" ]; then
    opts=${opts#-w}
    write=1
  fi
done

# Then, check for problems between the keyboard and chair
if [ -z "$2" ] ; then
  echo "Not enough arguments, try '`basename $0` -h' for help"
  exit 15
fi

# Finally, It begins...
mkdir -p "$backup_dir/iso" "$backup_dir/tar" "$backup_dir/tbz2"

for i in $opts; do
  archive="$backup_dir/tbz2/`echo $i | sed s/\\\//_/g`.tbz2"
  echo "Backing up $i to $archive"
  echo "================"
  $TAR $TAROPTS $archive "$i"
  echo
done

# Making one big tar
cd "$backup_dir/tbz2"
$TAR --create --file "$backup_dir/tar/`hostname`_`date +%Y%m%d`.tar" *

# Making the iso
echo "Making ISO file."
echo "================"
$MKISOFS $MKISOFSOPTS "$backup_dir/iso/`hostname`_`date +%Y%m%d`.iso" $backup_dir/tar/*.tar
echo "ISO file complete."
echo "=================="

if [ "$erase" = "1" ]; then
  # Blanking disc
  echo "Blanking CDRW."
  echo "=============="
  $CDRECORD $CDRECORDOPTS_BLANK
  echo "Blanking complete."
  echo "=================="
fi

if [ "$write" = "1" ]; then
  # Burning to disc
  echo "Burning ISO to disc."
  echo "===================="
  $CDRECORD $CDRECORDOPTS $backup_dir/iso/*.iso
  echo "Burning complete."
  echo "================="
fi

# And, We're done!
echo -e "\n`basename $0` $* done."

[sontek]

The problem I see with most backup scripts posted here is they are re-backing up everything everytime they run, which wastes hard drive space and bandwidth if you are backing it up to a remote location (I would recommend this, I use Amazon S3 Webservice, JungleDisk is also a good backup service, it uses Amazon).

I use rdiff-backup for my backs... It does a combination of incremental backups and mirroring. Here is what I do:

Code:

#!/bin/sh

# Where the backups go, this is an sshfs mount for me
DESTINATION="/home/sontek/backups/"
LOG="/var/log/backup/"
TIMESTAMP=`date +%m%d_%H%M`

# How long to keep backup history (6 months)
MAXAGE="6M"

# Options from man page
OPTIONS="--force --print-statistics --exclude **.iso"

# Make sure $DESTINATION exists
if [ ! -d "$DESTINATION" ]; then
    echo "Error: '$DESTINATION' does not exist!!"
    exit 1
fi

# Check if rdiff-backup exists
if ! which rdiff-backup; then
    echo "You need to install rdiff-backup"
    exit 1
fi

# Just put all the folders you want backed here.
for SOURCE in "/home/sontek/dotfiles/"
do
    if [ ! -d "$DESTINATION/$SOURCE" ]; then
        echo "Creating folder $DESTINATION/$SOURCE"
        mkdir -p "$DESTINATION/$SOURCE"
    fi

    echo "Backup: rdiff-backup $OPTIONS $SOURCE $DESTINATION/$SOURCE"
    rdiff-backup $OPTIONS "$SOURCE" "$DESTINATION/$SOURCE"

    # It went well, remove stuff older than MAXAGE
    if [ "$?" -eq 0 ]; then
        echo "Cleanup: rdiff-backup --force --remove-older-than $MAXAGE
        $DESTINATION/$SOURCE"
        rdiff-backup --force --remove-older-than $MAXAGE "$DESTINATION/$SOURCE"
    else
        echo $? > $LOG/fail$TIMESTAMP.log
    fi
done

and then if you ever want to grab the latest backup of a specific file:
rdiff-backup --restore-as-of (time) /home/backups/home/sontek/dotfiles/.vimrc

[sontek]

I also maintain my .vimrc, .bashrc, .screenrc, and some util scripts (including this backup script) on github http://github.com/sontek

[green_dragon37]

I would have preferred incremental backups when I came up with my script, but I wasn't keeping around the original ISO images, or keeping the CDR in the computer after I did a backup... I never bothered to go any further with it, either!

[sontek]

Haha, yeah... I don't trust myself with any physical media (usb sticks, dvds, tapes, etc), so I stick with all remote backups over the network either with rsync or sshfs + rdiff-backup. Its actually pretty fast after the initial backup and pretty easy to restore, and makes it much easier to do nightlies since you don't have to remember to put a new disk in the drive every night.

[Yossarian]

I feel bad.  Whenever I do backups I just manually copy my files to an external hard drive.  Nothing fancy or impressive about that!

[sontek]

My backup strategy is just another way to hide my inability to be organized... If I waited until I was formatting / thought of doing a backup, it'd never get done... So I have this script and just set it up in cron right after a format and just never think about it, and then 6 months down the road when I format and get the *OH CRAP* moment thinking of all the things I didn't back up, they are magically sitting on my Amazon S3 server