electroZ microsystems

BackupStrategy


Voici un script simple qui permet de faire le backup d'un server Linux. Le backup fait une version par jour durant 7 jours, c'est à dire que si on modifie un fichier tous les jours on pourra retrouver 7 versions du fichier. Seul les fichiers modifiés sont réécrit. Un rapport de backup est envoyé par email. Les bases de données MySQL spécifiées sont backupées.

Pour faire le backup d'un ordinateur Windows on monte un système de fichier partagé par l'ordinateur avec cifs puis on fait le backup à partitr du point de montage.


#!/bin/bash
# This script execute every thing to backup a website. 
# This script should be called from a crontab every working day at 20h00.
# cron:"00  20  *  * 1-5  root    /path/backup_strategy.bash"
# ED     fev 2011 |Created
# ED  17 fev 2011 |Added 7 days versioning by hard linking
# ED  17 fev 2011 |Send Report by e-mail
# ED  22 mar 2011 |Verify if the Day-0 backup is older than 1 Day so we can run 
#                 |the script in any time every 10min for instance


#### CONFIGURATION  ####
LOCAL_BACKUP="/backups"
REMOTE_BACKUP="namez@host:/backups"
SOURCE="/home"
REPORT_FILE="rootuser/BackupReport.txt"
EMAIL="the@mail.com"
SUBJECT="HOME DAILY BACKUP"
DBNAME="MySQL_DB"
DBUSER="user"
DBPASS="pass"
########################

#Test if the directory Day-0 from the backup directory is older than 1440min (23h30)
#so do the backup
if [ $(find $LOCAL_BACKUP -maxdepth 1 -type d -name Day-0 -mmin +1400  |wc -l) -eq 1 ]
then
   (
   time (
      echo ======= $SUBJECT ============================
      date
      echo 
      echo SOURCE DIRECTORY = $SOURCE
      echo
      echo LOCAL DESTINATION = $LOCAL_BACKUP
      echo
      echo REMOTE DESTINATION = $REMOTE_BACKUP
      echo
   
      echo ======= MSQL DUMP ========================================================
      mysqldump  --opt $DBNAME  > $SOURCE/database/$DBNAME_`date +\%Y\%m\%d\%H\%M\%S`.sql -u$DBUSER -p$DBPASS
      echo
   
   
      echo ======= SUPRESS OLD HARDLINKS DAY-7 AND DO THE DAY ROTATION ========
      #see http://www.mikerubel.org/computers/rsync_snapshots
      rm -rf $LOCAL_BACKUP/Day-7
      mv $LOCAL_BACKUP/Day-6     $LOCAL_BACKUP/Day-7
      mv $LOCAL_BACKUP/Day-5     $LOCAL_BACKUP/Day-6
      mv $LOCAL_BACKUP/Day-4     $LOCAL_BACKUP/Day-5
      mv $LOCAL_BACKUP/Day-3     $LOCAL_BACKUP/Day-4
      mv $LOCAL_BACKUP/Day-2     $LOCAL_BACKUP/Day-3
      mv $LOCAL_BACKUP/Day-1     $LOCAL_BACKUP/Day-2
      if [ ! -d $BACKUP/Day-0 ]; 
      then
         mkdir  $LOCAL_BACKUP/Day-0
      else 
         cp -al $LOCAL_BACKUP/Day-0 $LOCAL_BACKUP/Day-1
         touch  $LOCAL_BACKUP/Day-0
      fi 
    
      echo
      echo ======= BACKUP USEFULL FILES TO LOCAL_BACKUP ================
      rsync -av --delete $SOURCE/uploadedFiles  $LOCAL_BACKUP/Day-0
      rsync -av --delete $SOURCE/release        $LOCAL_BACKUP/Day-0
      rsync -av --delete $SOURCE/images         $LOCAL_BACKUP/Day-0
      rsync -av --delete $SOURCE/etc            $LOCAL_BACKUP/Day-0
      rsync -av --delete $SOURCE/database       $LOCAL_BACKUP/Day-0
   
      echo
      echo ======= RE-BACKUP TO THE DISTANT SERVER =====================
      #db:/root/.ssh/id_rsa.pub needs to be copied to /cs1:/home/usernam/.ssh/authorized_keys file
      rsync -av   /home/edroz/db_backups/Day-0/* edroz@cs1.epfl.ch:/export/users/edroz/db_backups/Day-0/
   
      echo
      echo ======= TOTAL BACKUP TIME ============================
      )
   ) &>$SOURCE/$REPORT_FILE
      
      #### BACKUP REPORT FILES TO LOCAL_BACKUP ####
      rsync -a  $SOURCE/$REPORT_FILE     $LOCAL_BACKUP/Day-0/$REPORT_FILE

      #### SEND THE REPORT BY E-MAIL ####
      mail -s "$SUBJECT" "$EMAIL" < $SOURCE/$REPORT_FILE

#else
#   mail -s "$SUBJECT" "$EMAIL" <<EOF
#      Nothing to do
#EOF
fi

Basé sur http://www.mikerubel.org/computers/rsync_snapshots

On peut rendre ce script plus generique de la facon suivante. On oublie pour le moment la partie MySQL. deux fichiers sont utilise: un pour le script et l'autre pour la configuration.

Configuration Daily.cfg:

#### CONFIGURATION  ####
SUBJECT="Daily Backup TRAIN/EDROZ"
BASENAME="Day"
BASETIME="-mmin +1400 "
SOURCE="/mnt/train/edroz/*"
BACKUP="/misc/backup/train/edroz"
REPORT_FILE="DailyBackupReportTrainEdroz.txt"
REPORT_PATH="/home/edroz/etc/log"
EMAIL="info@electroz.org"
PROFONDEUR=7

Configuration Weekly.cfg qui se base sur le dernier backup Daily:

#### CONFIGURATION  ####
SUBJECT="Weekly Backup TRAIN/EDROZ"
METHODE="BRANCHE"
BASENAME="Week"
BASETIME="-mtime +7"
BACKUP="/misc/backup/train/edroz"
SOURCE="$BACKUP/Day-0/*"
REPORT_FILE="WeeklyBackupReportTrainEdroz.txt"
REPORT_PATH="/home/edroz/etc/log"
EMAIL="info@electroz.org"
PROFONDEUR=4


Script backup.bash:

#!/bin/bash

# This script execute every thing to backup a website. 
# This script should be called from a crontab every working day at 20h00.
# cron:"00  20  *  * 1-5  root    /path/backup_strategy.bash"
# ED     fev 2011 |Created
# ED  17 fev 2011 |Added 7 days versioning by hard linking
# ED  17 fev 2011 |Send Report by e-mail
# ED  22 mar 2011 |Verify if the Day-0 backup is older than 1 Day so we can run 
#                 |the script in any time every 10min for instance
# ED   2 apr 2011 |Generic script and configuration file
# ED   6 apr 2011 |Support unmounted source filesystem
# ED   7 apr 2001 |Improve first use behavior (links or copy)


#### TEST CONFIGURATION FILE ####
if  [ ! -f $1 ]
then
   echo "ERROR:A configuration file is needed as argument of this command"
   exit
fi
source $1
if [ -z "$SOURCE"  ]
then
   echo "ERROR:SOURCE variable is not defined in the configuration file" 
   exit
fi

# Check if the source is accessible. Some time if the source is a network filsystem
# it is may be not mounted. We check it there is at least one file on the source directory.
if [ $(ls -l  $SOURCE 2>/dev/null  | wc -l) -eq 0 ]
then
   echo "La source n'est pas accessible ($SOURCE)"
   exit   
fi


# Configuration if not configured and first time use
if [ ! -d $BACKUP/$BASENAME-0 ]
then
   mkdir  $BACKUP/$BASENAME-0
   if [ $METHODE == "BRANCHE" ]
   then
      #no new spcae used, only links (For Weekly, Monthly from Day-0)
      echo "cp -al $SOURCE $BACKUP/$BASENAME-0"
      cp -al $SOURCE $BACKUP/$BASENAME-0   
   else
      echo "Rsync??"
      exit
      #All files copied (For original file to Day-0)
      rsync -av --delete  $SOURCE   $BACKUP/$BASENAME-0
   fi
fi

#Test if the directory Day-0 from the backup directory is older than 1440min (23h30)
#so do the backup
if [ $(find $BACKUP -maxdepth 1 -type d -name $BASENAME-0 $BASETIME  |wc -l) -eq 1 ]
then
   (
   time (
      echo ======= $SUBJECT ============================
      date
      echo LOCAL DESTINATION = $BACKUP
      echo

      echo ======= BACKUP ROTATION ========
      #see http://www.mikerubel.org/computers/rsync_snapshotsi
      echo rm $BACKUP/$BASENAME-$PROFONDEUR
      rm -rf $BACKUP/$BASENAME-$PROFONDEUR

      for (( i=$PROFONDEUR; i>=2; i-- ))
      do
         echo mv $BACKUP/$BASENAME-$(($i-1)) $BACKUP/$BASENAME-$i
         mv $BACKUP/$BASENAME-$(($i-1)) $BACKUP/$BASENAME-$i
      done

      echo copy $BACKUP/$BASENAME-0 $BACKUP/$BASENAME-1
      cp -al $BACKUP/$BASENAME-0 $BACKUP/$BASENAME-1
      touch  $BACKUP/$BASENAME-0
    
      echo
      echo ======= BACKUP USEFULL FILES   ================
      rsync -av --delete  $SOURCE   $BACKUP/$BASENAME-0

     echo ========AVAILABLE SPACE ===============================
      df -h |grep ^/dev
  
      echo ======= TOTAL BACKUP TIME ============================

      

      )
   ) &>$REPORT_PATH/$REPORT_FILE
      
      #### BACKUP JUTE CREATED REPORT FILES  ####
      rsync -a  $REPORT_PATH/$REPORT_FILE     $BACKUP/$BASENAME-0/$REPORT_FILE

      #### SEND THE REPORT BY E-MAIL ####
      mail -s "$SUBJECT" "$EMAIL" < $REPORT_PATH/$REPORT_FILE
else
   echo "Nothing to do for $SUBJECT"
fi


To do

Pour les systemes qui ne sont pas toujours en ligne on peut tester leur présence par un ping comme ceci:


   if ping -c 3 -w 10 128.178.190.119>/dev/null
   then
      echo Ping OK
   else
      echo Ping FAIL
   fi