added blog post 'bacula backup'

2 files changed, 205 insertions, 0 deletions

diff --git a/content/blog/bacula-offsite-backup.md b/content/blog/bacula-offsite-backup.md
new file mode 100644
index 0000000..9f9ca31
--- /dev/null
+++ b/content/blog/bacula-offsite-backup.md
@@ -0,0 +1,205 @@
++++
+date = "2017-02-23T11:45:05+01:00"
+description = ""
+title = "Offsite Backup with Bacula"
+categories = [ "Backup", "Bacula" ]
+thumbnail = "/images/blog/bacula-offsite-backup/hd.jpg"
++++
+
+## Intro
+
+Into the category of "What can I possibly do useful with a Raspberry PI?"
+falls the idea of origanizing a backup server using the
+[Bacula](http://blog.bacula.org/) backup software.
+
+## Details 
+I use an old USB disk and a USB hub as external storage for the
+Bacula volume data and for the catalog (stored in PostgreSQL).
+
+For full-filling the off-site requirement the jobs are copied with
+a 'Migration Job' to an external FTP server. Encryption is simply
+done with the 'openssl' command line tool.
+
+So, we go with two pool definitons:
+
+```
+# File Pool definition
+Pool {
+  Name = File
+  Pool Type = Backup
+  Recycle = yes
+  AutoPrune = yes
+  Volume Retention = 1 week
+  Maximum Volume Bytes = 512M
+  Volume Use Duration = 23 hours
+  Use Volume Once = yes
+  Maximum Volumes = 9999
+  LabelFormat = "Vol"
+  Storage = File
+  Next Pool = External
+  Action On Purge = Truncate
+}
+
+# External Pool on FTP server
+Pool {
+  Name = External
+  Pool Type = Backup
+  Recycle = yes
+  AutoPrune = yes
+  Volume Retention = 2 months
+  Maximum Volume Bytes = 512M
+  Use Volume Once = yes
+  Maximum Volumes = 9999  
+  LabelFormat = "ExternalStorage"
+  Storage = External 
+  Action On Purge = Truncate
+}
+```
+
+The storage daemon stores the files in two locations:
+
+```
+Device {
+  Name = File
+  Media Type = File
+  Archive Device = /data/work/bacula/files
+  LabelMedia = yes
+  Random Access = yes
+  Removable Media = no
+  Random Access = yes
+  AlwaysOpen = no
+}
+
+Device {
+  Name = External
+  Media Type = File
+  Archive Device = /data/work/bacula/spool
+  LabelMedia = yes
+  Random Access = yes
+  Removable Media = no
+  Random Access = yes
+  AlwaysOpen = no
+}
+
+```
+
+All jobs write to the `File` Pool. At the end of every day a migration
+jobs picks up those volumes and executes a script dealing with encryption
+and FTP transfer:
+
+```
+Job {
+  Name = "MigrationJob"
+  Type = Migrate
+  Level = Full
+  Client = myserver-fd
+  Schedule = "MigrationAfterBackup"
+  FileSet = "Full Set"
+  Messages = Standard
+  Pool = File
+  Maximum Concurrent Jobs = 1
+  Selection Type = Volume
+  Selection Pattern = "Vol.*"
+  RunScript {
+    Command = "/etc/bacula/scripts/ExternationMigration.sh"
+    RunsWhen = After
+    RunsOnClient = no
+    RunsOnSuccess = yes
+    RunsOnFailure = no
+  }
+}
+```
+
+The script itself looks as follows:
+
+```
+#!/bin/sh
+
+case "$0" in
+	/*)
+		base=`dirname $0`
+		;;
+
+	*)
+		base=`pwd`/`dirname $0`
+		;;
+esac
+
+. $base/ftp.inc
+
+FTP_SERVER=backupserver.somewhere.net
+FTP_USER=useruser
+FTP_PASS=passpass
+BACULADIR=/data/work/bacula
+SPOOLDIR=${BACULADIR}/spool
+STATEDIR=${BACULADIR}/External
+STATUSFILE_BACKUP=${STATEDIR}/state.backup
+STATUSFILE=${STATEDIR}/state
+
+if test "$(ls -A $SPOOLDIR 2>/dev/null)" != ""; then
+	for file in `ls ${SPOOLDIR}/ExternalStorage* | grep -v .enc`; do
+		if test ! -f $file.enc; then
+			cat $file | \
+				openssl enc -aes-256-cbc -salt \
+				-pass file:/etc/bacula/private/pwd > \
+				$file.enc
+			if test $? -ne 0; then
+				echo "--- ERROR: Error while encrypting volume '$file' (Check manually!)" 1>&2
+				exit 1
+			fi
+		fi
+	done
+	
+	global_lock
+
+	for file in  ${SPOOLDIR}/ExternalStorage*.enc; do
+		upload_file $file
+		rm -f $file
+		origfile=`echo $file | sed 's/\.enc$//g'`
+		rm -f $origfile
+	done
+	
+	global_unlock
+	
+else
+	echo "--- WARN: Nothing found to transfer? Probably ok.." 1>&2
+fi
+
+exit 0
+```
+
+The whole ftp transfer logic script is left out (too long), but basically
+it deals with setting the password in a `.netrc` file, write FTP jobs files
+and executes `ftp`. I also do some checking after transfers to handle
+transfer errors or out-of-disk-space situations.  
+
+## Conclusion
+
+This backup works reliably and fast enough, even a Raspberry B+ is fast
+enough to deal with the encryption of some gigabytes of data per day.
+The only drawback is restoring the data: you have to transfter the files
+back manually via FTP, then call 'openssl' to decrypt them and leave them
+in the `/data/work/bacula/spool` directory and wait for the bacula-sd
+daemon to pick them up.
+
+This backup works now for 2 years reliably, before it run on different
+hardware, but also 3-4 years.
+
+## Theory
+
+Let's see if we follow good practice:
+
+* *have a backup*: **tick**
+* *have a restore*: a backup is only a backup, when a restore has been done and
+  the data is the same after checking for differences. **tick**
+* *follow the [3-2-1 rule](http://dpbestflow.org/node/262)*:
+  3 backups, 2 different types of media, 1 remote location (offline and/or offsite):
+  As I am also using several home directories, one location on the NAS and
+  one location off-site, the '3' and the '1' part are fullfilled. What about
+  the '2'? Different media is maybe no longer valid nowadays. For things
+  like git repositories I follow the somewhat modified '2' format version
+  of keeping two different backup formats (in this case a raw workspace
+  with a local .git directory and an export from the server). **tick**
+* *have a fallback*: actually occasionally I'm not trusting my current
+  strategy and I have a manual backup in a tarfile onto a CD-ROM.
+  Just in case. :-) **tick**  
diff --git a/static/images/blog/bacula-offsite-backup/hd.jpg b/static/images/blog/bacula-offsite-backup/hd.jpg
new file mode 100644
index 0000000..c3cb97d
--- /dev/null
+++ b/static/images/blog/bacula-offsite-backup/hd.jpg