1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
|
+++
date = "2017-02-23T11:45:05+01:00"
description = ""
title = "Offsite Backup with Bacula"
categories = [ "Backup", "Bacula" ]
thumbnail = "/images/blog/bacula-offsite-backup/hd.jpg"
+++
## Intro
Into the category of "What can I possibly do useful with a Raspberry PI?"
falls the idea of origanizing a backup server using the
[Bacula](http://blog.bacula.org/) backup software.
## Details
I use an old USB disk and a USB hub as external storage for the
Bacula volume data and for the catalog (stored in PostgreSQL).
For full-filling the off-site requirement the jobs are copied with
a 'Migration Job' to an external FTP server. Encryption is simply
done with the 'openssl' command line tool.
So, we go with two pool definitons:
```
# File Pool definition
Pool {
Name = File
Pool Type = Backup
Recycle = yes
AutoPrune = yes
Volume Retention = 1 week
Maximum Volume Bytes = 512M
Volume Use Duration = 23 hours
Use Volume Once = yes
Maximum Volumes = 9999
LabelFormat = "Vol"
Storage = File
Next Pool = External
Action On Purge = Truncate
}
# External Pool on FTP server
Pool {
Name = External
Pool Type = Backup
Recycle = yes
AutoPrune = yes
Volume Retention = 2 months
Maximum Volume Bytes = 512M
Use Volume Once = yes
Maximum Volumes = 9999
LabelFormat = "ExternalStorage"
Storage = External
Action On Purge = Truncate
}
```
The storage daemon stores the files in two locations:
```
Device {
Name = File
Media Type = File
Archive Device = /data/work/bacula/files
LabelMedia = yes
Random Access = yes
Removable Media = no
Random Access = yes
AlwaysOpen = no
}
Device {
Name = External
Media Type = File
Archive Device = /data/work/bacula/spool
LabelMedia = yes
Random Access = yes
Removable Media = no
Random Access = yes
AlwaysOpen = no
}
```
All jobs write to the `File` Pool. At the end of every day a migration
jobs picks up those volumes and executes a script dealing with encryption
and FTP transfer:
```
Job {
Name = "MigrationJob"
Type = Migrate
Level = Full
Client = myserver-fd
Schedule = "MigrationAfterBackup"
FileSet = "Full Set"
Messages = Standard
Pool = File
Maximum Concurrent Jobs = 1
Selection Type = Volume
Selection Pattern = "Vol.*"
RunScript {
Command = "/etc/bacula/scripts/ExternationMigration.sh"
RunsWhen = After
RunsOnClient = no
RunsOnSuccess = yes
RunsOnFailure = no
}
}
```
The script itself looks as follows:
```
#!/bin/sh
case "$0" in
/*)
base=`dirname $0`
;;
*)
base=`pwd`/`dirname $0`
;;
esac
. $base/ftp.inc
FTP_SERVER=backupserver.somewhere.net
FTP_USER=useruser
FTP_PASS=passpass
BACULADIR=/data/work/bacula
SPOOLDIR=${BACULADIR}/spool
STATEDIR=${BACULADIR}/External
STATUSFILE_BACKUP=${STATEDIR}/state.backup
STATUSFILE=${STATEDIR}/state
if test "$(ls -A $SPOOLDIR 2>/dev/null)" != ""; then
for file in `ls ${SPOOLDIR}/ExternalStorage* | grep -v .enc`; do
if test ! -f $file.enc; then
cat $file | \
openssl enc -aes-256-cbc -salt \
-pass file:/etc/bacula/private/pwd > \
$file.enc
if test $? -ne 0; then
echo "--- ERROR: Error while encrypting volume '$file' (Check manually!)" 1>&2
exit 1
fi
fi
done
global_lock
for file in ${SPOOLDIR}/ExternalStorage*.enc; do
upload_file $file
rm -f $file
origfile=`echo $file | sed 's/\.enc$//g'`
rm -f $origfile
done
global_unlock
else
echo "--- WARN: Nothing found to transfer? Probably ok.." 1>&2
fi
exit 0
```
The whole ftp transfer logic script is left out (too long), but basically
it deals with setting the password in a `.netrc` file, writes FTP job files
and executes `ftp`. It also performs some checking after transfers to handle
transfer errors or out-of-disk-space situations.
## Conclusion
This backup works reliably and fast, even a Raspberry B+ is fast
enough to deal with the encryption of some gigabytes of data per day.
The only drawback is restoring the data: you have to transfer the files
back manually via FTP, then call 'openssl' to decrypt them and leave them
in the `/data/work/bacula/spool` directory and wait for the bacula-sd
daemon to pick them up.
This backup works now for 2 years reliably, before it run on different
hardware, but also 3-4 years.
## Theory
Let's see if we follow good practice:
* *have a backup*: **tick**
* *have a restore*: a backup is only a backup, when a restore has been done and
the data is the same after checking for differences. **tick**
* *follow the [3-2-1 rule](http://dpbestflow.org/node/262)*:
3 backups, 2 different types of media, 1 remote location (offline and/or offsite):
As I am also using several home directories, one location on the NAS and
one location off-site, the '3' and the '1' part are fullfilled. What about
the '2'? Different media is maybe no longer valid nowadays. For things
like git repositories I follow the somewhat modified '2' format version
of keeping two different backup formats (in this case a raw workspace
with a local .git directory and an export from the server). **tick**
* *have a fallback*: actually occasionally I'm not trusting my current
strategy and I have a manual backup in a tarfile onto a CD-ROM.
Just in case. :-) **tick**
|
