[HOWTO] fully automated incremental backups with borgbackup

DON'T post new tutorials here! Please use the "Pending Submissions" board so the staff can review them first.
Post Reply
User avatar
bad_brain
Site Owner
Site Owner
Posts: 11575
Joined: 06 Apr 2005, 16:00
16
Location: The zone.
Contact:

[HOWTO] fully automated incremental backups with borgbackup

Post by bad_brain »

always good to have a decent backup strategy, but often it's too much work to set up or expensive software licenses are involved.
I'm using the free and open source BorgBackup, so the only cost (in my example) is for the backup space.

OS used: Debian Buster
backup space is on a storage box from Hetzner
file transfer is done via SSH

Preparing the backup system
often, like in my case too, storage space does not come with shell access, which is a little confusing at the beginning.

log in via SFTP instead, go to the root directory of where you want to store the backups, and create a hidden directory named .ssh

next we need to create the keys so your source system can log in to the storage system and place the backups on there.
so log in to the source system and run ssh-keygen, don't use a password and simply keep the suggested default directory:

Code: Select all

server> ssh-keygen
Generating public/private rsa key pair.
Enter file in which to save the key (/root/.ssh/id_rsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /root/.ssh/id_rsa.
Your public key has been saved in /root/.ssh/id_rsa.pub.
The key fingerprint is:
cb:3c:a0:39:69:39:ec:35:d5:66:f3:c5:92:99:2f:e1 root@server
The key's randomart image is:
+--[ RSA 2048]----+
|                 |
|                 |
|                 |
|         .   =   |
|      . S = * o  |
|   . = = + + =   |
|    X o =   E .  |
|   o + . .   .   |
|    .            |
+-----------------+
when using the default SSH port 22 you'll need to convert the key into RFC4716 format:

Code: Select all

ssh-keygen -e -f .ssh/id_rsa.pub | grep -v "Comment:" > .ssh/id_rsa_rfc.pub
to make it easier and compatible to all ports put both keys into a file named storagebox_authorized_keys:

Code: Select all

cat .ssh/id_rsa.pub >> storagebox_authorized_keys
cat .ssh/id_rsa_rfc.pub >> storagebox_authorized_keys
download the file, upload it to the .ssh directory created on the storage box earlier. if there is no authorized_keys file present yet rename the uploaded file to it, if there is a file present already merge the files. chmod 700 it.

now back to the storage box root directory, create a directory for the actual backups (/backups/server1 in this example).

aaand your storage box is ready to go! next the BorgBackup setup.

Setting up BorgBackup
install it from the repository by apt-get install borgbackup, if your distro does not have it as package check the different install options in the docs.
now initiate the backup repository directory on the storage box via SSH from the source server, when prompted make sure to set a really decent key:

Code: Select all

borg init --encryption=repokey ssh://username@your-storage-system.com.:23/./backups/server1
Backing up
I use this little script which creates backups named by hostname+date, and it also takes care of the pruning so you'll not need a gigantic amount of space over time:

Code: Select all

# Setting this, so the repo does not need to be given on the commandline:
export BORG_REPO=ssh://username@your-storage-system.com.:23/./backups/server1

# See the section "Passphrase notes" for more infos.
export BORG_PASSPHRASE='supersecretpasshere'

# some helpers and error handling:
info() { printf "\n%s %s\n\n" "$( date )" "$*" >&2; }
trap 'echo $( date ) Backup interrupted >&2; exit 2' INT TERM

info "Starting backup"

# Backup the most important directories into an archive named after
# the machine this script is currently running on:

borg create                         \
    --filter AME                    \
    --list                          \
    --stats                         \
    --show-rc                       \
    --compression zlib              \
    ::'{hostname}-{now}'            \
    /path/you/want/to/back/up                        \

backup_exit=$?

info "Pruning repository"

# Use the `prune` subcommand to maintain 7 daily, 4 weekly and 6 monthly
# archives of THIS machine. The '{hostname}-' prefix is very important to
# limit prune's operation to this machine's archives and not apply to
# other machines' archives also:

borg prune                          \
    --list                          \
    --prefix '{hostname}-'          \
    --show-rc                       \
    --keep-daily    7               \
    --keep-weekly   4               \
    --keep-monthly  6               \

prune_exit=$?

# use highest exit code as global exit code
global_exit=$(( backup_exit > prune_exit ? backup_exit : prune_exit ))

if [ ${global_exit} -eq 0 ]; then
    info "Backup and Prune finished successfully"
elif [ ${global_exit} -eq 1 ]; then
    info "Backup and/or Prune finished with warnings"
else
    info "Backup and/or Prune finished with errors"
fi

exit ${global_exit}
place it in /usr/local/bin of the source server (I named it backup.sh in my example), and then run it.
if all was set up properly it'll list the files being backed up and finished with something similar to this:

Code: Select all

------------------------------------------------------------------------------
Archive name: noway-2021-03-23T01:00:02
Archive fingerprint: 8f4ad855a8273e7c9512b0759fc8bfe43f25d6403bf0089d83270c3a3601415a
Time (start): Tue, 2021-03-23 01:00:35
Time (end):   Tue, 2021-03-23 01:02:15
Duration: 1 minutes 40.02 seconds
Number of files: 345829
Utilization of max. archive size: 0%
------------------------------------------------------------------------------
                       Original size      Compressed size    Deduplicated size
This archive:               28.29 GB             22.52 GB             73.97 MB
All archives:              112.76 GB             89.72 GB             19.60 GB

                       Unique chunks         Total chunks
Chunk index:                  183013              1336711
------------------------------------------------------------------------------
terminating with success status, rc 0

Tue 23 Mar 2021 01:02:23 AM CET Pruning repository

Keeping archive: noway-2021-03-23T01:00:02            Tue, 2021-03-23 01:00:35 [8f4ad855a8273e7c9512b0759fc8bfe43f25d6403bf0089d83270c3a3601415a]
Keeping archive: noway-2021-03-22T01:00:02            Mon, 2021-03-22 01:00:31 [a7265b38cd6c83d3f0dc8630ee53ad74d6db4615dd6c9d1a05551809ec6c1e57]
Keeping archive: noway-2021-03-21T01:00:02            Sun, 2021-03-21 01:00:35 [137871932af59a079abab26de2f48378d9ac985194186bbbdac297bed47c7992]
Keeping archive: noway-2021-03-19T21:46:35            Fri, 2021-03-19 21:46:37 [40ce0fced5b0155dea51f4fc51e180260af2c2658072cc4282ada6ec70e90b35]
terminating with success status, rc 0

Tue 23 Mar 2021 01:02:25 AM CET Backup and Prune finished successfully
lastly, when the initial backup worked fine:

Setting up the cronjob
pretty straight forward, I prefer to write the output to a log file instead to /dev/null, just in case:

Code: Select all

00 1 * * * /usr/local/bin/backup.sh > /var/log/borg.log 2>&1



How to use the backups
when you check the backup folder you'll notice it's in an own format and not simply a copy of the files, that's because it's encrypted and also because of the incremental type backup.
to access the backed up files we'll use the super sexy mount option of borgbackup, which allows to mount a backup repository like an external drive on the source system:

Code: Select all

mkdir /mnt/backups
borg mount ssh://username@your-storage-system.com.:23/./backups/server1 /mnt/backups
and voila, all backups in their own directories by date:
Screen Shot 03-24-21 at 12.04 AM.PNG
Screen Shot 03-24-21 at 12.04 AM.PNG (7.61 KiB) Viewed 224 times
Image

Online
User avatar
ayu
Staff
Staff
Posts: 8070
Joined: 27 Aug 2005, 16:00
15
Contact:

Re: [HOWTO] fully automated incremental backups with borgbackup

Post by ayu »

Recently stopped using my old backup solution due to shitty Linux support.
Might try this out :-k .
"The best place to hide a tree, is in a forest"

User avatar
bad_brain
Site Owner
Site Owner
Posts: 11575
Joined: 06 Apr 2005, 16:00
16
Location: The zone.
Contact:

Re: [HOWTO] fully automated incremental backups with borgbackup

Post by bad_brain »

go for it! it's running since almost a week now with 5 servers that are backed up daily, works like a charm. :D
there is even a web iface available named borgweb, but the shell use is so simple that I did not use it.
make sure to read the docs for stuff like the different compression and encryption methods available!
Image

Post Reply