Encrypted remote backups with SSHFS and LUKS

First written 2016-03-03; Last updated 2021-07-25

back

I like to keep all my backups encrypted. For local backups it’s easy with LUKS, but for remote backups this is problematic as LUKS only works on the local system and I’d have to trust the remote host which is unacceptable.

There are many backup tools available for remote backups but they are quite complicated and sometimes use dubious cryptography or implementations. I prefer simple backup solutions. The following setup uses SSHFS combined with LUKS for encrypted remote backups.

Note
 This setup does not prevent the local computer from corrupting stored backups (e.g. when the local computer is compromised). See Append-only backups with restic and rclone to protect the backups; but with higher complexity and additional software on the remote.

(An alternative to SSHFS is NBD (network block device) over VPN, e.g. with NBD and Wireguard or OpenVPN.)

All commands must be run as root, see below for possible issues when running with sudo.

Note
 This setup requires a stable internet connection. If the host is disconnected during the backup then everything will hang and you’ll have to kill the sshfs process and then unmount the backup directory manually.

Setup

Mount the target directory from the remote host with SSHFS and create a sparse file for the backup (it can be resized later at any time).

sshfs backup-host:/path/to/backup/directory /mnt/backup-host
truncate -s 50G /mnt/backup-host/backup-file.luks

Now encrypt it as usual with LUKS by using cryptsetup, add any options you like, e.g. key files.

cryptsetup luksFormat backup-file.luks

Then we need to create a filesystem on the encrypted partition (lazy initialization is disabled to prevent performance issues until the initialization has finished in the background; instead wait once during mkfs).

cryptsetup luksOpen /mnt/backup-host/backup-file.luks backup-partition
mkfs.ext4 -m0 -E lazy_itable_init=0,lazy_journal_init=0 /dev/mapper/backup-partition

That concludes the setup.

Backup

Mount the remote backup (here using a key file).

sshfs backup-host:/path/to/backup/directory /mnt/backup-host
cryptsetup luksOpen /mnt/backup-host/backup-file.luks backup-partition --key-file /root/luks/backup.key
mount /dev/mapper/backup-partition /mnt/remote-backup

lsblk should now look sometimes like this.

...
loop0                     7:0    0   100G  0 loop
└─backup-partition      254:4    0   100G  0 crypt /mnt/remote-backup

Now perform the backup as usual like any other “local” backup, for example with rsync.

After everything is done unmount the file system:

umount /mnt/remote-backup
cryptsetup close backup-partition
fusermount -u /mnt/backup-host

Resize

To increase the backup image use truncate and resize2fs.

truncate -s 200G /mnt/backup-host/backup-file.luks

Decrypt and mount the partition as described above. Then resize it.

resize2fs /dev/mapper/backup-partition

TRIM/Discard

One remaining issue with this setup is that the sparse file on the remote host doesn’t shrink when files are deleted. SSHFS doesn’t support TRIM (limitation of the SFTP protocol) but nbd-server supports it.

The following setup explains how to use TRIM over LUKS and NBD.

Install NBD:

apt-get install nbd-server nbd-client

nbd-server's configuration isn’t straight-forward (e.g. you can’t specify the trim option globally), so here’s a minimal configuration file:

[generic]
// ...
[default] // can be any name
exportname = /path/to/your/file
trim = true

Make sure to restrict access to NBD via firewall to prevent others from writing the (encrypted) file.

Then run the NBD server on the remote host (-d runs it in the foreground):

nbd-server -d -C /path/to/config

And connect from the local host (default port is 10809):

nbd-client -N default -n $host $port /dev/nbd0

Then mount it as usual, however note the --allow-discards option which is required for TRIM:

cryptsetup luksOpen --allow-discards /dev/nbd0 backup-partition
mount /dev/mapper/backup-partition /mnt/remote-backup

Now you can finally TRIM deleted space:

fstrim -v /mnt/remote-backup

You don’t have to use NBD all the time and can use SSHFS for regular backups and only run TRIM once in a while; all files deleted in the meantime will be TRIMed.

Possible issues

If you’re starting sshfs as non-root user then running cryptsetup will fail as FUSE prevents root from entering the mounted file system (this is designed to prevent attacks which confuse or hang root processes). Either run sshfs as root or mount with sshfs -o allow_root (this requires enabling user_allow_other in /etc/fuse.conf which might have security implications).

back

Last updated 2021-07-25

Impressum Datenschutzerklärung