no way to compare when less than two revisions

Differences

This shows you the differences between two versions of the page.

@@ Line 1: / Line 1: @@
+**CAVEAT : WORK IN PROGRESS **
+====== rsnapshot zfs helper ======
+This is a plugin for rsnapshot (and of course rsnapshot-backup) to be used with zfs as storage backend.
+===== general idea =====
+The main objective why you may consider using zfs might be:
+rsnapshot begins its job with creating a directory filled with hard links as "copy" of yesterday's backup,
+which subseqently is updated with rsync. This '''cp -al daily.1 daily.0''' takes a significant amount of time,
+around 10-20% of the total time the backup takes. Worse, it creates a lot of disk activity, putting stress
+on hard drives, blocking concurrent backups and creating high loads on the backup system. To some extend
+the deletion of expired backups will add to this overhead.
+As zfs uses copy-on-write there is no need to somehow emulate this with '''cp -al ...'''.
+===== example for rsnapshot ======
+[[project:rsnapshot-zfs-helper:rsnapshot-example|Example for using rsnapshot-zfs-helper with rsnapshot]]
+===== example for rsnapshot-backup ======
+[[project:rsnapshot-zfs-helper:rsnapshot-backup-example|Example for using rsnapshot-zfs-helper with rsnapshot-backup]]
+==== Example for creating a zpool: ====
+<code>
+##
+## setup zfs (optional)
+##
+# 'bpo' kernel won't do the trick
+apt-get install linux-image-amd64
+apt-get clean
+reboot
+- add "contrib" to the apt-sources
+apt-get update
+apt-get install zfs-dkms zfsutils-linux
+apt-get clean
+/sbin/modprobe zfs
+## create partition for zfs:
+Device     Boot   Start       End   Sectors  Size Id Type
+...
+/dev/sda3       4194304 209715199 205520896   98G bf Solaris
+## create zpool without mount point
+zpool create zp-bak /dev/sda3 -m none
+# create prereq. datasets
+zfs create zp-bak/backup                        -o compression=lz4 -o snapdir=visible -o xattr=sa -o acltype=posixacl -o mountpoint=/backup
+zfs create zp-bak/backup/hosts                  -o compression=lz4 -o snapdir=visible -o xattr=sa -o acltype=posixacl -o mountpoint=/backup/hosts
+df
+->
+   Filesystem     1K-blocks    Used Available Use% Mounted on
+   ...
+   zp-bak/backup        99041152     128  99041024   1% /backup
+   zp-bak/backup/hosts  99041152     128  99041024   1% /backup/hosts
+</code>
+Create a zfs dataset for a single backup:
+<code>
+zfs create <ZPOOL>/backup/hosts/<BACKUP-CLIENT> -o compression=lz4 -o snapdir=visible -o xattr=sa -o acltype=posixacl -o mountpoint=/backup/hosts/<BACKUP-CLIENT>/daily.0
+# e.g.
+zfs create zp-bak/backup/hosts/srv42 -o compression=lz4 -o snapdir=visible -o xattr=sa -o acltype=posixacl -o mountpoint=/backup/hosts/srv42/daily.0
+</code>
+Create cron jobs for backup promotion to higher rotation levels
+<code>
+23 * * * root /usr/sbin/rsnapshot-zfs-helper /etc/rsnapshot-backup.d/enabled/conf.<BACKUP-CLIENT> daily
+23 * * 5 root /usr/sbin/rsnapshot-zfs-helper /etc/rsnapshot-backup.d/enabled/conf.<BACKUP-CLIENT> weekly
+23 1 * * root /usr/sbin/rsnapshot-zfs-helper /etc/rsnapshot-backup.d/enabled/conf.<BACKUP-CLIENT> monthly
+</code>
+===== how it works =====
+  * '''/usr/sbin/rsnapshot-zfs-helper ... preexec''' removes expired backups (= destroy zfs snapshots)
+  * the removal of expired backups using ''rm'' will never occur because there is only a '''daily.0''' as far as '''rsnapshot''' is concerned
+  * for the same reason, there will be no renaming or rotation of previous backups
+  * '''rsync''' updates '''daily.0''' as usual
+  * '''/usr/sbin/rsnapshot-zfs-helper ... postexec''' creates a zfs snapshot using the name extension '''@daily.<ISO8601-DATE-UTC>'''
+===== FAQ =====
+  * Why does '''rsnapshot-zfs-helper''' use timestamps instead of enumeration when naming the snapshots? **A:** Because I like it better that way: you see what you get. No renaming required. No moving targets.
+  * Why the weird ISO date format? **A:** because it's an international standard. Lots of code is available to handle it. It's human readable. It scales well. No built in misinterpretation.
+  * But why then use UTC? **A:** I would have preferred to use ISO-8601 with timezone notation. But zfs does not accept '+' as a character in snapshot names. So I used the format closest by - the UTC timestamp.
+  * What's the deal with '''cmd_cp /bin/true'''? **A:** Think of it as NOOP.
+  * Why is the .zfs subdir hidden ('''snapdir=hidden''')? **A:** Because rsync would traverse it and all the subsequent snapshots resulting in really bad performance.