Better Butter Archive
A btrfs reflink and delta based revisional temporal backup solution
I call this Better Butter Archive which is a way of efficiently storing backups 'time machine' style. It uses an innovations it btrfs to make a nice KISS principled way of managing backups. As backups appear as directories for their revisions. One can work on them with great *NIX shell power.
This was originally an idea from a developer I refuse to name but I made my own. It is more efficient than other similar solutions and uses a unique feature in btrfs aka the butter filesystem or better filesystem. The unique feature is called a reference link which makes two files share one pool of data. Yet either file can change, and it will only physically store on the disk changes that make the files different.
You might wonder with how I've described this simple system what use I am. With the butter filesystem one has to take care and select 'correct' kernels to update to. It can require recoveries to be performed after crashes. There are security issues where much is done as root with this system.
On the 'temporal archive' there is simply directories for systems being backed up. In these directories in another called current. A backup loaded on the server uses rsync to write and change things in 'current' for a particular system. It then creates a reflinked copy of the current directory and populates a snapshot which is a directory named as the date it was created.
This setup of doing snapshots unlike other methods of archiving file temporally such as in XFS keeps the snapshots. With my system of snapshots one can compare changes between with a simple 'diff' command. One can chroot into a backup of any good Linux distro and start MySQL and get some old tables back. There are endless possibilities, and affords one an easy way to audit and compare changes to a system.
Part of this system is managing the butter filesystem which is far better now than before, but it can crash and lose data. This can be avoided if one sticks to basic features. Another issue is kernel updates and changes breaking things. I employ one of these servers here and have never had a crash. It has great recovery tools to repair such an archive.
I have some nice scripts I can put on the server to delete old files. There are two methods of backup involved one can use. An archive can be downloaded to the server, and an archive can be uploaded from a server. The upload method is good for backing up an SELinux protected system that doesn't let a remote rsync call read certain files. With such systems backups may have to be run manually as sysadm_r to get all important files.
A leading principle of this system is the KISS principled. It uses well managed and peer reviewed Gentoo Linux with my own simple Python scripts that power it. Reference links on the filesystem is a new thing, but I have pioneered in a simple way. My simple solution has strong results and gives users of this offering a powerful tool.
With a Temporal Archive of Butter allows you to unleash UNIX on your old data. One works with these snapshots that appear as simple directories, with simple and familiar commands. It can allow one to roll back a very minor change to a system or very big ones. You will find if you are nervous about loss, just make serious mistakes too often, and that other backup systems have redundant copies of the same stuff - this server offering could be for you.
In future some more simple scripts will make this system more advanced. I want to make it a bit like git with branch and merge features. It will be an extension that works with the scripts I've already done. I also want to make some security improvements as since most the scripts are rsync are running as root, to get whole systems, one could retroactively change previous backups.
In its current state one gets a nice system where one can interactively work with snapshots of their system. There are tools to upload a backup, download a backup, to delete old files. Coming soon will be tools to make git-like branches, merges, and even a commit. It will be much simpler.
When the merge/branch/commit features are implemented this will be more than just for archives. One could use this system to 'stage' a server from a snapshot to a new system. This will turn the tool from a backup system, to that and also a simple as possible way to manage development on Gentoo systems.
This simple product is one my most principled yet. As such I offer no price and would like to know what you are using it for. As such you might not need much support once up and running. This product has a great security record and I use it myself and it has ran stable for years. The scripts will be placed on a Github in complete form.
Effectively this is an open source project, yet I have great experience in managing the nuances of a butter filesystem. I can also get this service running on a Gentoo server that is targeted. You can buy support at reasonable rates with this system, and might not always need it. The system has ran stable for quite a while.