prune

SYNOPSIS:
   duplicacy prune - Prune snapshots by revision, tag, or retention policy

USAGE:
   duplicacy prune [command options]

OPTIONS:
   -id         delete snapshots with the specified id instead of the default one
   -all, -a                 match against all snapshot IDs
   -r  [+]        delete snapshots with the specified revisions
   -t  [+]             delete snapshots with the specified tags
   -keep  [+]          keep 1 snapshot every n days for snapshots older than m days
   -exhaustive              find all unreferenced chunks by scanning the storage
   -exclusive               assume exclusive access to the storage (disable two-step fossil collection)
   -dry-run, -d             show what would have been deleted
   -delete-only             delete fossils previously collected (if deletable) and don't collect fossils
   -collect-only            identify and collect fossils, but don't delete fossils previously collected
   -ignore  [+]         ignore the specified snapshot id when deciding if fossils can be deleted
   -storage   prune snapshots from the specified storage

The prune command implements the two-step fossil collection algorithm. It will first find fossil collection files from previous runs and check if contained fossils are eligible for permanent deletion (the fossil deletion step). Then it will search for snapshots to be deleted, mark unreferenced chunks as fossils (by renaming) and save them in a new fossil collection file stored locally (the fossil collection step).

If a snapshot id is specified, that snapshot id will be used instead of the default one. The -a option will find snapshots with any id. Snapshots to be deleted can be specified by revision numbers, by a tag, by retention policies, or by any combination of them.

The retention policies are specified by the -keep option, which accepts an argument in the form of two numbers n:m, where n indicates the number of days between two consecutive snapshots to keep, and m means that the policy only applies to snapshots at least m day old. If n is zero, any snapshots older than m days will be removed.

Here are a few sample retention policies:

$ duplicacy prune -keep 1:7       # Keep 1 snapshot per day for snapshots older than 7 days
$ duplicacy prune -keep 7:30      # Keep 1 snapshot every 7 days for snapshots older than 30 days
$ duplicacy prune -keep 30:180    # Keep 1 snapshot every 30 days for snapshots older than 180 days
$ duplicacy prune -keep 0:360     # Keep no snapshots older than 360 days

Multiple -keep options must be sorted by their m values in decreasing order. For instance, to combine the above policies into one line, it would become:

$ duplicacy prune -keep 0:360 -keep 30:180 -keep 7:30 -keep 1:7

The -exhaustive option will scan the list of all chunks in the storage, therefore it will find not only unreferenced chunks from deleted snapshots, but also chunks that become unreferenced for other reasons, such as those from an incomplete backup. It will also find any file that does not look like a chunk file. In contrast, a default prune command will only identify chunks referenced by deleted snapshots but not any other snapshots.

The -exclusive option will assume that no other clients are accessing the storage, effectively disabling the two-step fossil collection algorithm. With this option, the prune command will immediately remove unreferenced chunks.

The -dry-run option is used to test what changes the prune command would have done. It is guaranteed not to make any changes on the storage, not even creating the local fossil collection file. The following command checks if the chunk directory is clean (i.e., if there are any unreferenced chunks, temporary files, or anything else):

$ duplicacy prune -d -exclusive -exhaustive    #  Prints out nothing if the chunk directory is clean

The -delete-only option will skip the fossil collection step, while the -collect-only option will skip the fossil deletion step.

For fossils collected in the fossil collection step to be eligible for safe deletion in the fossil deletion step, at least one new snapshot from each snapshot id must be created between two runs of the prune command. However, some repository may not be set up to back up with a regular schedule, and thus literally blocking other repositories from deleting any fossils. Duplicacy by default will ignore repositories that have no new backup in the past 7 days. It also provide an -ignore option that can be used to skip certain repositories when deciding the deletion criteria.

You can use the -storage option to select a different storage other than the default one.

• The -exhaustive option is only needed when there are known unreferenced chunks in the storage, for example, when a backup is interrupted by user and terminated due to an error and the files in the repository change afterwards. It is not recommended to run the prune command regularly with this option without a recent incomplete backup, mainly because if there is an ongoing backup from a different computer, the prune command will mark as fossils all new chunks unloaded by that backup. Although in the fossil deletion step the prune command can correctly identify these chunks are actually referenced and thus turn them back into chunks, the cost of extra API calls can be excessive.

• There are two corner cases that a fossil still needed may be mistakenly deleted. When there is a backup taking more than 7 days that started before the chunk was marked as fossil, then the prune command will think the repository has become inactive which will then be excluded from the criteria for determining safe fossils to be deleted. The other case happens when an initial backup from a newly recreated repository that also started before the chunk was marked as fossil. Since the prune command doesn't know the existence of such a repository at the fossil deletion time, it may think the fossil isn't needed any more by any backup and thus delete it permanently. Therefore, a check command must be used if a backup is an initial backup or takes more than 7 days. Once a backup passes the check command, it is guaranteed that it won't be affected by any future prune operations.