- How-to guides
- Reference guides
Biggish data ¶
Borg itself is great for efficiently de-duplicating data across successive backup archives, even when dealing with very large repositories. But you may find that while borgmatic's default mode of "prune, create, and check" works well on small repositories, it's not so great on larger ones. That's because running the default pruning and consistency checks take a long time on large repositories.
A la carte actions ¶
If you find yourself in this situation, you have some options. First, you can run borgmatic's pruning, creating, or checking actions separately. For instance, the following optional actions are available:
check actions? Try the old-style
--check. Or upgrade borgmatic!)
You can run with only one of these actions provided, or you can mix and match
any number of them in a single borgmatic run. This supports approaches like
skipping certain actions while running others. For instance, this skips
prune and only runs
borgmatic create check
Or, you can make backups with
create on a frequent schedule (e.g. with
borgmatic create called from one cron job), while only running expensive
consistency checks with
check on a much less frequent basis (e.g. with
borgmatic check called from a separate cron job).
Consistency check configuration ¶
Another option is to customize your consistency checks. The default consistency checks run both full-repository checks and per-archive checks within each repository.
But if you find that archive checks are too slow, for example, you can
configure borgmatic to run repository checks only. Configure this in the
consistency section of borgmatic configuration:
Here are the available checks from fastest to slowest:
repository: Checks the consistency of the repository itself.
archives: Checks all of the archives in the repository.
extract: Performs an extraction dry-run of the most recent archive.
data: Verifies the data integrity of all archives contents, decrypting and decompressing all data (implies
See Borg's check documentation for more information.
If that's still too slow, you can disable consistency checks entirely, either for a single repository or for all repositories.
Disabling all consistency checks looks like this:
Or, if you have multiple repositories in your borgmatic configuration file, you can keep running consistency checks, but only against a subset of the repositories:
Finally, you can override your configuration file's consistency checks, and run particular checks via the command-line. For instance:
borgmatic check --only data --only extract
This is useful for running slow consistency checks on an infrequent basis, separate from your regular checks.
Broken pipe with remote repository ¶
When running borgmatic on a large remote repository, you may receive errors like the following, particularly while "borg check" is validating backups for consistency:
Write failed: Broken pipe
borg: Error: Connection closed by remote host
This error can be caused by an ssh timeout, which you can rectify by adding
the following to the
~/.ssh/config file on the client:
This should make the client keep the connection alive while validating backups.
Improve this documentation
Have an idea on how to make this documentation even better? Use our issue tracker to send your feedback!