Repository-to-repository replication / copy

[Shingedo]

Repository-to-repository replication / copy

Would it be possible to add a way to copy/synchronize an existing Vykar repository to another repository, similar to Duplicacy's copy command?

A motivating use case would be:

Files → Vykar repository on NAS → periodically replicate → Vykar repository on USB/off-site storage

The important aspect is that the second repository is not available all the time. The NAS repository is always available and receives regular backups, while the USB/off-site repository might only be connected once a week or once a month.

Configuring both as regular Vykar backup repositories therefore isn't quite the solution: when the second repository is offline, it cannot receive the new snapshots. When it becomes available again, I'd want to bring it up to date by copying the existing repository history, rather than running a new backup from the original files. Otherwise the two repositories would naturally contain different amounts of backup history.

Ideally, the operation would efficiently transfer the existing snapshots and only the data missing from the destination, and could be run again whenever the second location becomes available.

One aspect that seems particularly interesting is that Vykar repositories currently have their own encryption keys. For this use case, I would imagine something more like a repository clone: the destination could share the source repository's encryption identity/key, while still being a separate repository whose snapshots could potentially diverge over time — for example, allowing snapshots to be pruned independently. Whether this should be a one-way replication/copy or eventually support some form of synchronization could be left open for discussion.

I found Discussion #144, which is related, but that discussion is about creating a 1:1 mirror of the source files without snapshots. I'm interested in the somewhat different use case of replicating an existing Vykar repository, including its snapshot history, to another repository.

Is this something that could fit into Vykar's design, or has repository-to-repository replication already been considered?

[m3nu]

To achieve this you can add 2 repos to a Vykar config file and back up to both.

If you need to sync a raw repo instead you can use something like rclone or rsync.

[Shingedo]

As i had written, syncing to two repos while one of them is not available all the time would create less backup data, so that is basically a non-starter. (Another thing that is a bit of a pain for multiple repos is that they have their own keys, so you need to export multiple keys and store them somewhere. Being able to give multiple repos the same key somehow would be nice to have, but that is a separate discussion)

Something like rclone could work to a certain degree, but that would only solve a 1 to 1 copy, not as i had written separated purging. The other thing is that rclone needs to be then carfully synchronized with the execution of vykar.
And rclone isn't really well supported on non *nix systems as far as a know, so on windows you would have to look at something else again.

[goebbe]

@Shingedo Rclone is also available for Windows: https://rclone.org/downloads/
(just in case somebody uses a Windows sever for backup storage)