Seafile Backups with restic #65

New Issue

lukas · 2024-12-23T14:42:15+01:00

lukas commented

2024-12-23 14:42:15 +01:00

Currently using Scaleway Glacier for backups of Seafile.
With the recent change to restic for backups (#59) i wanted to also use it for Seafile backups.

Due to the offline-nature I cannot use restic with Scaleway Glacier, which is why I have been looking to change our backup target while staying cost-effective. Storj DCS costs 5,96€/TB (4$/TB + ~2,2$ segment fees per used TB), Backblaze 6,87€/TB (6$/TB + USt.) and GCP Archive 1,38€/TB (1,16€/TB + USt.) so I think the winner is clear. Only downside with GCP Archive-tier is their 1 year minimum retention, but when I configure restic to keep backups for a long time, this should not incur much cost - and the storage is quite cheap at that so I don't feel like keeping deleted data for long is a big issue - especially since restic is also encrypted.

My local HDD + tape backups are enough to plausibly restore all data in case of an error with primary storage, so it's unlikely I'll have to restore from GCP.

Currently using Scaleway Glacier for backups of Seafile. With the recent change to restic for backups (#59) i wanted to also use it for Seafile backups. Due to the offline-nature I cannot use restic with Scaleway Glacier, which is why I have been looking to change our backup target while staying cost-effective. Storj DCS costs 5,96€/TB (4$/TB + ~2,2$ segment fees per used TB), Backblaze 6,87€/TB (6$/TB + USt.) and GCP Archive 1,38€/TB (1,16€/TB + USt.) so I think the winner is clear. Only downside with GCP Archive-tier is their 1 year minimum retention, but when I configure restic to keep backups for a long time, this should not incur much cost - and the storage is quite cheap at that so I don't feel like keeping deleted data for long is a big issue - especially since restic is also encrypted. My local HDD + tape backups are enough to plausibly restore all data in case of an error with primary storage, so it's unlikely I'll have to restore from GCP.

lukas added the

labels 2024-12-23 14:42:15 +01:00

lukas self-assigned this 2024-12-23 14:42:15 +01:00

lukas added this to the Issue Board project 2024-12-23 14:42:15 +01:00

lukas changed title from ~~Seafile Backups~~ to Seafile Backups with restic

2024-12-23 14:42:23 +01:00

lukas commented

2024-12-24 23:52:25 +01:00

Discovered road block: restic in it's current configuration uses too much memory (as our data set consists of a huge amount of small files) but the host system does not have sufficient resources to allow restic such usage.

lukas commented

2024-12-24 23:56:37 +01:00

Since I run my setup off of juicefs, i could literally use any other server, mount the juciefs there and backup it up. Maybe I could find a service which allows me to run a job (like a container image set up for everything) and execute it with enough RAM/CPU/network)

lukas commented

2024-12-25 00:18:57 +01:00

If I used Scaleway for such backups, the costs would likely be extremly low for just running the backup daily and a clean up job every month or so.

If I used Scaleway for such backups, the costs would likely be extremly low for just running the backup daily and a clean up job every month or so. <img width="631" alt="grafik" src="attachments/0e41cdd4-141d-4f51-bdfc-f71e17469153">

grafik.png

81 KiB

~~lukas referenced this issue 2024-12-27 14:53:44 +01:00~~

Seafile v12 #68

lukas commented

2024-12-27 17:00:23 +01:00

Closing for now, this setup would be complicated and left root credentials on another prodivers server. Mounting the fusefs might also proove difficult as I needed to somehow forward access to my redis db.
The local backups are versioned either way, so I could just make a new version there.

Closing for now, this setup would be complicated and left root credentials on another prodivers server. Mounting the fusefs might also proove difficult as I needed to somehow forward access to my redis db. The local backups are versioned either way, so I could just make a new version there.

lukas closed this issue

2024-12-27 17:00:23 +01:00

lukas added

and removed

labels 2024-12-27 17:00:36 +01:00

lukas added the

Service

Seafile