Use IONOS S3 as primary object storage #72

Closed
opened 2025-03-14 16:38:03 +01:00 by lukas · 2 comments
Owner

The currently used object storage is Hetzner's own, unfortunately it's way too slow for my usecase and without Seafile this storage is used only by applications storing a lot less than 1TB, meaning Hetzner bills me for atleast 1TB of object storage.

The S3 service provided by IONOS is vastly more performant in terms of TTFB, total time it takes for an upload fully succeed and API response latency in general, vastly speeding up perceived speed of applications that natively use S3. While IONOS is 2€/TB/m more expensive than Hetzner, it's actually even cheaper as I don't need to pay for a whole TB minimum and their 2TB of bandwith is more than enough for me.

I have migrated GitLab and Mattermost to IONOS S3 as of now, with my Szurubooru instances Szuru and Kirei still pending. I will also use this opportunity to migrate Nextcloud to a JuiceFS-based object storage.

The currently used object storage is Hetzner's own, unfortunately it's way too slow for my usecase and without Seafile this storage is used only by applications storing a lot less than 1TB, meaning Hetzner bills me for atleast 1TB of object storage. The S3 service provided by IONOS is vastly more performant in terms of TTFB, total time it takes for an upload fully succeed and API response latency in general, vastly speeding up perceived speed of applications that natively use S3. While IONOS is 2€/TB/m more expensive than Hetzner, it's actually even cheaper as I don't need to pay for a whole TB minimum and their 2TB of bandwith is more than enough for me. I have migrated GitLab and Mattermost to IONOS S3 as of now, with my Szurubooru instances Szuru and Kirei still pending. I will also use this opportunity to migrate Nextcloud to a JuiceFS-based object storage.
lukas added the
Status
Planned
Priority
Medium
Domain
libre.moe
labels 2025-03-14 16:38:03 +01:00
lukas self-assigned this 2025-03-14 16:38:03 +01:00
lukas added this to the Issue Board project 2025-03-14 16:38:03 +01:00
Author
Owner

While B2 would also be better than Hetzner in terms of speed, Mattermost ran into issues as B2 uses a different shashing algorithm than S3 and therefore checksums differ, resulting in failure to read and write successfully to object storage. TO avoid these issues with GitLab and other native applications, and to prevent scattering my setup by using 2 object storage providers as primary storage, I decided to only use IONOS S3 as primary, with B2 being used solely for backing up Hetzner and now IONOS S3.

Backups of S3 native apps will just be made using good old rclone, mirroring the storage to Backblaze and leaving the versioning to Backblaze with 30 days of history.

While B2 would also be better than Hetzner in terms of speed, Mattermost ran into issues as B2 uses a different shashing algorithm than S3 and therefore checksums differ, resulting in failure to read and write successfully to object storage. TO avoid these issues with GitLab and other native applications, and to prevent scattering my setup by using 2 object storage providers as primary storage, I decided to only use IONOS S3 as primary, with B2 being used solely for backing up Hetzner and now IONOS S3. Backups of S3 native apps will just be made using good old rclone, mirroring the storage to Backblaze and leaving the versioning to Backblaze with 30 days of history.
lukas moved this to In Progress in Issue Board on 2025-03-14 16:42:25 +01:00
Author
Owner

Backups are confirmed working, closing.

Backups are confirmed working, closing.
lukas closed this issue 2025-03-15 08:28:20 +01:00
lukas moved this to Review in Issue Board on 2025-03-15 08:28:28 +01:00
Sign in to join this conversation.
No description provided.