Skip to main content

Migrating buckets from other providers

As the centron Object Storage is S3-compatible, an existing bucket from AWS S3 or another S3-compatible provider can be copied directly. This guide uses rclone (recommended) and, as an alternative, demonstrates the MinIO Client.

Preparation​

  1. Create a destination bucket in centron – see Create bucket . Bucket names are not transferable; choose a new name if necessary.
  2. Have the access details for both sides ready (source and destination).
  3. Check data volume: Outgoing traffic may incur charges with the source provider.
  4. Feature mapping: ACLs, bucket policies, object metadata and storage classes are only partially carried over, depending on the tool. Version histories are not migrated – only the current state.

Migration using rclone​

Configuring remotes​

~/.config/rclone/rclone.conf:

[quelle]
type = s3
provider = AWS                # bzw. "Other" für andere Anbieter
access_key_id = QUELLE_ACCESS_KEY
secret_access_key = QUELLE_SECRET_KEY
region = eu-central-1         # Region des Quellanbieters
# endpoint = https://...      # bei Nicht-AWS-Anbietern setzen

[centron]
type = s3
provider = Other
access_key_id = CENTRON_ACCESS_KEY
secret_access_key = CENTRON_SECRET_KEY
region = 
endpoint = https://s3.internet1.de

Test run and copy​

# Trockenlauf: zeigt, was kopiert würde
rclone copy quelle:alter-bucket centron:neuer-bucket --dry-run -P

# Eigentliche Migration (parallelisiert)
rclone copy quelle:alter-bucket centron:neuer-bucket \
  -P --transfers 16 --checkers 32

Verification​

# Vergleicht Größe und Hashes beider Seiten
rclone check quelle:alter-bucket centron:neuer-bucket

# Objektanzahl und Gesamtgröße
rclone size quelle:alter-bucket
rclone size centron:neuer-bucket

Transferring latecomers​

If data continued to be written to the source during the migration, carry out a final synchronisation (ideally after a write pause):

rclone sync quelle:alter-bucket centron:neuer-bucket -P
Warnung

rclone sync deletes objects in the target that do not (or no longer) exist in the source. Use rclone copy for purely additive runs.

Alternative: MinIO Client (mc)​

mc alias set quelle https://s3.eu-central-1.amazonaws.com QUELLE_ACCESS_KEY QUELLE_SECRET_KEY
mc alias set centron https://s3.internet1.de CENTRON_ACCESS_KEY CENTRON_SECRET_KEY

mc mirror quelle/alter-bucket centron/neuer-bucket

Follow-up work​

  • Migrate applications and backup jobs to the new endpoint/bucket
  • Reconfigure public shares, CORS and lifecycle rules in the destination
  • Once verification is complete: Clean up the source to avoid duplicate costs

Tips for large-scale migrations​

  • Carry out migration from a centron cloud server – short paths to the destination endpoint, no local bandwidth restrictions
  • Increase --checkers when dealing with millions of small objects; keep --transfers at a moderate level when dealing with a few large objects
  • rclone copy is idempotent: simply restart any interrupted runs
Last updated on