One of the best pieces of self-hosted software ever to exist.

Edit: This is Immich! for the folks who don’t know.

  • Dave@lemmy.nz
    link
    fedilink
    English
    arrow-up
    0
    ·
    1 month ago

    I do nightly borg backups of much more than 200gb. The idea of incremental backups is you’re only doing the changes, and photos don’t tend to change.

    What challenge did you come across with a 200GB backup?

    • sonofearth@lemmy.worldOP
      link
      fedilink
      English
      arrow-up
      0
      ·
      1 month ago

      I wasn’t bothered by it until you said nightly lol. So basically it takes a heck lot of time for the backup to finish - like almost 6 hrs even with 200MB of changes or even there are no changes. It could be because of integrity check or pruning old ones. I don’t know the exact issue honestly. It could also be because of slow hardrives. It could be my repo size as well.

      I used my local LLM to make the script and tested it with a test container to see if it was working which it was so I just went ahead with it. If you can help here are the last backup logs and my backup script.

      [START] Sunday 26 October 2025 06:00:01 AM IST: Starting Borg backup...
      Creating archive at "[REDACTED_REPO_PATH]::immich-2025-10-26_06-00-01"
      ------------------------------------------------------------------------------
      Repository: [REDACTED_REPO_PATH]
      Archive name: immich-2025-10-26_06-00-01
      Archive fingerprint: [REDACTED_FINGERPRINT_1]
      Time (start): Sun, 2025-10-26 06:00:03
      Time (end):   Sun, 2025-10-26 11:40:22
      Duration: 5 hours 40 minutes 19.06 seconds
      Number of files: 34775
      Utilization of max. archive size: 0%
      ------------------------------------------------------------------------------
                             Original size      Compressed size    Deduplicated size
      This archive:              207.73 GB            205.39 GB            210.94 MB
      All archives:                2.60 TB              2.57 TB            206.75 GB
      
                             Unique chunks         Total chunks
      Chunk index:                  113167              1402143
      ------------------------------------------------------------------------------
      [PRUNE] Sunday 26 October 2025 11:40:23 AM IST: Pruning old backups...
      Keeping archive (rule: daily #1):            immich-2025-10-26_06-00-01           Sun, 2025-10-26 06:00:03 [REDACTED_FINGERPRINT_1]
      Keeping archive (rule: daily #2):            immich-2025-10-19_06-00-02           Sun, 2025-10-19 06:00:03 [REDACTED_FINGERPRINT_2]
      Keeping archive (rule: daily #3):            immich-2025-10-12_06-00-01           Sun, 2025-10-12 06:00:02 [REDACTED_FINGERPRINT_3]
      Keeping archive (rule: daily #4):            immich-2025-10-05_06-00-02           Sun, 2025-10-05 06:00:03 [REDACTED_FINGERPRINT_4]
      Keeping archive (rule: daily #5):            immich-2025-09-28_06-00-02           Sun, 2025-09-28 06:00:03 [REDACTED_FINGERPRINT_5]
      Keeping archive (rule: daily #6):            immich-2025-09-21_06-00-01           Sun, 2025-09-21 06:00:03 [REDACTED_FINGERPRINT_6]
      Keeping archive (rule: daily #7):            immich-2025-09-14_06-00-02           Sun, 2025-09-14 06:00:04 [REDACTED_FINGERPRINT_7]
      Keeping archive (rule: weekly #1):           immich-2025-09-07_06-00-02           Sun, 2025-09-07 06:00:03 [REDACTED_FINGERPRINT_8]
      Keeping archive (rule: weekly #2):           immich-2025-08-31_06-00-02           Sun, 2025-08-31 06:00:03 [REDACTED_FINGERPRINT_9]
      Keeping archive (rule: weekly #3):           immich-2025-08-17_06-00-01           Sun, 2025-08-17 06:00:03 [REDACTED_FINGERPRINT_10]
      Keeping archive (rule: weekly #4):           immich-2025-08-10_06-00-01           Sun, 2025-08-10 06:00:02 [REDACTED_FINGERPRINT_11]
      Pruning archive (1/1):                       immich-2025-08-03_06-00-01           Sun, 2025-08-03 06:00:03 [REDACTED_FINGERPRINT_12]
      Keeping archive (rule: monthly #1):          immich-2025-07-28_10-54-59           Mon, 2025-07-28 10:54:59 [REDACTED_FINGERPRINT_13]
      [CHECK] Sunday 26 October 2025 11:40:25 AM IST: Verifying backup integrity...
      Failed to securely erase old repository config file (hardlinks not supported). Old repokey data, if any, might persist on physical storage.
      Failed to securely erase old repository config file (hardlinks not supported). Old repokey data, if any, might persist on physical storage.
      [DONE] Sunday 26 October 2025 12:09:38 PM IST: Backup completed.
      
      #!/bin/bash
      
      export BORG_REPO="/home/user/immich_backup/borg"
      export BORG_PASSPHRASE="[REDACTED_BORG_PASSPHRASE]"
      SOURCE="/home/user/containers/immich/library"
      LOG="/home/user/borg-backup.log"
      ARCHIVE="immich-$(date +%Y-%m-%d_%H-%M-%S)"
      
      echo "[START] $(date): Starting Borg backup..." >> "$LOG"
      
      # Skip if source is empty or unmounted
      if [ -z "$(ls -A "$SOURCE")" ]; then
          echo "[SKIP] $(date): Source directory is empty or unmounted. Skipping." >> "$LOG"
          exit 1
      fi
      
      borg create --verbose --stats \
          "$BORG_REPO"::"$ARCHIVE" \
          "$SOURCE" >> "$LOG" 2>&1
      
      echo "[PRUNE] $(date): Pruning old backups..." >> "$LOG"
      borg prune -v --list "$BORG_REPO" \
        --keep-daily=7 --keep-weekly=4 --keep-monthly=6 >> "$LOG" 2>&1
      
      # Optional: Run a full repo integrity check
      echo "[CHECK] $(date): Verifying backup integrity..." >> "$LOG"
      borg check "$BORG_REPO" >> "$LOG" 2>&1
      
      echo "[DONE] $(date): Backup completed." >> "$LOG"