# Offsite Backup Replication + Failure Alerting

This runbook configures:

1. Encrypted local DB backups
2. Replication to S3-compatible object storage (AWS S3, Backblaze B2 S3, Wasabi)
3. Webhook/email alerting on backup or restore-test failure

## 1) Configure secrets file

Create or edit:

`/opt/proxpanel/.backup.env`

Reference template:

`/opt/proxpanel/infra/deploy/.backup.env.example`

## 2) Example provider configs

### AWS S3

```bash
OFFSITE_BACKUP_ENABLED=true
OFFSITE_S3_BUCKET=my-proxpanel-backups
OFFSITE_S3_REGION=us-east-1
OFFSITE_S3_PREFIX=proxpanel/db
OFFSITE_S3_ACCESS_KEY_ID=AKIA...
OFFSITE_S3_SECRET_ACCESS_KEY=...
OFFSITE_S3_ENDPOINT_URL=
```

### Wasabi

```bash
OFFSITE_BACKUP_ENABLED=true
OFFSITE_S3_BUCKET=my-proxpanel-backups
OFFSITE_S3_REGION=us-east-1
OFFSITE_S3_PREFIX=proxpanel/db
OFFSITE_S3_ENDPOINT_URL=https://s3.us-east-1.wasabisys.com
OFFSITE_S3_ACCESS_KEY_ID=...
OFFSITE_S3_SECRET_ACCESS_KEY=...
```

### Backblaze B2 (S3 Compatible)

```bash
OFFSITE_BACKUP_ENABLED=true
OFFSITE_S3_BUCKET=my-proxpanel-backups
OFFSITE_S3_REGION=us-west-002
OFFSITE_S3_PREFIX=proxpanel/db
OFFSITE_S3_ENDPOINT_URL=https://s3.us-west-002.backblazeb2.com
OFFSITE_S3_ACCESS_KEY_ID=...
OFFSITE_S3_SECRET_ACCESS_KEY=...
```

## 3) Configure alerting

Set one or both:

```bash
BACKUP_ALERT_WEBHOOK_URL=https://hooks.example.com/proxpanel-backup
BACKUP_ALERT_EMAIL_WEBHOOK_URL=https://mailer.example.com/send
BACKUP_ALERT_EMAIL_TO=ops@votcloud.com
```

## 4) Apply cron schedule

```bash
sudo bash /opt/proxpanel/infra/deploy/configure-db-backup-cron.sh --run-now
```

## 5) Verification

1. Check local encrypted backup exists in `/opt/proxpanel-backups/daily/<timestamp>/`.
2. Check offsite files:
   - `proxpanel.sql.enc`
   - `proxpanel.sql.enc.sha256`
3. Check logs:
   - `/var/log/proxpanel-db-backup.log`
   - `/var/log/proxpanel-db-restore-test.log`
4. Trigger controlled failure and confirm alert received (webhook/email).