All CronJobs running across the k3s cluster, organized by category. Schedules are UTC.
| Job | Namespace | Schedule | Destination |
|---|---|---|---|
etcd-backup |
kube-system |
0 2 * * * (2:00 AM) |
s3://k3s-homelab-backups-855878721457/etcd-snapshots/ |
Retention: 7 snapshots. Backup alert: EtcdBackupJobFailed in PrometheusRules.
Note: k3s also runs a built-in etcd snapshot every 6h via
--etcd-snapshot-schedule-cron. The CronJob above is the primary path pushing to S3.
| Job | Namespace | Schedule | S3 Path |
|---|---|---|---|
postgres-backup |
cardboard |
0 3 * * * (3:00 AM) |
k3s-homelab-backups-855878721457/postgres-backups/cardboard/ |
openclaw-memory-db-backup |
open-webui |
0 3 * * * (3:00 AM) |
zolty-homelab-backups/openclaw-memory/ |
postgres-backup |
ham |
15 3 * * * (3:15 AM) |
k3s-homelab-backups-855878721457/postgres-backups/ham/ |
postgres-backup |
trade-bot |
15 3 * * * (3:15 AM) |
k3s-homelab-backups-855878721457/postgres-backups/trade-bot/ |
postgres-backup |
aja-recipes |
20 3 * * * (3:20 AM) |
k3s-homelab-backups-855878721457/postgres-backups/aja-recipes/ |
postgres-backup |
dnd |
25 3 * * * (3:25 AM) |
k3s-homelab-backups-855878721457/postgres-backups/dnd/ |
postgres-backup |
wiki |
30 3 * * * (3:30 AM) |
k3s-homelab-backups-855878721457/postgres-backups/wiki/ |
jellyfin-pg-backup |
media |
30 3 * * * (3:30 AM) |
k3s-homelab-backups-855878721457/postgres-backups/jellyfin/ |
postgres-backup |
digital-signage |
35 3 * * * (3:35 AM) |
k3s-homelab-backups-855878721457/postgres-backups/digital-signage/ |
postgres-backup |
openclaw-ops |
40 3 * * * (3:40 AM) |
k3s-homelab-backups-855878721457/postgres-backups/openclaw-ops/ |
postgres-backup |
openclaw-personal |
45 3 * * * (3:45 AM) |
k3s-homelab-backups-855878721457/postgres-backups/openclaw-personal/ |
postgres-backup |
media-profiler |
50 3 * * * (3:50 AM) |
k3s-homelab-backups-855878721457/postgres-backups/media-profiler/ |
media-controller-postgres-backup |
media |
0 4 * * * (4:00 AM) |
k3s-homelab-backups-855878721457/postgres-backups/media-controller/ |
postgres-backup |
authentik |
0 4 * * * (4:00 AM) |
k3s-homelab-backups-855878721457/postgres-backups/authentik/ |
postgres-backup |
polymarket-lab |
5 4 * * * (4:05 AM) |
k3s-homelab-backups-855878721457/postgres-backups/polymarket-lab/ |
All pg_dump jobs gzip the output and upload via the k3s-backups IAM user. Retention: 7 days.
All 15 PostgreSQL/TimescaleDB databases have automated daily backup CronJobs. Backup window runs 3:00–4:05 AM UTC.
openclaw-memory-db-backupdeploys toopen-webuinamespace and uses a separate S3 bucket (zolty-homelab-backups) — different from the main backup bucket.
TimescaleDB note: The
polymarket-labbackup uses standardpg_dump. It is compatible but restores hypertables as regular tables. Future improvement: migrate totimescaledb-backupfor full chunk-level recovery.
| Job | Schedule | Task | Destination |
|---|---|---|---|
daily-snapshot RecurringJob |
0 4 * * * (4:00 AM) |
snapshot | Local Longhorn snapshots (retain: 7) |
daily-s3-backup RecurringJob |
0 5 * * * (5:00 AM) |
backup | s3://k3s-homelab-backups-855878721457/longhorn/ (retain: 7) |
Concurrency: 2 (snapshot), 1 (backup). Both jobs target all volumes in the default group. Managed by Longhorn RecurringJob CRD (not a K8s CronJob).
| Schedule Name | Namespace | Cron | TTL |
|---|---|---|---|
k3s-daily-backup |
velero |
0 2 * * * (2:00 AM) |
720h (30 days) |
k3s-weekly-backup |
velero |
0 3 * * 0 (Sundays 3:00 AM) |
8640h (~360 days) |
k3s-monthly-backup |
velero |
0 4 1 * * (1st of month 4:00 AM) |
8760h (365 days) |
| velero-cleanup | velero | 0 5 * * * (5:00 AM daily) | Enforce backup retention policies, prune expired backups |
Velero backs up Kubernetes object state (Deployments, ConfigMaps, Secrets, etc.). It does NOT back up PVC data -- combine with PostgreSQL pg_dump for full recovery.
| Job | Namespace | Schedule | Purpose |
|---|---|---|---|
plex-library-scan |
media |
0 */6 * * * (every 6h) |
Triggers Plex library refresh for sections 1, 2, 4 via internal API |
rclone-sync |
media |
0 */4 * * * (every 4h) |
SFTP sync from RapidSeedbox (45.128.27.65) to NAS /volume1/media/downloads/ |
Job: plex-library-scan
Image: curlimages/curl:8.12.1
Sections scanned: 1, 2, 4 (Movies, TV Shows, Music)
API: POST /library/sections/{id}/refresh?X-Plex-Token=<token>
Secret: plex-token (key: token)
Timeout: 5 minutes (activeDeadlineSeconds: 300)
Note: Section IDs 1, 2, 4 are set at cluster bootstrap. If libraries are deleted/recreated in Plex, IDs may shift -- verify via Plex web UI Settings -> Libraries -> each library's numeric ID in the URL.
Job: rclone-sync
Image: rclone/rclone:1.68
Source: sftp://svc-rclone@45.128.27.65:2222/home/svc-rclone/downloads/
Dest: /mnt/media/downloads/ (NAS NFS RWX mount)
Flag: --transfers=4 --checkers=8 --delete-empty-src-dirs
Secret: rclone-config (contains SFTP key and rclone.conf)
| Job | Namespace | Schedule | Purpose |
|---|---|---|---|
cardboard-scraper |
cardboard |
0 23 * * * (11:00 PM UTC / 6:00 PM ET daily) |
Scrape TCGPlayer item prices for tracked cards |
cardboard-set-scraper |
cardboard |
0 2 * * 1 (Mondays 2:00 AM UTC) |
Scrape sealed product prices (booster boxes, ETBs, bundles) |
cardboard-external-scraper |
cardboard |
0 3 */3 * * (every 3 days 3:00 AM UTC) |
Scrape eBay sold listings + MTGGoldfish market prices |
| Job | Namespace | Schedule | Purpose |
|---|---|---|---|
etcd-backup |
kube-system |
0 2 * * * |
etcd snapshot to S3 (see Backup section above) |
ddns-updater |
public-ingress |
*/5 * * * * (every 5m) |
Updates Route53 *.k3s.strommen.systems A record if public IP changes |
claw-alert-monitor |
cluster-health-monitor |
0 * * * * (hourly) |
Poll AlertManager, auto-remediate CrashLoopBackOff/PVC/cert issues, Slack digest |
claw-auto-remediation |
open-webui |
*/10 * * * * (every 10 min) |
Lightweight remediation check |
weekly-skill-tests |
open-webui |
0 6 * * 0 (Sundays 6:00 AM) |
Test Claude skill effectiveness |
daily-error-report |
open-webui |
0 13 * * * (8:00 AM ET / 13:00 UTC) |
Report on critical issues from nightly review → Slack |
ecr-token-refresh |
kube-system |
0 */6 * * * (every 6h) |
Refresh ECR pull tokens across all namespaces — legacy, active until auto-brand ECR→Harbor migration completes |
Namespace note:
claw-auto-remediation,weekly-skill-tests,daily-error-report,morning-mood-boost, andnightly-wiki-reviewmanifests are filed underkubernetes/apps/cluster-health-monitor/but deploy into theopen-webuinamespace.ECR note:
ecr-token-refreshis a legacy CronJob that refreshes ECR pull tokens every 6h. It remains active becauseauto-brandstill uses ECR. Once auto-brand migrates to Harbor, this job can be removed.
Job: ddns-updater
Image: amazon/aws-cli:2.15.0
Target: Route53 hosted zone Z1LLBOMFGEFI6S
Record: *.k3s.strommen.systems (wildcard A)
Command: curl -sf ifconfig.me | compare with current R53 value -> update if changed
Credentials: ddns-aws-credentials secret (cert-manager-dns01 IAM user recycled)
openclaw-ops namespace)| Job | Schedule | Purpose |
|---|---|---|
openclaw-ops-kg-worker |
0 2 * * * (2:00 AM daily) |
Knowledge graph build -- crawls repo structure, upserts into kg_nodes/kg_edges in openclaw-ops-db |
openclaw-ops-jellyfin-watch-worker |
0 6 * * * (6:00 AM daily) |
Monitors upstream Jellyfin GitHub releases; evaluates HA compatibility via Claude; opens rebuild PR if release is safe |
Jellyfin Watch Worker details:
openclaw-ops-jellyfin-config ConfigMap for currently deployed tag (update manually on each release)jellyfin_releases PostgreSQL table in openclaw-ops-dbopenclaw-personal namespace)| Job | Schedule | Purpose |
|---|---|---|
openclaw-personal-job-worker |
0 6 * * * (6:00 AM daily) |
Scans job boards, evaluates fit, saves to PostgreSQL |
openclaw-personal-resume-worker |
0 8 1 * * (8:00 AM on 1st of month) |
Monthly resume refresh and tailoring pipeline |
openclaw-personal-interview-worker |
On-demand (Job, not CronJob) | Interview prep task runner |
| Job | Namespace | Schedule | Purpose |
|---|---|---|---|
morning-mood-boost |
open-webui |
0 11 * * * (11:00 AM UTC / 7:00 AM ET) |
Fetch good-news brief, post to OpenClaw |
aws-daily-brief |
open-webui |
0 13 * * * (1:00 PM UTC / 9:00 AM ET) |
AWS daily cost and usage brief -> Slack |
aws-weekly-deep-dive |
open-webui |
0 13 * * 1 (Mondays 1:00 PM UTC) |
Detailed weekly AWS cost analysis |
nightly-wiki-review |
open-webui |
30 4 * * * (4:30 AM UTC / 11:30 PM ET) |
Validate wiki documentation, send Slack report |
open-webui namespace)| Job | Namespace | Schedule | Purpose |
|---|---|---|---|
etsy-scraper |
open-webui |
On-demand (Job, not CronJob) | Scrape Etsy listings for Steve Lee Pottery — triggered manually as needed |
| Job | Namespace | Schedule | Purpose |
|---|---|---|---|
rag-ingester |
rag |
*/10 * * * * (every 10 min) |
Embed new documents into Qdrant documents collection |
polymarket-rag-bridge |
polymarket-lab |
*/15 * * * * (every 15 min) |
Embed Polymarket snapshots into Qdrant polymarket-markets |
rag-repo-ingester |
rag |
0 4 * * * (4:00 AM daily) |
Clone home_k3s_cluster repo, embed docs into Qdrant for RAG context |
Files in kubernetes/apps/log-aggregation/ -- deployed to monitoring namespace. See Log Aggregation.
| Job | Namespace | Schedule | Purpose |
|---|---|---|---|
anthropic-usage-to-loki |
monitoring |
30 * * * * (every hour :30) |
Push Anthropic API usage metrics to Loki |
cloudtrail-to-loki |
monitoring |
15 * * * * (every hour :15) |
Push AWS CloudTrail events to Loki |
All scheduled jobs are monitored via PrometheusRules:
| Alert | Trigger | Severity |
|---|---|---|
EtcdBackupJobFailed |
etcd CronJob last completion > 26h ago | Critical |
PostgreSQLBackupJobFailed |
Any postgres backup CronJob last completion > 26h | Critical |
LonghornBackupJobFailed |
Longhorn backup task failed | Warning |
BackupJobStale |
No successful backup in 36 hours | Warning |
kubectl get cronjob -A
kubectl get jobs -A --sort-by=.metadata.creationTimestamp | tail -20
kubectl get job -n cardboard -l app=postgres-backup --sort-by=.metadata.creationTimestamp
kubectl logs -n media job/plex-library-scan-<timestamp>
kubectl logs -n kube-system job/etcd-backup-<timestamp>
aws s3 ls s3://k3s-homelab-backups-855878721457/etcd-snapshots/
aws s3 ls s3://k3s-homelab-backups-855878721457/postgres-backups/cardboard/
00:00-02:00 ------------------------------------------
02:00 etcd snapshot + Velero daily backup (k3s-daily-backup)
openclaw-ops-kg-worker (knowledge graph)
02:00-03:00 ------------------------------------------
03:00 cardboard postgres backup
openclaw-memory-db backup (open-webui ns, separate bucket)
03:15 ham postgres backup
trade-bot postgres backup
03:20 aja-recipes postgres backup
03:25 dnd postgres backup
03:30 wiki postgres backup
jellyfin postgres backup
03:35 digital-signage postgres backup
03:40 openclaw-ops postgres backup
03:45 openclaw-personal postgres backup
03:50 media-profiler postgres backup
04:00 media-controller postgres backup
authentik postgres backup
Velero monthly backup (1st of month only -- k3s-monthly-backup)
rag repo-ingester (daily)
Longhorn local snapshot (daily-snapshot RecurringJob)
04:05 polymarket-lab postgres backup (TimescaleDB via pg_dump)
04:30 nightly-wiki-review
05:00 Longhorn S3 upload (daily-s3-backup RecurringJob)
06:00 Plex library scan (+ every 6h)
openclaw-ops-jellyfin-watch-worker
openclaw-personal-job-worker
weekly-skill-tests (Sundays only, open-webui ns)
Velero weekly backup (Sundays only -- k3s-weekly-backup)
08:00 (1st) openclaw-personal-resume-worker (monthly)
11:00 morning-mood-boost (good news brief)
13:00 aws-daily-brief (daily AWS cost)
daily-error-report (critical issues report, open-webui ns)
13:00 Mon aws-weekly-deep-dive (Mondays only)
23:00 cardboard-scraper (TCGPlayer daily prices)
---- rclone sync every 4h (00, 04, 08, 12, 16, 20)
---- DDNS update every 5 minutes
---- rag-ingester every 10 minutes
---- polymarket-rag-bridge every 15 minutes
---- claw-auto-remediation every 10 minutes (open-webui ns)
---- cloudtrail-to-loki every hour :15 (monitoring ns)
---- anthropic-usage-to-loki every hour :30 (monitoring ns)
---- claw-alert-monitor every hour :00 (cluster-health-monitor ns)
---- ecr-token-refresh every 6h (kube-system ns, legacy)
All backup jobs complete by 04:30 AM UTC. Alerts fire if any job hasn't completed within 26h of its expected window.