FigJam Diagram: Etsy Scraper — StoneWareArts Listing Sync (expires 2026-04-13)
One-off Kubernetes Job that scrapes the StoneWareArts Etsy shop using Playwright (headless Chromium), then updates the steve-lee-index ConfigMap with current product data for the Steve Lee portfolio site.
| Namespace | open-webui |
| Type | Job (manual trigger, not scheduled) |
| Image | harbor.k3s.internal.strommen.systems/production/etsy-scraper:latest |
| Shop | StoneWareArts on Etsy |
The Job uses an init container + main container pattern:
scraper — runs Playwright to scrape all Etsy listings, writes JSON to shared emptyDir at /output/stonewarearts.jsonsite-updater — reads the JSON, patches the steve-lee-index ConfigMap in the open-webui namespace via Kubernetes APIFrom etsy-scraper-config ConfigMap:
| Key | Value | Purpose |
|---|---|---|
SHOP_NAME |
stonewarearts |
Etsy shop to scrape |
MIN_DELAY / MAX_DELAY |
2.0 / 6.0 |
Random delay between page loads |
DETAIL_MIN_DELAY / DETAIL_MAX_DELAY |
3.0 / 8.0 |
Delay between detail pages |
MAX_PAGES |
0 (unlimited) |
Max pagination pages |
MAX_LISTINGS |
0 (unlimited) |
Max listings to fetch |
MAX_PRODUCTS_ON_SITE |
12 |
Products shown on Steve Lee site |
HEADLESS |
true |
Run Chromium headless |
The Job uses a dedicated etsy-scraper ServiceAccount with a Role scoped to get and patch only the steve-lee-index ConfigMap.
| Container | CPU req/limit | Memory req/limit |
|---|---|---|
| scraper (init) | 500m / 1000m | 1Gi / 2Gi |
| site-updater | 100m / 200m | 128Mi / 256Mi |
# Delete previous job (if exists) and re-apply
kubectl delete job etsy-scraper -n open-webui --ignore-not-found
kubectl apply -f kubernetes/apps/etsy-scraper/etsy-scraper-job.yaml
kubectl logs -n open-webui job/etsy-scraper -c scraper -f
kubernetes/apps/etsy-scraper/etsy-scraper-job.yaml