chore: gitignore cron Telegram env; ship pipeline+Telegram script and deploy steps

Made-with: Cursor

.gitignore

@@ -1,6 +1,8 @@
 # Environment files
 .env
 *.env.local
+# Cron/Telegram secrets (copy template from scripts/jobber-cron.env.example)
+scripts/jobber-cron.env
 
 # Dependencies
 node_modules/

DEPLOY_GITEA_VM_CRON_TELEGRAM.md

@@ -1,13 +1,26 @@
-# Deploy on a VM or container, run the pipeline on a schedule, notify Telegram
+# Deploy on a VM, run the pipeline on a schedule, notify Telegram
 
-This guide assumes you already pushed this repo to Gitea, for example:
+End-to-end checklist:
+
+1. Push this repo to your Git host (Gitea, GitHub, etc.).
+2. On the server: install Docker + Compose v2, clone the repo, copy `.env.example` → `.env`, run `docker compose up -d`.
+3. Confirm the UI/API on the mapped host port (default **3005** → container **3001**).
+4. Add a cron job that `POST`s `/api/pipeline/run` (see §3).
+5. Optional: Telegram via `scripts/jobber-pipeline-telegram.sh`, pipeline webhook relay, or n8n (see §4).
+
+---
+
+## Git remote (example)
+
+Replace host, user/org, and repo name with yours:
 
 ```bash
-git remote add gitea gitea@10.0.30.169:ilia/Jobber.git   # or: git remote set-url gitea ...
+git remote add gitea git@GITEA_HOST:YOUR_USER/Jobber.git
+# or: git remote set-url gitea ...
 git push -u gitea main
 ```
 
-If you have **uncommitted** changes, commit them first, then push again:
+If you have uncommitted changes:
 
 ```bash
 git add -A && git commit -m "Your message" && git push gitea main
@@ -18,21 +31,21 @@ git add -A && git commit -m "Your message" && git push gitea main
 ## 1. Deploy on a Linux VM (bare metal or cloud)
 
 1. Install **Docker** and **Docker Compose** (plugin v2).
-2. Clone from your Gitea server (SSH or HTTPS):
+2. Clone from your Git server (SSH or HTTPS):
 
    ```bash
-   git clone gitea@10.0.30.169:ilia/Jobber.git
+   git clone git@GITEA_HOST:YOUR_USER/Jobber.git
-   cd Jobber   # or job-ops if you kept that folder name
+   cd Jobber
    ```
 
-3. Copy and edit environment:
+3. Environment:
 
    ```bash
    cp .env.example .env
    # Edit .env: MODEL / LLM keys, RXRESUME_*, search settings, etc.
    ```
 
-4. Start the stack:
+4. Start:
 
    ```bash
    docker compose up -d
@@ -40,36 +53,36 @@ git add -A && git commit -m "Your message" && git push gitea main
 
 5. Open the UI: `http://<VM-IP>:3005` (port mapped in `docker-compose.yml`).
 
-6. Persist data: the compose file mounts `./data` — back up that directory.
+6. Persist data: compose mounts `./data` — back up that directory.
 
 ---
 
 ## 2. Deploy as a container (same image, any host)
 
-Same as the VM path: only Docker is required. On the VM:
+Same as the VM path: only Docker is required.
 
 - Ensure port **3005** (or your chosen host port) is reachable if you use the UI from another machine.
-- For **only** API/cron use from localhost, you can bind to `127.0.0.1:3005` by changing the `ports:` line in `docker-compose.yml` if you edit it (e.g. `"127.0.0.1:3005:3001"`).
+- For **only** API/cron from localhost, bind to `127.0.0.1:3005` by changing the `ports:` line in `docker-compose.yml` (e.g. `"127.0.0.1:3005:3001"`).
 
-Inside the container the app listens on **3001**; the host maps **3005 → 3001** by default.
+Inside the container the app listens on **3001**; the default host map is **3005 → 3001**.
 
 **Cron on the host** should call the API on the host:
 
-- UI: `http://127.0.0.1:3005` (browser)
+- Browser: `http://127.0.0.1:3005`
-- **API (orchestrator)**: `http://127.0.0.1:3005` — same port; requests to `/api/...` are served by the app behind the reverse proxy built into the image.
+- **API**: same origin; `/api/...` is served by the app.
 
-If your setup exposes the API only on an internal Docker network, use the container name and port `3001` from another container, or publish `3005` on the host and use `127.0.0.1:3005` from cron.
+If the API is only on a Docker network, use the container name and port `3001` from another container, or publish `3005` on the host and use `127.0.0.1:3005` from cron.
 
 ---
 
-## 3. Run the pipeline three times a day (cron)
+## 3. Run the pipeline on a schedule (cron)
 
-`POST /api/pipeline/run` **starts** the pipeline in the **background** and returns immediately (`{ ok: true, data: { message: "Pipeline started" } }`). That is enough for scheduling.
+`POST /api/pipeline/run` **starts** the pipeline in the **background** and returns quickly (`{ ok: true, data: { message: "Pipeline started" } }`). That is enough for scheduling.
 
-Example **crontab** entries (host time zone — adjust hours as you like):
+Example crontab (host timezone — adjust hours):
 
 ```cron
-# 08:00, 14:00, 20:00 daily — trigger JobOps pipeline
+# 08:00, 14:00, 20:00 daily
 0 8,14,20 * * * /usr/local/bin/jobops-pipeline-run.sh >> /var/log/jobops-pipeline.log 2>&1
 ```
 
@@ -79,7 +92,7 @@ Create `/usr/local/bin/jobops-pipeline-run.sh`:
 #!/usr/bin/env bash
 set -euo pipefail
 BASE_URL="${JOBOPS_URL:-http://127.0.0.1:3005}"
-# If you set BASIC_AUTH_USER / BASIC_AUTH_PASSWORD in .env, uncomment:
+# If BASIC_AUTH_USER / BASIC_AUTH_PASSWORD are set in .env, uncomment:
 # AUTH=(-u "${BASIC_AUTH_USER:?}:${BASIC_AUTH_PASSWORD:?}")
 
 curl -sS -X POST "${BASE_URL}/api/pipeline/run" \
@@ -94,22 +107,22 @@ echo >> /var/log/jobops-pipeline.log
 sudo chmod +x /usr/local/bin/jobops-pipeline-run.sh
 ```
 
-Optional: set `JOBOPS_URL` in root’s crontab or in `/etc/environment` if the app is on another host.
+Set `JOBOPS_URL` in root’s crontab or `/etc/environment` if the app is on another host.
 
-**Basic Auth:** When `BASIC_AUTH_USER` and `BASIC_AUTH_PASSWORD` are set in `.env`, all non-GET API calls need Basic auth — use `curl -u user:pass` as above.
+**Basic auth:** When `BASIC_AUTH_USER` and `BASIC_AUTH_PASSWORD` are in `.env`, non-GET API calls need Basic auth — use `curl -u user:pass` as above.
 
 ---
 
 ## 4. Telegram notifications
 
-JobOps does **not** send Telegram directly. Practical options:
+The app does **not** send Telegram by itself. Practical options:
 
 ### Option A — Pipeline webhook (recommended)
 
 1. In the app: **Settings → Webhooks** (or env `PIPELINE_WEBHOOK_URL` / `WEBHOOK_SECRET`) set a URL that receives JSON when a run **completes or fails**.
-2. Point that URL to a **small relay** that translates the JSON into a Telegram `sendMessage` call.
+2. Point that URL to a small relay that maps the JSON to Telegram `sendMessage`.
 
-Telegram API:
+Telegram HTTP API:
 
 ```text
 https://api.telegram.org/bot<BOT_TOKEN>/sendMessage
@@ -124,56 +137,147 @@ Body (JSON):
 }
 ```
 
-You can host the relay on the same VM (Flask/FastAPI/Node, or **n8n** / **Webhook.site** + automation). Keep the **bot token** and **chat id** in env vars, not in the JobOps UI if possible.
+Host the relay on the same VM (Flask/FastAPI/Node, or n8n). Keep **bot token** and **chat id** in environment variables.
 
-Webhook payload shape (sanitized) includes fields like `event`, `pipelineRunId`, `jobsDiscovered`, `jobsProcessed`, `error` — see server code `notify-webhook.ts`.
+Payload shape (sanitized) includes fields like `event`, `pipelineRunId`, `jobsDiscovered`, `jobsProcessed`, `error` — see `orchestrator/src/server/pipeline/steps/notify-webhook.ts`.
 
-### Option B — Cron wrapper: poll status, then Telegram
+### Option B — Shipped script: run pipeline + Telegram summary (cron-friendly)
 
-Because `/api/pipeline/run` returns before the run finishes, a simple approach:
+The repo includes `scripts/jobber-pipeline-telegram.sh`: it `POST`s `/api/pipeline/run`, polls `GET /api/pipeline/status` until the run ends, then sends one Telegram with **status**, **jobsDiscovered**, and **jobsProcessed** (and **errorMessage** if failed).
 
-1. Cron calls `jobops-pipeline-run.sh` (as above).
+**1. Dependencies on the host** (LXC/VM that runs cron):
-2. A **second** script (or same script extended) polls `GET /api/pipeline/status` until `isRunning` is false, then reads `GET /api/pipeline/runs` for the latest run and sends a short message via `curl` to Telegram.
 
-Example **send** (replace token and chat id):
+```bash
+apt-get update && apt-get install -y jq curl
+```
+
+**2. Install script and secrets** (after `git pull` in `/opt/Jobber` or your clone path):
+
+```bash
+install -m 755 /opt/Jobber/scripts/jobber-pipeline-telegram.sh /usr/local/bin/jobber-pipeline-telegram.sh
+cp /opt/Jobber/scripts/jobber-cron.env.example /root/.jobber-cron.env
+chmod 600 /root/.jobber-cron.env
+nano /root/.jobber-cron.env
+```
+
+Fill **`TELEGRAM_BOT_TOKEN`** (from @BotFather) and **`TELEGRAM_CHAT_ID`**. For a **private** chat with your bot, use `message.chat.id` from `getUpdates` (same as your Telegram user id in the JSON). **`JOBOPS_URL`** defaults to `http://127.0.0.1:3005` when Jobber runs on the same host.
+
+**3. Manual test** (before cron):
+
+```bash
+/usr/local/bin/jobber-pipeline-telegram.sh
+```
+
+You should get one Telegram when the pipeline finishes. Optional log: append `>> /var/log/jobber-pipeline.log 2>&1` on the cron line.
+
+**4. Cron** (example: 08:00, 14:00, 20:00 host local time — `crontab -e`):
+
+```cron
+0 8,14,20 * * * /usr/local/bin/jobber-pipeline-telegram.sh >> /var/log/jobber-pipeline.log 2>&1
+```
+
+**Security:** Never commit `/root/.jobber-cron.env` or paste bot tokens in Git. Revoke the token in BotFather if it was exposed.
+
+### Option B2 — Minimal curl-only (no wait-for-finish)
+
+If you only want to **trigger** the pipeline from cron without this script, use §3. For a one-off Telegram without polling:
 
 ```bash
 TELEGRAM_BOT_TOKEN="123456:ABC..."
 CHAT_ID="your_numeric_chat_id"
-MSG="$(printf 'JobOps pipeline finished. Check dashboard.')"
+MSG="$(printf 'Pipeline finished. Check dashboard.')"
 curl -sS -X POST "https://api.telegram.org/bot${TELEGRAM_BOT_TOKEN}/sendMessage" \
   -H "Content-Type: application/json" \
   -d "{\"chat_id\":\"${CHAT_ID}\",\"text\":$(echo "$MSG" | jq -Rs .)}"
 ```
 
-Get **chat_id**: message your bot, then open `https://api.telegram.org/bot<TOKEN>/getUpdates` and read `message.chat.id`.
+**chat_id:** Message your bot, then open `https://api.telegram.org/bot<TOKEN>/getUpdates` and read `message.chat.id` (if `result` is empty, send **Start** to the bot first, or call `deleteWebhook` if a webhook was set).
 
 ### Option C — External automation
 
-Use **n8n**, **Grafana OnCall**, or similar: trigger on schedule → HTTP POST ` /api/pipeline/run` → wait/poll → Telegram node.
+Use n8n, Grafana OnCall, or similar: schedule → `POST /api/pipeline/run` → wait/poll → Telegram node.
 
 ---
 
 ## 5. Security notes
 
-- Do not commit `.env` or Telegram tokens to Git.
+- Do not commit `.env` or Telegram tokens.
-- Prefer **Basic Auth** on the instance if it is reachable from the internet.
+- Prefer Basic Auth if the instance is reachable from the internet.
-- Restrict firewall so only your IP (or VPN) can reach port 3005 if exposed.
+- Restrict firewall so only your IP (or VPN) can reach the published port if exposed.
 
 ---
 
-## 6. Git remotes quick reference
+## 6. Git remotes (reference)
 
 ```bash
 git remote -v
-git push gitea main          # your Gitea
+git push origin main    # or: git push gitea main — whatever you configured
-git push origin main         # upstream GitHub (if you have rights)
 ```
 
 ---
 
-## Related project docs
+## Related
 
-- Self-hosting: docs site **Self-Hosting** guide (if present in your tree).
+- Env knobs: `PIPELINE_WEBHOOK_URL`, `WEBHOOK_SECRET`, `BASIC_AUTH_USER`, `BASIC_AUTH_PASSWORD` in `.env.example`.
-- Webhooks: **Settings** documentation for pipeline / job-complete webhooks.
+- Local docs: `npm run docs:dev` from the repo root.
-- Optional env: `PIPELINE_WEBHOOK_URL`, `WEBHOOK_SECRET`, `BASIC_AUTH_USER`, `BASIC_AUTH_PASSWORD` in `.env.example`.
+
+---
+
+## 7. Proxmox: VM vs LXC, sizing, fast setup
+
+### VM or container (LXC)?
+
+| | **QEMU VM (recommended)** | **LXC** |
+|---|---------------------------|--------|
+| **Docker** | Works the same as on any Linux server. | Possible with `nesting=1` (and sometimes `keyctl`); more Proxmox/Docker footguns. |
+| **This app** | Playwright/Firefox + Node inside Docker — predictable. | Same stack *can* work in nested Docker, but troubleshooting is harder. |
+| **Overhead** | Slightly higher RAM for a full kernel. | Lower overhead per CT. |
+
+**Choose a VM** unless you already run Docker in LXC on this cluster and know the knobs. For speed and fewer surprises: **Ubuntu 24.04 LTS cloud image**, 2–4 vCPU, 4–8 GB RAM, **≥ 40 GB** disk (Docker layers + `./data`).
+
+**Rough sizing**
+
+- **Light personal use:** 2 vCPU, **4 GB RAM**, 40 GB disk — often enough.
+- **Comfortable (pipelines + browsers + headroom):** 4 vCPU, **6–8 GB RAM**, 64 GB disk.
+- **Tight:** 2 GB RAM can work for idle UI only; **scraping/LLM runs will swap or OOM** — avoid.
+
+### Proxmox UI (once per guest)
+
+1. **Create VM** → ISO or cloud-init image (e.g. Ubuntu 24.04).
+2. **Network**: bridge (e.g. `vmbr0`) so the guest gets a LAN IP.
+3. **Disk**: virtio, discard on if SSD.
+4. **CPU type:** `host` if single-node and you want a tiny perf edge; `kvm64` is fine.
+5. After install: **Guest agent** optional but handy for IP in Proxmox UI.
+
+### One-shot shell setup (inside the Ubuntu VM)
+
+Run as a user with `sudo`. Set `REPO_URL` to your Git remote (HTTPS or SSH). First build can take several minutes.
+
+```bash
+set -euo pipefail
+REPO_URL="${REPO_URL:-https://github.com/YOUR_USER/Jobber.git}"  # change
+APP_DIR="${APP_DIR:-$HOME/Jobber}"
+
+sudo apt-get update
+sudo apt-get install -y ca-certificates curl git
+
+# Docker Engine + Compose plugin (official convenience script; review if you prefer manual repo install)
+curl -fsSL https://get.docker.com | sudo sh
+sudo usermod -aG docker "$USER"
+# Log out and back in so `docker` works without sudo, or use `newgrp docker` for this session:
+newgrp docker || true
+
+git clone "$REPO_URL" "$APP_DIR"
+cd "$APP_DIR"
+cp .env.example .env
+echo "Edit .env now (LLM keys, RXRESUME, etc.), then run: docker compose up -d --build"
+```
+
+Then edit `.env`, then:
+
+```bash
+cd "$APP_DIR"
+docker compose up -d --build
+```
+
+Open `http://<VM-IP>:3005`. Persist backups of `$APP_DIR/data` and your `.env`.

scripts/jobber-cron.env.example

@@ -0,0 +1,11 @@
+# Copy to /root/.jobber-cron.env and chmod 600. Do not commit real values.
+#
+# TELEGRAM_CHAT_ID: from getUpdates → result[0].message.chat.id
+#   Private DMs: usually the same as your Telegram user id.
+TELEGRAM_BOT_TOKEN=""
+TELEGRAM_CHAT_ID=""
+JOBOPS_URL="http://127.0.0.1:3005"
+
+# Optional — only if BASIC_AUTH_USER / BASIC_AUTH_PASSWORD are set in Jobber .env
+# BASIC_AUTH_USER=""
+# BASIC_AUTH_PASSWORD=""

scripts/jobber-pipeline-telegram.sh

@@ -0,0 +1,77 @@
+#!/usr/bin/env bash
+# Run Jobber pipeline, wait until it finishes, send summary to Telegram.
+# Secrets: copy scripts/jobber-cron.env.example to /root/.jobber-cron.env (chmod 600).
+set -euo pipefail
+
+ENV_FILE="${JOBBER_CRON_ENV:-/root/.jobber-cron.env}"
+if [[ ! -f "$ENV_FILE" ]]; then
+  echo "Missing env file: $ENV_FILE (set JOBBER_CRON_ENV or create the default path)" >&2
+  exit 1
+fi
+# shellcheck source=/dev/null
+source "$ENV_FILE"
+
+: "${TELEGRAM_BOT_TOKEN:?Set TELEGRAM_BOT_TOKEN in $ENV_FILE}"
+: "${TELEGRAM_CHAT_ID:?Set TELEGRAM_CHAT_ID in $ENV_FILE}"
+
+BASE="${JOBOPS_URL:-http://127.0.0.1:3005}"
+AUTH=()
+if [[ -n "${BASIC_AUTH_USER:-}" && -n "${BASIC_AUTH_PASSWORD:-}" ]]; then
+  AUTH=(-u "${BASIC_AUTH_USER}:${BASIC_AUTH_PASSWORD}")
+fi
+
+send_tg() {
+  local msg="$1"
+  curl -sS -X POST "https://api.telegram.org/bot${TELEGRAM_BOT_TOKEN}/sendMessage" \
+    -H "Content-Type: application/json" \
+    -d "$(jq -n --arg c "$TELEGRAM_CHAT_ID" --arg t "$msg" '{chat_id: $c, text: $t}')" >/dev/null
+}
+
+fetch_status() {
+  curl -sS "${AUTH[@]}" "${BASE}/api/pipeline/status"
+}
+
+body="$(fetch_status)"
+if ! echo "$body" | jq -e '.ok == true' >/dev/null 2>&1; then
+  send_tg "Jobber: /api/pipeline/status failed (before run). Check container."
+  exit 1
+fi
+
+if echo "$body" | jq -e '.data.isRunning == true' >/dev/null 2>&1; then
+  send_tg "Jobber: pipeline already running; skipping scheduled run."
+  exit 0
+fi
+
+resp="$(curl -sS "${AUTH[@]}" -X POST "${BASE}/api/pipeline/run" \
+  -H "Content-Type: application/json" -d '{}')"
+if ! echo "$resp" | jq -e '.ok == true' >/dev/null 2>&1; then
+  send_tg "Jobber: POST /api/pipeline/run failed: $(echo "$resp" | jq -c .)"
+  exit 1
+fi
+
+was_running=0
+for _ in $(seq 1 720); do
+  sleep 30
+  body="$(fetch_status)"
+  if ! echo "$body" | jq -e '.ok == true' >/dev/null 2>&1; then
+    send_tg "Jobber: status check failed mid-run."
+    exit 1
+  fi
+  running="$(echo "$body" | jq -r '.data.isRunning')"
+  if [[ "$running" == "true" ]]; then
+    was_running=1
+  elif [[ "$was_running" -eq 1 ]]; then
+    lr="$(echo "$body" | jq '.data.lastRun')"
+    st="$(echo "$lr" | jq -r '.status // "unknown"')"
+    disc="$(echo "$lr" | jq -r '.jobsDiscovered // 0')"
+    proc="$(echo "$lr" | jq -r '.jobsProcessed // 0')"
+    err="$(echo "$lr" | jq -r '.errorMessage // empty')"
+    msg="Jobber pipeline finished: ${st}. Discovered: ${disc}, processed: ${proc}."
+    [[ -n "$err" ]] && msg="${msg}"$'\n'"Error: ${err}"
+    send_tg "$msg"
+    exit 0
+  fi
+done
+
+send_tg "Jobber: timed out waiting for pipeline (6h). Check server."
+exit 1