Jobber/AGENTS.md
Shaheer Sarfaraz 032626bd7d
Fix #162: real-time bulk action streaming progress (#187)
* initial

* refactor: centralize SSE plumbing for client and server

* docs: add centralized SSE usage standards to agents guide

* use sse to stream actions to the client

* ui: align bulk progress toast with default sonner style

* ui: remove hide action from bulk progress toast

* full width progress bar

* fix(stream): track client disconnect and writability

* fix(stream): stop bulk loop when SSE client disconnects

* fix(stream): avoid writing error/end to closed SSE response

* fix(stream): gate started/progress frames on writable SSE socket

* types(api): narrow SSE stream payload input contract

* refactor(ui): share clamp helper for bulk progress

* fix(stream): add heartbeat to bulk action SSE route

* feat(stream): include completed count in bulk completion event

* fix(client-sse): separate parse vs handler errors and cancel reader
2026-02-18 15:54:39 +00:00

4.3 KiB

Error/Logging/Sanitization Standards

This project uses strict operability and privacy defaults for server-side code.

API Response Contract

For all /api/* routes, return:

  • Success: { ok: true, data, meta?: { requestId } }
  • Error: { ok: false, error: { code, message, details? }, meta: { requestId } }

Use consistent status/code mapping:

  • 400 INVALID_REQUEST
  • 401 UNAUTHORIZED
  • 403 FORBIDDEN
  • 404 NOT_FOUND
  • 408 REQUEST_TIMEOUT
  • 409 CONFLICT
  • 422 UNPROCESSABLE_ENTITY
  • 500 INTERNAL_ERROR
  • 502 UPSTREAM_ERROR
  • 503 SERVICE_UNAVAILABLE

Correlation IDs

  • Honor inbound x-request-id when present; otherwise generate one.
  • Always return x-request-id header.
  • Include request ID in API responses (meta.requestId) and logs.
  • Propagate context into async flows (especially pipeline run and per-job work) so logs include pipelineRunId / jobId when available.

Logging Rules

  • Use the shared logger wrapper (infra/logger.ts) in core server paths.
  • Do not add direct console.log, console.warn, or console.error in core paths.
  • Log structured objects, not free-form dumps.
  • Include useful context fields (e.g. requestId, pipelineRunId, jobId, route, status).

SSE Standards

  • Use centralized SSE helpers by default.
  • Server: use orchestrator/src/server/infra/sse.ts for setup, data writes, comments, and heartbeats.
  • Client (EventSource): use orchestrator/src/client/lib/sse.ts for subscription/open/message/error plumbing.
  • Do not duplicate raw SSE setup (Content-Type, Connection, heartbeat loops, or ad-hoc JSON.parse event parsing) when these helpers apply.
  • Keep feature payload types domain-local (pipeline, ghostwriter, bulk actions), but reuse shared transport plumbing.

Redaction and Sanitization

  • Always sanitize objects before logging or returning in error details.
  • Redact sensitive keys by default (authorization, cookie, password, secret, token, apiKey, etc.).
  • Truncate large payloads and long strings.
  • Do not throw/log raw upstream response bodies, full webhook bodies, or large JSON.stringify(...) blobs.

Webhook and LLM Payload Defaults

  • Webhooks: send minimal whitelisted payloads by default.
  • LLM prompts: send only required profile/job fields; avoid unnecessary PII.
  • Document external payload behavior when adding new integrations.

PR Checklist (Routes/Services)

  • API responses follow { ok, data/error, meta.requestId }.
  • Status/code mapping is correct and consistent.
  • Request/correlation IDs appear in logs and async workflows.
  • No raw sensitive payload logging or raw upstream body throws.
  • New/changed webhook or LLM payloads are sanitized and documented.

Documentation Standards (Condensed)

When adding or updating user-facing docs:

  • Use this feature-page structure:
    1. What it is
    2. Why it exists
    3. How to use it
    4. Common problems
    5. Related pages
  • Include frontmatter keys: id, title, description, sidebar_position.
  • Prefer concrete, step-by-step instructions over abstract explanation.
  • Include copy-pasteable examples where relevant.
  • State defaults and constraints explicitly.
  • Link related docs with /docs/... URLs.
  • Any user-visible behavior change should include corresponding docs updates.

Validation / Verification

Before marking work complete, verify changes with the same checks used by CI.

Required CI-parity checks

Run from repository root:

  1. ./orchestrator/node_modules/.bin/biome ci .
  2. npm run check:types:shared
  3. npm --workspace orchestrator run check:types
  4. npm --workspace gradcracker-extractor run check:types
  5. npm --workspace ukvisajobs-extractor run check:types
  6. npm --workspace orchestrator run build:client
  7. npm --workspace orchestrator run test:run

Native module note (better-sqlite3)

If tests fail with a Node ABI mismatch for better-sqlite3, rebuild it before running tests:

  • npm --workspace orchestrator rebuild better-sqlite3

CI runs on Node 22. If local behavior differs, verify with Node 22 before concluding a change is valid.

Scope-specific checks

  • For focused changes, run targeted tests first (for touched files/modules), then still run the full CI-parity list above before finalizing.
  • A change is considered valid only when all required checks pass without ignored failures.