diff --git a/.gitea/workflows/ci.yml b/.gitea/workflows/ci.yml index e42ab45..cab844f 100644 --- a/.gitea/workflows/ci.yml +++ b/.gitea/workflows/ci.yml @@ -45,6 +45,25 @@ jobs: - name: Build run: npm run build + # Real browser+HTTP self-test (not mocks). Catches BasePage/ApiClient/network + # regressions before consumers pin a broken tag. See docs/SELFTEST.md. + selftest: + needs: [skip-ci-check, build-and-test] + if: needs.skip-ci-check.outputs.should-skip != '1' + runs-on: [homelab, self-hosted, linux] + container: + image: mcr.microsoft.com/playwright:v1.61.1-jammy + steps: + - uses: actions/checkout@v4 + - name: Install + run: npm ci + - name: Self-test against fake site + run: npm run selftest + env: + CI: 'true' + PLAYKIT_SELFTEST_PORT: '4173' + PLAYKIT_BASE_URL: 'http://127.0.0.1:4173' + secret-scan: needs: skip-ci-check if: needs.skip-ci-check.outputs.should-skip != '1' diff --git a/CHANGELOG.md b/CHANGELOG.md index e2adc5d..5d0b4a7 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,12 +2,16 @@ ## Unreleased +- **Self-test suite** — `selftest/` fake site + Playwright CI job (`docs/SELFTEST.md`, `npm run selftest`) +- **Outline sync** — `scripts/outline-sync-playkit.py` creates/updates QA & Dev → Playkit (needs `documents.update` on the Outline API key) +- Ops soak: Pushgateway (`:9091`) + Grafana `live-playkit` board **applied** on observability LXC (`make deploy-observability`); Prometheus scrape job `pushgateway` healthy; `RELEASE_TOKEN` present and Gitea release `v0.3.1` exists + ## 0.3.1 — 2026-07-14 - **Network interception / error monitor** — `interceptNetworkCall()`, `startNetworkErrorMonitor()` (see `docs/NETWORK.md`) - Tests: expand coverage for network helpers — `NetworkErrorMonitor` (record/dedupe/exclude/minStatus/assert), `interceptNetworkCall` (spy/fulfill/handler/method continue/non-JSON), plus `globToRegExp` / `responseMatchesFilter` - Docs: `docs/OUTLINE.md` checklist (update Outline QA & Dev Playkit page on each release); `docs/IDEAS.md` for OSS-borrowed backlog; adoption pause (no new consumer repos until soak) -- CI: add tag-triggered `release` job (`.gitea/workflows/ci.yml`) — re-runs build/test, verifies tag matches `package.json` version and `CHANGELOG.md` documents it, creates a Gitea release with an `npm pack` tarball attached. Requires a one-time `GITEA_TOKEN` Actions secret. +- CI: add tag-triggered `release` job (`.gitea/workflows/ci.yml`) — re-runs build/test, verifies tag matches `package.json` version and `CHANGELOG.md` documents it, creates a Gitea release with an `npm pack` tarball attached. Requires a one-time `RELEASE_TOKEN` Actions secret. - Docs: bump install pin examples from `v0.1.0` to `v0.3.0` (README, CONSUMER.md) - Docs: lead with Mailpit (homelab default) instead of Mailtrap in README email section; use `createMailInbox()` + `readMailHtml()` in the example instead of a provider-specific client - Ops: Pushgateway + `live-playkit` Grafana board now provisioned via ansible `deploy/observability/` (pending `make deploy-observability`); removed the standalone `dashboards/playkit-overview.json` (superseded) and the `dashboards` entry from `package.json` `files` diff --git a/README.md b/README.md index 1f6ba0e..7d8cf6f 100644 --- a/README.md +++ b/README.md @@ -79,7 +79,7 @@ test('sign-out stays on public host', async ({ page, playkitConfig, timings }) = | `interceptNetworkCall` | Spy or mock the next matching page network call | | `startNetworkErrorMonitor` | Fail tests that silently collect HTTP 4xx/5xx | -See also `docs/NETWORK.md`, `docs/IDEAS.md` (OSS-borrowed backlog), `docs/OUTLINE.md` (live docs sync). +See also `docs/NETWORK.md`, `docs/SELFTEST.md` (kit CI fake site), `docs/IDEAS.md` (OSS-borrowed backlog), `docs/OUTLINE.md` (live docs sync). ## Network (page traffic) @@ -130,9 +130,21 @@ a real address will not appear in either. See ansible `docs/hardening/SECRETS.md npm ci npm run typecheck npm test +npm run selftest # fake site + Chromium (docs/SELFTEST.md) npm run build ``` +## Outline live docs + +After each release (or when consumer-facing docs change): + +```bash +# needs OUTLINE_URL + OUTLINE_API_KEY (ansible: make vault-export-env) +python3 scripts/outline-sync-playkit.py +``` + +See `docs/OUTLINE.md`. + ## Release 1. Bump `version` in `package.json` @@ -141,7 +153,7 @@ npm run build Pushing the tag triggers `.gitea/workflows/ci.yml`'s `release` job: it re-runs typecheck/test/build, verifies the tag matches `package.json`'s `version` and that `CHANGELOG.md` documents it, then creates a Gitea release (with the `npm pack` tarball attached) via the API using a repo-scoped `RELEASE_TOKEN` Actions secret. If any check fails, no release is created — fix and re-tag. Consumers still pin the git tag (`#vX.Y.Z`); the Gitea release is for visibility/changelog, not an npm registry publish (see ROADMAP "private Gitea npm registry"). -4. **Update Outline** — after the release is green, sync the Playkit page under Outline **QA & Dev** (`notes.levkin.ca`). Checklist: `docs/OUTLINE.md`. +4. **Update Outline** — `python3 scripts/outline-sync-playkit.py` (checklist: `docs/OUTLINE.md`). **One-time setup:** add a `RELEASE_TOKEN` secret (repo `Settings → Actions → Secrets` on `ilia/playkit`) scoped to create releases on this repo — separate from the `PLAYKIT_GIT_TOKEN` consumers use to clone it. (Named `RELEASE_TOKEN`, not `GITEA_TOKEN` — Gitea's Actions secrets API rejects that literal name as reserved, same reason `PLAYKIT_GIT_TOKEN` isn't called `GITEA_TOKEN` either.) diff --git a/ROADMAP.md b/ROADMAP.md index df7f767..958c1b7 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -25,11 +25,11 @@ Living plan for making `@levkin/playkit` more useful across Levkin repos. - [ ] **Consumer template** — `npx @levkin/playkit init` scaffolding `e2e/` + CI snippet - [ ] **Deploy-smoke CLI** — `playkit smoke --project punimtag` post-deploy gate - [ ] **Retry policy presets** — flaky-network vs strict-CI profiles -- [ ] **Self-test against a real fake site** — today's unit tests only mock HTTP/mail; `examples/` specs are explicitly *not run* in kit CI. Stand up a tiny demo app (or point at an existing DEV LXC) and run the browser/API/mail helpers against it in CI, so a regression in `BasePage`/`ApiClient`/`assertPublicHost` is caught before a consumer pins a broken tag. +- [x] **Self-test against a real fake site** — `selftest/` tiny Node app + Playwright suite in CI (`docs/SELFTEST.md`) - [x] **Network interception / network error monitor** — `interceptNetworkCall()`, `startNetworkErrorMonitor()` / `assertNoErrors()` (see `docs/NETWORK.md`) -- [x] **Tag-triggered release workflow** — `.gitea/workflows/ci.yml` `release` job now runs on `vX.Y.Z` tag push: re-verifies build/test, checks tag == `package.json` version, checks `CHANGELOG.md` has that version's section, then creates a Gitea release (npm-pack tarball attached) via the API. Needs a one-time `GITEA_TOKEN` Actions secret on this repo (see README "Release"). Still open: `build-and-test`/`secret-scan` also re-run on the tag push (same `on.push` trigger) — harmless redundancy, not wired to skip. -- [ ] **Live docs** — Outline page under **QA & Dev** (`notes.levkin.ca`). Sync checklist in `docs/OUTLINE.md` — update Outline whenever playkit is released. -- [x] **Pushgateway + dashboard wired in ansible** — `pushgateway` service, Prometheus scrape job, and a generated `live-playkit` Grafana board now live in ansible `deploy/observability/` (superseding the old standalone `dashboards/playkit-overview.json`, which is removed). Ops still needs to run `make deploy-observability` against the LXC before `PLAYKIT_METRICS_ENABLED=true` does anything in CI — check with the ansible repo owner before flipping that on. +- [x] **Tag-triggered release workflow** — `.gitea/workflows/ci.yml` `release` job now runs on `vX.Y.Z` tag push: re-verifies build/test, checks tag == `package.json` version, checks `CHANGELOG.md` has that version's section, then creates a Gitea release (npm-pack tarball attached) via the API. Needs a one-time `RELEASE_TOKEN` Actions secret on this repo (see README "Release"). Still open: `build-and-test`/`secret-scan` also re-run on the tag push (same `on.push` trigger) — harmless redundancy, not wired to skip. +- [x] **Live docs** — Outline page under **QA & Dev** (`notes.levkin.ca`). Sync with `python3 scripts/outline-sync-playkit.py` (checklist in `docs/OUTLINE.md`). +- [x] **Pushgateway + dashboard wired in ansible** — applied via `make deploy-observability` (LXC `10.0.10.24`): `pushgateway` container, Prometheus scrape job, Grafana `live-playkit` board. Flip consumer CI metrics with `PLAYKIT_METRICS_ENABLED=true` + `PLAYKIT_PUSHGATEWAY_URL=http://10.0.10.24:9091`. ## Adoption pause diff --git a/docs/OUTLINE.md b/docs/OUTLINE.md index 0cffc25..2fa858f 100644 --- a/docs/OUTLINE.md +++ b/docs/OUTLINE.md @@ -4,6 +4,24 @@ Canonical prose stays in git (`README.md`, `docs/*`, `ROADMAP.md`). Browsable front door: **Outline** → collection **QA & Dev** → doc **Playkit** (`https://notes.levkin.ca`). +## Sync script (preferred) + +From this repo, with Outline credentials loaded: + +```bash +# from ansible: make vault-export-env && set -a && source .env && set +a +python3 scripts/outline-sync-playkit.py +python3 scripts/outline-sync-playkit.py --dry-run +``` + +Creates or updates **QA & Dev → Playkit** with the current `package.json` +version, install pin, what’s-in-the-box digest, and links to repo docs. + +**Required API scopes** (Outline → Settings → API & Access): at least +`collections.list`, `documents.list`, `documents.info`, `documents.create`, +`documents.update`. Without `documents.update` the script can create a first +doc but cannot refresh an existing Playkit page (HTTP 403). + ## When to update Outline Update the Outline Playkit page **whenever playkit ships a release** (after the @@ -11,13 +29,12 @@ tag / Gitea release is green), or whenever you merge a docs-only change that changes consumer behavior: 1. Tag / release finished (or main docs PR merged) -2. Open Outline → QA & Dev → Playkit -3. Sync at least: current version pin, “what’s in the box”, install snippet, - link to CHANGELOG / ROADMAP / NETWORK.md -4. Optional: `make outline-setup` from ansible only if you maintain seed notes there — - prefer editing the living page by hand so it stays readable +2. Run `python3 scripts/outline-sync-playkit.py` +3. Spot-check in Outline (search “Playkit” under QA & Dev) +4. Optional: `make outline-setup` from ansible only if collections are missing — + prefer the sync script for the living Playkit page -Paste template (adjust version): +Paste template (if editing by hand — adjust version): ```markdown # @levkin/playkit @@ -29,6 +46,6 @@ Shared Playwright + API e2e kit. **Source of truth is the git repo.** - Consumers: punimtag (e2e/) — *pause further adoption until soak completes* ## Quick links -- README · CONSUMER.md · NETWORK.md · IDEAS.md · ROADMAP · CHANGELOG +- README · CONSUMER.md · NETWORK.md · IDEAS.md · SELFTEST.md · ROADMAP · CHANGELOG - Metrics: dash.levkin.ca → Live — Playkit e2e ``` diff --git a/docs/SELFTEST.md b/docs/SELFTEST.md new file mode 100644 index 0000000..eafa5a8 --- /dev/null +++ b/docs/SELFTEST.md @@ -0,0 +1,44 @@ +# Kit CI self-test (fake site) + +Unit tests in `src/**/*.test.ts` mock HTTP/mail. They do **not** prove that +`BasePage`, `ApiClient`, or network helpers work against a real browser + HTTP +server. The **selftest** suite fills that gap without depending on punimtag DEV. + +## Layout + +``` +selftest/ + demo-site/server.mjs # tiny Node HTTP app (:4173) + playwright.config.ts # starts webServer, runs Chromium + tests/ + browser.spec.ts # BasePage, host guards, intercept, error monitor + api.spec.ts # ApiClient + Zod schema +``` + +Mail stays unit-tested only (Mailpit/Mailtrap need live SMTP traps). Selftest +does not start Mailpit. + +## Local + +```bash +npx playwright install chromium # once +npm run selftest +``` + +Env (optional): + +| Var | Default | +|-----|---------| +| `PLAYKIT_SELFTEST_PORT` | `4173` | +| `PLAYKIT_BASE_URL` | `http://127.0.0.1:4173` | + +Selftest sets `assertPublicHost(..., false)` when touching `127.0.0.1` — consumer +public e2e must keep the default forbid-private behavior. + +## CI + +`.gitea/workflows/ci.yml` job `selftest` runs in +`mcr.microsoft.com/playwright:v1.61.1-jammy` (must match the pinned +`@playwright/test` version) and executes `npm run selftest` after unit tests. +Failures there mean a regression in kit browser/API helpers *before* a consumer +pins a broken tag. diff --git a/package.json b/package.json index d1c7ec5..c02dc01 100644 --- a/package.json +++ b/package.json @@ -42,6 +42,7 @@ "typecheck": "tsc --noEmit", "test": "vitest run", "test:watch": "vitest", + "selftest": "playwright test -c selftest/playwright.config.ts", "lint": "tsc --noEmit", "prepublishOnly": "npm run build", "example:api": "tsx examples/api/health.example.ts" @@ -58,7 +59,7 @@ } }, "devDependencies": { - "@playwright/test": "^1.52.0", + "@playwright/test": "1.61.1", "@types/node": "^22.15.0", "tsup": "^8.4.0", "tsx": "^4.19.0", diff --git a/scripts/outline-sync-playkit.py b/scripts/outline-sync-playkit.py new file mode 100755 index 0000000..8ede84a --- /dev/null +++ b/scripts/outline-sync-playkit.py @@ -0,0 +1,234 @@ +#!/usr/bin/env python3 +"""Create or update Outline → QA & Dev → Playkit (living front door). + +Canonical source stays in git; this pushes a digest + links to notes.levkin.ca. + +Credentials (required): + OUTLINE_URL + OUTLINE_TOKEN or OUTLINE_API_KEY + +Usage (from ansible vault env, or any shell with those vars): + python3 scripts/outline-sync-playkit.py + python3 scripts/outline-sync-playkit.py --dry-run +""" +from __future__ import annotations + +import argparse +import json +import os +import sys +import urllib.error +import urllib.request +from pathlib import Path + +REPO_ROOT = Path(__file__).resolve().parents[1] +COLLECTION_NAME = "QA & Dev" +DOC_TITLE = "Playkit" + + +def load_package_version() -> str: + pkg = json.loads((REPO_ROOT / "package.json").read_text(encoding="utf-8")) + return str(pkg["version"]) + + +def playkit_markdown(version: str) -> str: + return f"""# @levkin/playkit + +Shared Playwright + API e2e kit for Levkin repos. **Source of truth is the git repo** — this page is the browsable front door. + +- **Repo:** https://git.levkin.ca/ilia/playkit +- **Current:** v{version} +- **Consumers:** punimtag (`e2e/`) — *pause further adoption until soak completes* +- **Install (git pin):** `npm i git+https://git.levkin.ca/ilia/playkit.git#v{version}` + +## What's in the box + +- Browser: `BasePage`, retries (`click`/`fill`/`safeGoto`), public-host guards (`assertPublicHost`, `waitForUrlHost`) +- Network: `interceptNetworkCall`, `startNetworkErrorMonitor` / `assertNoErrors` — see NETWORK.md +- API: `ApiClient` + Zod `schema` / `assertSchema` +- Mail: `createMailInbox()` (Mailpit default, Mailtrap optional) +- Auth helpers: `saveStorageState` / `storageStateUse`, `playkitFailureArtifacts()` +- Metrics: `TimingCollector` → Prometheus Pushgateway → Grafana **Live — Playkit e2e** + +## Quick links (in-repo) + +| Doc | Purpose | +|-----|---------| +| `README.md` | Install, env vars, release steps | +| `docs/CONSUMER.md` | How to wire a consumer `e2e/` | +| `docs/NETWORK.md` | Page traffic spy / error monitor | +| `docs/IDEAS.md` | OSS-borrowed backlog | +| `docs/OUTLINE.md` | When / how to re-sync this page | +| `docs/SELFTEST.md` | Kit CI fake-site self-test | +| `ROADMAP.md` | v0.4+ plan | +| `CHANGELOG.md` | Per-version notes | + +## Metrics + +- Pushgateway: `http://10.0.10.24:9091` (LAN) +- Grafana: `dash.levkin.ca` → **Live — Playkit e2e** (`live-playkit`) +- Enable in CI with `PLAYKIT_METRICS_ENABLED=true` + `PLAYKIT_PUSHGATEWAY_URL` + +## Release checklist (operators) + +1. Tag `vX.Y.Z` (must match `package.json` + CHANGELOG section) → Gitea `release` job uses `RELEASE_TOKEN` +2. Re-run this script: `python3 scripts/outline-sync-playkit.py` +3. Confirm Grafana board + Pushgateway scrape after `make deploy-observability` (ansible) + +_Last synced by `scripts/outline-sync-playkit.py` for v{version}._ +""" + + +class OutlineError(RuntimeError): + pass + + +class OutlineClient: + def __init__(self, base_url: str, token: str, *, dry_run: bool = False) -> None: + self.api_base = f"{base_url.rstrip('/')}/api" + self.token = token + self.dry_run = dry_run + + def call(self, method: str, body: dict | None = None) -> dict: + payload = json.dumps(body or {}).encode() + req = urllib.request.Request( + f"{self.api_base}/{method}", + data=payload, + headers={ + "Authorization": f"Bearer {self.token}", + "Content-Type": "application/json", + }, + method="POST", + ) + if self.dry_run: + print(f"DRY-RUN POST {method}: {json.dumps(body or {}, indent=2)[:800]}") + return {"ok": True, "data": {"id": "dry-run"}} + + try: + with urllib.request.urlopen(req, timeout=60) as resp: + raw = resp.read().decode() + except urllib.error.HTTPError as exc: + detail = exc.read().decode(errors="replace") + raise OutlineError(f"{method} failed (HTTP {exc.code}): {detail[:500]}") from exc + except urllib.error.URLError as exc: + raise OutlineError(f"{method} request failed: {exc}") from exc + + data = json.loads(raw) + if not data.get("ok"): + raise OutlineError(f"{method} returned ok=false: {raw[:500]}") + return data + + def list_collections(self) -> list[dict]: + out: list[dict] = [] + offset = 0 + while True: + batch = self.call("collections.list", {"limit": 100, "offset": offset}).get("data") or [] + out.extend(batch) + if len(batch) < 100: + break + offset += 100 + return out + + def list_documents(self, collection_id: str) -> list[dict]: + out: list[dict] = [] + offset = 0 + while True: + batch = ( + self.call( + "documents.list", + {"limit": 100, "offset": offset, "collectionId": collection_id}, + ).get("data") + or [] + ) + out.extend(batch) + if len(batch) < 100: + break + offset += 100 + return out + + +def require_env() -> tuple[str, str]: + # Allow ansible `.env` next to sibling repo without overwriting set keys + ansible_env = REPO_ROOT.parent / "ansible" / ".env" + if ansible_env.is_file(): + for line in ansible_env.read_text(encoding="utf-8").splitlines(): + line = line.strip() + if not line or line.startswith("#") or "=" not in line: + continue + key, _, value = line.partition("=") + key = key.strip() + if key and key not in os.environ: + os.environ[key] = value.strip().strip('"').strip("'") + + url = os.environ.get("OUTLINE_URL", "").strip() + token = ( + os.environ.get("OUTLINE_TOKEN", "").strip() + or os.environ.get("OUTLINE_API_KEY", "").strip() + ) + missing = [n for n, v in (("OUTLINE_URL", url), ("OUTLINE_TOKEN|OUTLINE_API_KEY", token)) if not v] + if missing: + print( + "ERROR: missing " + + ", ".join(missing) + + "\nLoad: cd ../ansible && make vault-export-env && set -a && source .env && set +a", + file=sys.stderr, + ) + sys.exit(1) + return url, token + + +def main() -> int: + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("--dry-run", action="store_true") + args = parser.parse_args() + + version = load_package_version() + text = playkit_markdown(version) + url, token = require_env() + client = OutlineClient(url, token, dry_run=args.dry_run) + + collections = {c["name"]: c["id"] for c in client.list_collections()} + if COLLECTION_NAME not in collections: + raise OutlineError( + f'collection "{COLLECTION_NAME}" not found — run ansible scripts/outline-setup.py first' + ) + coll_id = collections[COLLECTION_NAME] + docs = {d["title"]: d["id"] for d in client.list_documents(coll_id)} + + try: + if DOC_TITLE in docs: + client.call( + "documents.update", + {"id": docs[DOC_TITLE], "title": DOC_TITLE, "text": text, "publish": True}, + ) + print(f"updated: {COLLECTION_NAME} / {DOC_TITLE} (v{version})") + else: + client.call( + "documents.create", + { + "title": DOC_TITLE, + "text": text, + "collectionId": coll_id, + "publish": True, + }, + ) + print(f"created: {COLLECTION_NAME} / {DOC_TITLE} (v{version})") + except OutlineError as exc: + if "403" in str(exc) or "authorization_error" in str(exc): + print( + "\nERROR: Outline API key can list/create but not update documents.\n" + "In Outline → Settings → API & Access, edit the key and add scopes:\n" + " documents.update documents.delete documents.archive\n" + "(full recommended set is in ansible docs/guides/authentik-apps.md Outline section).\n" + "Then re-run: python3 scripts/outline-sync-playkit.py\n" + "Also delete any leftover 'Playkit sync probe' doc under QA & Dev if present.\n", + file=sys.stderr, + ) + raise + + print(f"Outline: {url.rstrip('/')}/ → search '{DOC_TITLE}' under {COLLECTION_NAME}") + return 0 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/selftest/demo-site/server.mjs b/selftest/demo-site/server.mjs new file mode 100644 index 0000000..17d450c --- /dev/null +++ b/selftest/demo-site/server.mjs @@ -0,0 +1,87 @@ +#!/usr/bin/env node +/** + * Tiny fake app for playkit CI self-tests. + * Endpoints: GET /, GET /form, GET /api/health, GET /api/items, GET /api/boom (500) + */ +import http from 'node:http'; +import { URL } from 'node:url'; + +const PORT = Number(process.env.PLAYKIT_SELFTEST_PORT || 4173); +const HOST = process.env.PLAYKIT_SELFTEST_HOST || '127.0.0.1'; + +const HOME = ` + +Playkit selftest + +

Playkit selftest home

+

Open form

+ + + + +`; + +const FORM = ` + +Form — Playkit selftest + +

Demo form

+ + + + + +`; + +function json(res, status, body) { + const payload = JSON.stringify(body); + res.writeHead(status, { + 'Content-Type': 'application/json', + 'Content-Length': Buffer.byteLength(payload), + }); + res.end(payload); +} + +function html(res, status, body) { + res.writeHead(status, { 'Content-Type': 'text/html; charset=utf-8' }); + res.end(body); +} + +const server = http.createServer((req, res) => { + const u = new URL(req.url || '/', `http://${HOST}:${PORT}`); + if (req.method === 'GET' && u.pathname === '/') return html(res, 200, HOME); + if (req.method === 'GET' && u.pathname === '/form') return html(res, 200, FORM); + if (req.method === 'GET' && u.pathname === '/api/health') { + return json(res, 200, { status: 'ok', service: 'playkit-selftest' }); + } + if (req.method === 'GET' && u.pathname === '/api/items') { + return json(res, 200, { items: [{ id: 1, name: 'alpha' }, { id: 2, name: 'beta' }] }); + } + if (req.method === 'GET' && u.pathname === '/api/boom') { + return json(res, 500, { error: 'intentional' }); + } + json(res, 404, { error: 'not found' }); +}); + +server.listen(PORT, HOST, () => { + console.log(`playkit selftest listening on http://${HOST}:${PORT}`); +}); diff --git a/selftest/playwright.config.ts b/selftest/playwright.config.ts new file mode 100644 index 0000000..674df91 --- /dev/null +++ b/selftest/playwright.config.ts @@ -0,0 +1,29 @@ +import { defineConfig, devices } from '@playwright/test'; + +const port = Number(process.env.PLAYKIT_SELFTEST_PORT || 4173); +const baseURL = process.env.PLAYKIT_BASE_URL || `http://127.0.0.1:${port}`; + +export default defineConfig({ + testDir: './tests', + fullyParallel: true, + forbidOnly: !!process.env.CI, + retries: process.env.CI ? 1 : 0, + workers: process.env.CI ? 2 : undefined, + reporter: process.env.CI ? 'list' : [['list'], ['html', { open: 'never' }]], + use: { + baseURL, + ...devices['Desktop Chrome'], + trace: 'on-first-retry', + }, + webServer: { + command: `node demo-site/server.mjs`, + url: baseURL, + reuseExistingServer: !process.env.CI, + timeout: 30_000, + env: { + ...process.env, + PLAYKIT_SELFTEST_PORT: String(port), + PLAYKIT_SELFTEST_HOST: '127.0.0.1', + }, + }, +}); diff --git a/selftest/tests/api.spec.ts b/selftest/tests/api.spec.ts new file mode 100644 index 0000000..e6eee13 --- /dev/null +++ b/selftest/tests/api.spec.ts @@ -0,0 +1,43 @@ +import { test, expect } from '@playwright/test'; +import { z } from 'zod'; +import { ApiClient, TimingCollector, assertSchema } from '../../src/index.js'; + +const baseUrl = process.env.PLAYKIT_BASE_URL || 'http://127.0.0.1:4173'; + +const HealthSchema = z.object({ + status: z.literal('ok'), + service: z.string(), +}); + +const ItemsSchema = z.object({ + items: z.array(z.object({ id: z.number(), name: z.string() })), +}); + +test.describe('playkit selftest — ApiClient', () => { + test('GET /api/health with Zod schema', async () => { + const timings = new TimingCollector(); + const api = new ApiClient({ baseUrl }); + + const res = await timings.measure('api_health', () => + api.get<{ status: string; service: string }>('/api/health', { + expectedStatus: 200, + schema: HealthSchema, + }), + ); + + expect(res.status).toBe(200); + expect(res.data.status).toBe('ok'); + // signature: assertSchema(data, schema) + expect(assertSchema(res.data, HealthSchema).service).toBe('playkit-selftest'); + }); + + test('GET /api/items + expectedStatus on boom', async () => { + const api = new ApiClient({ baseUrl }); + const items = await api.get('/api/items', { schema: ItemsSchema }); + expect(items.data.items[0]?.name).toBe('alpha'); + expect(items.data.items).toHaveLength(2); + + const boom = await api.get('/api/boom', { expectedStatus: 500 }); + expect(boom.status).toBe(500); + }); +}); diff --git a/selftest/tests/browser.spec.ts b/selftest/tests/browser.spec.ts new file mode 100644 index 0000000..b18a6a6 --- /dev/null +++ b/selftest/tests/browser.spec.ts @@ -0,0 +1,59 @@ +import { test, expect } from '@playwright/test'; +import { + BasePage, + assertPublicHost, + waitForUrlHost, + TimingCollector, + interceptNetworkCall, + startNetworkErrorMonitor, +} from '../../src/index.js'; + +const baseUrl = process.env.PLAYKIT_BASE_URL || 'http://127.0.0.1:4173'; + +test.describe('playkit selftest — browser helpers', () => { + test('BasePage click/fill + public-host relax for localhost', async ({ page }) => { + // Intentional LAN/selftest: forbidPrivateHosts must be off + assertPublicHost(baseUrl, false); + expect(() => assertPublicHost(baseUrl, true)).toThrow(/private host/i); + + const timings = new TimingCollector(); + const home = new BasePage(page, baseUrl); + + await timings.measure('open_home', () => home.open('/')); + await waitForUrlHost(page, '127.0.0.1'); + await expect(page.getByRole('heading', { name: 'Playkit selftest home' })).toBeVisible(); + + await timings.measure('open_form', () => home.open('/form')); + await home.fill(page.locator('#name'), 'Kolby'); + await home.click(page.locator('#submit')); + await expect(page.getByText('Hello, Kolby')).toBeVisible(); + + expect(timings.getSamples().length).toBeGreaterThanOrEqual(2); + }); + + test('network intercept + error monitor', async ({ page }) => { + const home = new BasePage(page, baseUrl); + await home.open('/'); + + const items = interceptNetworkCall({ + page, + method: 'GET', + url: '**/api/items', + }); + await page.getByRole('button', { name: 'Load items' }).click(); + const { status, responseJson } = await items; + expect(status).toBe(200); + expect(responseJson).toMatchObject({ items: expect.any(Array) }); + await expect(page.getByText('alpha')).toBeVisible(); + + const net = startNetworkErrorMonitor(page, { + excludePatterns: [/\/api\/health/], + }); + try { + await page.goto(`${baseUrl}/api/boom`); + // stay on JSON page — monitor should have recorded 500 + } finally { + expect(() => net.assertNoErrors()).toThrow(/network errors/i); + } + }); +});