test: timing budgets + CI hardening (artifact v3 pin, npm cache retry)
CI / skip-ci-check (pull_request) Successful in 5s
CI / docker-ci (pull_request) Successful in 7s
CI / secret-scan (pull_request) Successful in 12s
CI / viewer-unit (pull_request) Successful in 1m29s
CI / e2e (pull_request) Successful in 4m8s
CI / admin-unit (pull_request) Successful in 4m24s

Closes three items from the outstanding e2e/CI gap list:

- Timing budgets: timings.measure() only ever recorded durations for the
  (still-unwired) Pushgateway export — nothing failed CI when a step got
  slow. Add e2e/timing-budgets.ts (expectWithinBudget + shared BUDGET_MS
  buckets) and wire it into every measure() call site across the suite.
  Mail-wait steps are deliberately left unbudgeted (external mail-trap
  delivery latency, not a code performance signal).

- actions/upload-artifact@v4 doesn't work against this Gitea/act runner's
  artifact backend — pin to v3 for the e2e failure-report upload.

- Shared act_runner npm cache has corrupted platform-native tarballs before
  (@next/swc-linux-x64-musl) and reds viewer-unit/admin-unit/e2e with no
  product bug involved. All three npm ci steps now retry once after
  `npm cache clean --force` on first failure.

Verified: full local suite green against DEV (37 passed, 6 skipped, no
budget assertion failures) before wiring into CI.
This commit is contained in:
2026-07-15 08:56:31 -04:00
parent 96351288f0
commit c0c997f796
22 changed files with 155 additions and 6 deletions
@@ -1,6 +1,7 @@
import type { Page } from '@playwright/test';
import { test, expect } from '../fixtures';
import { LoginPage } from '../pages/LoginPage';
import { BUDGET_MS, expectWithinBudget } from '../timing-budgets';
/**
* Real Manage Users CRUD through the admin UI (`admin.manage-users.spec.ts`
@@ -62,6 +63,8 @@ test.describe('admin manage users: create/edit/delete @smoke', () => {
await timings.measure('create_user', () =>
manageUsersPanel.createUser({ email, password: THROWAWAY_PASSWORD, name: 'E2E Throwaway' }),
);
// Generous bucket — shared-CI DEV LXC under concurrent load (see file header).
expectWithinBudget(timings, 'create_user', BUDGET_MS.heavyCrud);
expect(await manageUsersPanel.statusBadgeText(email)).toMatch(/active/i);
expect(await manageUsersPanel.roleBadgeText(email)).toMatch(/user/i);
expect(await manageUsersPanel.writeAccessText(email)).toMatch(/no/i);
@@ -71,9 +74,11 @@ test.describe('admin manage users: create/edit/delete @smoke', () => {
await manageUsersPanel.setWriteAccess(true);
await manageUsersPanel.saveEdit();
});
expectWithinBudget(timings, 'edit_user', BUDGET_MS.heavyCrud);
expect(await manageUsersPanel.writeAccessText(email)).toMatch(/yes/i);
await timings.measure('delete_user', () => manageUsersPanel.deleteUser(email));
expectWithinBudget(timings, 'delete_user', BUDGET_MS.heavyCrud);
await manageUsersPanel.waitForRowGone(email);
await accountMenu.closeManageUsers();
@@ -112,6 +117,7 @@ test.describe('admin manage users: causal cross-session effect @smoke', () => {
await timings.measure('create_throwaway', () =>
manageUsersPanel.createUser({ email, password: THROWAWAY_PASSWORD, name: 'E2E Throwaway' }),
);
expectWithinBudget(timings, 'create_throwaway', BUDGET_MS.heavyCrud);
await accountMenu.closeManageUsers();
// Independent second session/actor — the throwaway user, logged in
@@ -126,6 +132,9 @@ test.describe('admin manage users: causal cross-session effect @smoke', () => {
await userPage.waitForURL((url) => !url.pathname.includes('/login'), { timeout: 30_000 });
await userPage.getByLabel('Account menu').waitFor({ state: 'visible', timeout: 30_000 });
});
// Login flows on this file have hit ~90s under a concurrent CI burst
// (see comment above) — budget matches the file's own 120s headroom.
expectWithinBudget(timings, 'throwaway_login', BUDGET_MS.heavyCrud);
const preRes = await userPage.request.get(`${playkitConfig.baseUrl}/api/auth/session`);
const preBody = await preRes.json();
@@ -139,6 +148,7 @@ test.describe('admin manage users: causal cross-session effect @smoke', () => {
await manageUsersPanel.setActive(false);
await manageUsersPanel.saveEdit();
});
expectWithinBudget(timings, 'deactivate_user', BUDGET_MS.heavyCrud);
// The table defaults to (and refetches with) the "Active only" filter,
// so the just-deactivated row drops out of view immediately — switch to
// "All" before reading its badge, or this hangs forever waiting for a
@@ -154,6 +164,7 @@ test.describe('admin manage users: causal cross-session effect @smoke', () => {
const postBody = await postRes.json();
expect(postBody?.user).toBeFalsy();
});
expectWithinBudget(timings, 'session_revoked', BUDGET_MS.uiAction);
} finally {
await userContext?.close();
await deleteUserByEmail(page, playkitConfig.baseUrl, email).catch(() => undefined);