Author SHA1 Message Date
ilia ea00c8dbb4 ci: sync gitleaks allowlist 2026-07-12 16:23:47 -05:00
ilia 27b21b4701 ci: sync workflow template 2026-07-12 16:23:46 -05:00
156 changed files with 3514 additions and 10299 deletions
-44
View File
@@ -254,47 +254,3 @@ ADZUNA_APP_KEY=
# Comma-separated paths under https://arc.dev used when seeding defaults (e.g. Playwright + Cypress feeds).
# ARC_REMOTE_JOBS_PATHS=/remote-jobs/playwright,/remote-jobs/cypress
# Prefer Settings for overrides: arcRemoteJobsPaths (JSON array), arcMaxJobsPerPath (default 120).
# =============================================================================
# New public job boards — optional (auto-enabled, no credentials)
# =============================================================================
# Caps via Settings: workingnomadsMaxJobsPerTerm, testdevjobsMaxJobsPerTerm,
# testdevjobsMaxPages, builtinMaxJobsPerTerm, builtinMaxPagesPerTerm,
# wellfoundMaxJobsPerTerm, googleJobsMaxJobsPerTerm (Wellfound and Google
# Jobs both use Camoufox when installed).
# =============================================================================
# Google Jobs — optional (auto-enabled, no credentials, unofficial scrape)
# =============================================================================
# GOOGLE_JOBS_PROXY=http://user:pass@proxy-host:port
# ^ Most reliable way to avoid Google's CAPTCHA/rate-limit wall (residential or
# mobile proxy recommended; datacenter IPs, including most VPS/cloud hosts,
# are aggressively blocked). Applies to both manual runs and the pipeline.
#
# GOOGLE_JOBS_HEADLESS=false
# ^ Manual/local use ONLY (do not set this in the server .env for the
# unattended pipeline/cron — a headless server has no display for a human
# to see the browser). Opens a real Firefox window so you can solve a
# CAPTCHA by hand; waits up to GOOGLE_JOBS_UNBLOCK_TIMEOUT_MS (default
# 300000 = 5 min) before giving up. Run directly, e.g.:
# GOOGLE_JOBS_HEADLESS=false npm --workspace google-jobs-extractor run start
# Session cookies are saved to extractors/google-jobs/storage/state.json
# and reused by later runs (including headless/automated ones), which can
# reduce how often future runs hit a CAPTCHA.
#
# Caps via Settings: googleJobsMaxJobsPerTerm (default 30, max 150).
#
# Note: there is no country/location override for this extractor — Google
# resolves location from the browser's real (or GOOGLE_JOBS_PROXY's) IP.
# Forcing a mismatched country has been observed to make Google's Jobs
# vertical return "no matches" even though the tab loads correctly. To
# target a different country's jobs, use a proxy with an exit node there.
# =============================================================================
# Teamtailor / Huntflow / Factorial / Career pages — optional
# =============================================================================
# TEAMTAILOR_COMPANIES=testgorilla,polestar
# HUNTFLOW_TENANTS=apicworld
# FACTORIAL_TENANTS=yourbourse
# CAREERS_PAGE_URLS=https://sentry.io/careers/,https://example.com/careers
# Prefer Settings → Job sources & ATS for the same fields.
-1
View File
@@ -1,5 +1,4 @@
---
# ci-sync: 2026-05-30T02:31:18Z
# Homelab CI — Docker/heavy lane (git-ci-02)
name: CI
+1 -6
View File
@@ -18,11 +18,6 @@ data/
# Extractor storage outputs and cached auth (per-extractor runtime data)
extractors/*/storage/
# OS / editor
# OS files
.DS_Store
Thumbs.db
.vscode/
# Local scratch / extractor debug dumps (not part of the app)
storage/
solution-workload-test.*
-4
View File
@@ -20,13 +20,9 @@ paths = [
'''(?i).*\.example\.(yml|yaml|env|json|toml)$''',
'''(?i).*vault\.example\.(yml|yaml)$''',
'''(?i).*\.env\.example$''',
'''(?i)docs-site/docs/extractors/.*''',
'''(?i)extractors/golangjobs/.*''',
'''(?i)orchestrator/src/server/infra/product-analytics\.ts$''',
]
regexes = [
'''(?i)(invalid|fake|dummy|placeholder|example|changeme|change_me|not-a-real)''',
'''(?i)sk-or-invalid''',
'''(?i)msk-or-invalid''',
'''(?i)YOUR_API_KEY''',
]
-8
View File
@@ -55,14 +55,6 @@ git add -A && git commit -m "Your message" && git push gitea main
5. Open the UI: `http://<VM-IP>:3005` (port mapped in `docker-compose.yml`).
**Local machine (loopback only, avoids binding `0.0.0.0:3005`):**
```bash
docker compose -f docker-compose.yml -f docker-compose.localhost.yml up -d --build
```
UI: `http://127.0.0.1:13005`
6. Persist data: compose mounts `./data` — back up that directory.
---
+1 -2
View File
@@ -12,9 +12,8 @@
"!!**/.venv",
"!!docs-site/.docusaurus",
"!!docs-site/build",
"!!extractors/*/storage",
"!!extractors/jobspy/storage",
"!!orchestrator/storage",
"!!storage",
"!!data"
]
},
-6
View File
@@ -1,6 +0,0 @@
# Local-only overlay: single loopback port (does not publish 0.0.0.0:3005).
# Usage: docker compose -f docker-compose.yml -f docker-compose.localhost.yml up -d --build
services:
job-ops:
ports: !override
- "127.0.0.1:13005:3001"
+2 -3
View File
@@ -32,11 +32,10 @@ Eluta surfaces Canadian roles indexed directly from employer career sites, often
### Empty feeds
- The `location` string may be too broad or spelled differently than Eluta expects. Try a major city plus province (e.g. `Calgary, AB`).
- Eluta's RSS is a small recent slice of *all* jobs for that location (often retail/hospitality-heavy). Pipeline search terms like `software engineer` can legitimately match zero items even when the feed itself is healthy. Broaden terms or add more metro locations.
### RSS HTTP errors / "Too Many Requests"
### RSS HTTP errors
- Eluta rate-limits aggressive or browser-like clients. The extractor uses a dedicated `JobOps/... Eluta RSS consumer` User-Agent; retry later or reduce the number of location feeds per run.
- Eluta may block unusual clients; the extractor sends a conventional User-Agent. Retry later or reduce the number of location feeds per run.
## Related pages
-107
View File
@@ -1,107 +0,0 @@
---
id: google-jobs
title: Google Jobs Extractor
description: Browser-backed scraping of the Google for Jobs search widget.
sidebar_position: 8
---
## What it is
Original feature: [Google for Jobs](https://www.google.com/search?q=jobs&udm=8)
Google Jobs is a browser-backed extractor that opens Google's Jobs vertical (`udm=8`) in Firefox (Camoufox when available) for each pipeline search term, scrolls the results panel to load additional cards, then clicks into each card to resolve a real "Apply on &lt;site&gt;" link and description before mapping rows into the orchestrator `CreateJobInput` shape.
Implementation split:
1. `extractors/google-jobs/src/main.ts` builds the search URL, waits for the jobs list, scrolls to lazy-load more cards, extracts card metadata (title/employer/location/"via" source), clicks each card to read the detail pane, and writes dataset JSON.
2. `extractors/google-jobs/src/run.ts` spawns the browser subprocess, applies a hard timeout, reads the dataset, and maps rows to `CreateJobInput`.
3. `extractors/google-jobs/manifest.ts` wires pipeline `searchTerms` into the extractor.
### A note on the search URL and location
Google's Jobs vertical is reached with `udm=8` (its current internal vertical id). An older, commonly-documented deep link — `ibp=htl;jobs` — still works but gets silently redirected to `udm=8` anyway, and combining it with extra parameters (a `num=` result-count hint, or a `gl=` country override that doesn't match the browser's real/proxied IP) has been observed to make Google return "no matches" or "can't determine location" even though the tab itself loads correctly.
Because of that, this extractor does **not** send a `gl` (country) parameter — location is resolved by Google from the browser's real IP (or `GOOGLE_JOBS_PROXY`'s exit IP), which Camoufox's `geoip` option keeps internally consistent. To target a different country's jobs, use a residential proxy with an exit node in that country rather than trying to override the country via a parameter.
### Dealing with Google's CAPTCHA / bot detection
Google actively fingerprints and challenges automated browser traffic, especially from datacenter/VPS IPs (which most servers running JobOps' cron pipeline will have). The extractor has layered mitigations, from always-on to opt-in:
1. **Always on:** Camoufox launches with `humanize: true` and `geoip: true` (human-like cursor movement, IP-consistent locale/geolocation) to reduce — not eliminate — fingerprint-based detection.
2. **Always on:** browser cookies/session are persisted to `extractors/google-jobs/storage/state.json` and reused on the next run. A session that has cleared a CAPTCHA once (including via option 4 below) tends to see fewer challenges for a while afterward.
3. **Opt-in, most reliable:** set `GOOGLE_JOBS_PROXY` (e.g. a residential/mobile proxy URL) to avoid datacenter-IP blocks in the first place. This is the standard real-world fix; unproxied runs from cloud servers should expect to be blocked often.
4. **Opt-in, manual/local only:** set `GOOGLE_JOBS_HEADLESS=false` and run the extractor directly (not via the pipeline) to open a real, visible Firefox window. If Google shows a CAPTCHA, solve it yourself in that window — the script polls and automatically resumes once it clears (up to `GOOGLE_JOBS_UNBLOCK_TIMEOUT_MS`, default 5 minutes). The resulting session cookies are saved and reused by later automated/headless runs. **Do not** set `GOOGLE_JOBS_HEADLESS=false` in the server's always-on `.env` — a headless server has no display for a human to interact with, and the run will just wait until the timeout.
## Why it exists
Google for Jobs aggregates postings across the open web — including boards and employer career pages that JobOps does not otherwise scrape. Adding it as a proper source closes some of the gap between "jobs found by manually Googling a role" and "jobs discovered by the pipeline."
It is **not** a general web crawler and does not replace dedicated extractors: it only surfaces whatever Google's Jobs widget chooses to show for a query, subject to Google's own ranking, freshness, and regional availability.
## How to use it
1. Open **Run jobs** and choose **Automatic**.
2. **Google Jobs** is enabled by default in **Sources** (toggle it off if you do not want it for this run).
3. Set your existing automatic run knobs:
- `searchTerms` become the `"<term> jobs"` query sent to Google per term.
- `googleJobsMaxJobsPerTerm` (Settings) caps jobs scraped per run (default `30`, max `150`).
- Location is derived from the browser's real/proxied IP, not from a country setting — see [the note above](#a-note-on-the-search-url-and-location).
4. Start the run and watch progress in the pipeline progress card.
Local run example (headless, no proxy):
```bash
GOOGLE_JOBS_SEARCH_TERMS='["automation engineer"]' \
GOOGLE_JOBS_MAX_JOBS='10' \
npm --workspace google-jobs-extractor run start
```
Local run example (headed, solve the CAPTCHA yourself if one appears):
```bash
GOOGLE_JOBS_HEADLESS=false \
GOOGLE_JOBS_SEARCH_TERMS='["automation engineer"]' \
npm --workspace google-jobs-extractor run start
```
Defaults and constraints:
- No credentials required; this is a direct HTML/DOM scrape of Google's public search results, not an official API (Google does not offer a free Google Jobs API).
- Google's markup and CSS class names change frequently and are obfuscated; the scraper favors known current card classes (e.g. `.EimVGf`) with multiple fallback selectors and a leaf-node text heuristic, but selectors can still rot and return zero results until updated.
- Every card carries Google's own `data-share-url` permalink for that specific listing, which the extractor uses as `jobUrl` — so a job stays unique and reviewable even if the "click into the card for a detail pane" step below fails for it.
- Because resolving a real application link requires clicking each card in a real browser, this extractor is slower per job than API-backed sources — keep `googleJobsMaxJobsPerTerm` modest for frequent runs.
- If no external "Apply on &lt;site&gt;" link is found for a card, the extractor falls back to Google's own permalink for that listing so it's still reviewable, but not directly one-click-applyable.
- After scraping, JobOps probes each apply URL (preferring the external link over Google's share permalink). Listings that return **404/410** or pages that say the job expired / is no longer available are **dropped before import**. Network failures fail-open (job is kept).
- Automated, frequent, or high-volume scraping of Google search results may be against Google's Terms of Service; treat this extractor as best-effort and keep run frequency/volume conservative.
- See [Dealing with Google's CAPTCHA / bot detection](#dealing-with-googles-captcha--bot-detection) above for proxy and interactive-solve options.
## Common problems
### Google Jobs returns 0 jobs for a term
The extractor logs one of two distinct messages to help tell these apart:
- `Google's Jobs tab reported no matches for "<term>"` — Google correctly landed on the Jobs tab but genuinely has no results for that query/locale. Try a broader, more common phrasing (e.g. `"automation engineer"` instead of a very narrow title).
- `Jobs list did not appear for "<term>"` — an unrecognized page state (consent wall, layout change, or rotted selectors). The extractor saves a screenshot and full HTML dump to `extractors/google-jobs/storage/debug-last.png` / `debug-last.html` for inspection; compare it against a fresh manual search to see what changed.
### Results look sparse or missing company/location
- The list-view heuristics rely on leaf DOM nodes near the job title; if Google restructures the card markup, company/location extraction can degrade even when titles still work. This is expected best-effort behavior for an unofficial scrape.
### Run is slow
- Each job requires a real click + detail-pane render, unlike single-request API extractors. Lower `googleJobsMaxJobsPerTerm` or reduce the number of search terms for faster runs.
### Blocked, CAPTCHA'd, or consistently empty results
- Google may rate-limit or challenge automated browser traffic, especially from datacenter/VPS IPs. In order of effort:
1. Set `GOOGLE_JOBS_PROXY` to a residential/mobile proxy — the most reliable fix for unattended/server runs.
2. Run locally with `GOOGLE_JOBS_HEADLESS=false` and solve the CAPTCHA yourself once; the saved session (`storage/state.json`) may reduce blocks on subsequent headless runs for a while.
3. Reduce run frequency, lower the per-term cap, or disable the source for that run and rely on other extractors / Manual Import instead.
## Related pages
- [Extractors Overview](/docs/next/extractors/overview)
- [Manual Import](/docs/next/extractors/manual)
- [Pipeline Run](/docs/next/features/pipeline-run)
- [Settings](/docs/next/features/settings)
-8
View File
@@ -46,11 +46,3 @@ Set `JOBOPS_EMIT_PROGRESS=1` for structured progress lines consumable by orchest
- Uses Playwright + Crawlee via Camoufox.
- Low concurrency and longer timeouts for stability.
- Employer logo (`figure img` alt text) and employer link are optional. Cards that omit them used to throw a Playwright `getAttribute` timeout and fail the whole region page (0 jobs saved even after cards were counted). Missing employer fields are now skipped per card.
## Common problems
### Region pages time out / run returns 0 jobs
- Gradcracker Livewire markup sometimes omits the employer logo on a card. Older builds aborted the whole page on that timeout. Current builds continue and leave employer blank.
- Anti-bot / Camoufox issues: ensure Camoufox is installed (`npx camoufox-js fetch`) and prefer headed runs when debugging.
-45
View File
@@ -1,45 +0,0 @@
---
id: himalayas
title: Himalayas Extractor
description: Public remote-jobs API pagination and client-side term filtering.
sidebar_position: 16
---
## What it is
Original site: [himalayas.app](https://himalayas.app)
The extractor lives in `extractors/himalayas/manifest.ts`. It paginates the public JSON API (`https://himalayas.app/jobs/api?limit=&offset=`), filters rows client-side by pipeline search terms (title + categories), and maps matches into `CreateJobInput`.
## Why it exists
Himalayas is a large remote-jobs index with a stable, unauthenticated API — useful for remote-first discovery without browser automation or credentials.
## How to use it
1. Enable **Himalayas** in pipeline sources.
2. Optionally set **Himalayas max jobs per term** (`himalayasMaxJobsPerTerm`, default `100`).
3. Pipeline `searchTerms` filter titles/categories client-side (substring match, case-insensitive).
4. Run the pipeline.
### Defaults and constraints
- The upstream API **silently caps `limit` at 20** per request. The extractor uses `PAGE_SIZE = 20` and paginates with `offset` (up to 10 pages / 200 rows scanned per run before the per-term cap).
- There is no server-side search — if a term is rare, early pages may not contain matches even though later pages do. Raise the page budget only by changing the extractor constants if you need deeper scans.
- No auth required.
## Common problems
### Zero jobs for a term I know exists
- The term may not appear in the first ~200 API rows (newest-first feed). Try a shorter token (`software`, `engineer`, `SDET`) or accept that Himalayas skews toward whatever is currently at the top of the feed.
- Confirm you are not hitting an old build that requested `limit=50` and stopped after one page (fixed: the API returns at most 20, which used to look like end-of-feed).
### Rate limits / HTTP errors
- Retry later; the public API is generally stable but can throttle burst traffic.
## Related pages
- [Extractors Overview](/docs/next/extractors/overview)
- [Add an Extractor](/docs/next/workflows/add-an-extractor)
+2 -8
View File
@@ -11,11 +11,11 @@ Original website: [hiring.cafe](https://hiring.cafe)
Special thanks: Initial implementation inspiration came from [umur957/hiring-cafe-job-scraper](https://github.com/umur957/hiring-cafe-job-scraper).
Hiring Cafe is a browser-backed extractor that loads the site in Firefox (Camoufox when available), then fetches paginated search results through Hiring Cafe's Next.js SSR data endpoint and maps rows into the orchestrator `CreateJobInput` shape.
Hiring Cafe is a browser-backed extractor that queries Hiring Cafe search APIs and maps results into the orchestrator `CreateJobInput` shape.
Implementation split:
1. `extractors/hiringcafe/src/main.ts` builds search state, reads the page `buildId`, calls `/_next/data/{buildId}/index.json?searchState=...&page=...`, and writes dataset JSON.
1. `extractors/hiringcafe/src/main.ts` builds search state, calls Hiring Cafe APIs, and writes dataset JSON.
2. `orchestrator/src/server/services/hiring-cafe.ts` runs the extractor, streams progress events, and maps rows for pipeline import.
## Why it exists
@@ -57,12 +57,6 @@ npm --workspace hiringcafe-extractor run start
## Common problems
### Hiring Cafe returns 401 Unauthorized or 404 on `/api/search-jobs`
- Hiring Cafe removed or locked down the legacy GET `/api/search-jobs` and `/api/search-jobs/get-total-count` endpoints.
- Current JobOps builds use the Next.js SSR transport (`ssrHits`, `ssrTotalCount`, `ssrIsLastPage`) instead.
- Upgrade to a build that includes this change if logs still show `401` / `404` on the old API paths.
### Hiring Cafe returns 429 / Vercel security checkpoint
- The extractor first attempts Camoufox-backed Firefox and falls back to vanilla Firefox startup if Camoufox is unstable locally.
+1 -5
View File
@@ -61,8 +61,4 @@ The service in `orchestrator/src/server/services/jobspy.ts`:
- A run returned fewer LinkedIn descriptions than expected.
`JOBSPY_LINKEDIN_FETCH_DESCRIPTION=0` disables description fetching to speed up runs.
- Different cities need different workplace-type filters.
This is not supported in the current automatic-run flow. JobSpy receives one global workplace-type selection per run/query invocation.
- Glassdoor always returns 0 jobs (fixed).
Upstream `python-jobspy` (through at least `1.1.82`) discards an entire page of Glassdoor results whenever Glassdoor's GraphQL response includes *any* `errors` entry — even for an unrelated field. In practice, Glassdoor's backend consistently returns a partial error for `jobsPageSeoData` (`SUBREQUEST_HTTP_ERROR` / "dns error" on Glassdoor's own infra) while the actual job listings still come back fine, so every Glassdoor run silently returned 0 jobs. `extractors/jobspy/scrape_jobs.py` monkeypatches `Glassdoor._fetch_jobs_page` at import time to only treat the response as failed when `data.jobListings` is actually missing, and logs a warning (not an error) when it ignores an unrelated partial error. `requirements.txt` pins `python-jobspy==1.1.82` so an upstream version bump doesn't silently drop this patch — re-verify the patched method still matches before removing the pin.
- A Glassdoor run logs `429` / "Blocked by Glassdoor for too many requests".
This is a real rate limit from running too many Glassdoor requests back-to-back (e.g. repeated manual test runs). Space out runs; the automatic pipeline's normal cadence does not trigger this in practice.
This is not supported in the current automatic-run flow. JobSpy receives one global workplace-type selection per run/query invocation.
+1 -5
View File
@@ -16,7 +16,7 @@ Extractor integrations are now registered through manifests and loaded automatic
| [Gradcracker](/docs/next/extractors/gradcracker) | UK graduate roles from Gradcracker | Crawling stability depends on page structure and anti-bot behavior; tuned for low concurrency | `GRADCRACKER_SEARCH_TERMS`, `GRADCRACKER_MAX_JOBS_PER_TERM`, `JOBOPS_SKIP_APPLY_FOR_EXISTING` | Scrapes listing metadata, then detail pages and apply URL resolution |
| [JobSpy](/docs/next/extractors/jobspy) | Multi-source discovery (Indeed, LinkedIn, Glassdoor) | Requires Python wrapper execution per term; source availability and quality vary by site/location | `JOBSPY_SITES`, `JOBSPY_SEARCH_TERMS`, `JOBSPY_RESULTS_WANTED`, `JOBSPY_HOURS_OLD`, `JOBSPY_LINKEDIN_FETCH_DESCRIPTION` | Produces JSON per term, then orchestrator normalizes and de-duplicates by `jobUrl` |
| [Adzuna](/docs/next/extractors/adzuna) | API-based multi-country discovery with low scraping overhead | Requires valid App ID/App Key; country must be in Adzuna-supported list | `ADZUNA_APP_ID`, `ADZUNA_APP_KEY`, `ADZUNA_MAX_JOBS_PER_TERM` | API pagination to dataset output; orchestrator maps progress and de-duplicates by `sourceJobId`/`jobUrl` |
| [Hiring Cafe](/docs/next/extractors/hiring-cafe) | Browser-backed discovery via Hiring Cafe Next.js SSR search payload | Subject to upstream anti-bot checks; uses browser context and JSON `searchState` query params | `HIRING_CAFE_SEARCH_TERMS`, `HIRING_CAFE_COUNTRY`, `HIRING_CAFE_MAX_JOBS_PER_TERM`, `HIRING_CAFE_DATE_FETCHED_PAST_N_DAYS` | Uses existing pipeline term/country/budget knobs and maps directly to normalized jobs |
| [Hiring Cafe](/docs/next/extractors/hiring-cafe) | Browser-backed discovery using Hiring Cafe search APIs | Subject to upstream anti-bot checks; uses browser context and encoded search-state payloads | `HIRING_CAFE_SEARCH_TERMS`, `HIRING_CAFE_COUNTRY`, `HIRING_CAFE_MAX_JOBS_PER_TERM`, `HIRING_CAFE_DATE_FETCHED_PAST_N_DAYS` | Uses existing pipeline term/country/budget knobs and maps directly to normalized jobs |
| [startup.jobs](/docs/next/extractors/startup-jobs) | Startup-focused discovery through the published `startup-jobs-scraper` package | No credentials required; detail enrichment depends on Playwright browser binaries being installed | existing pipeline `searchTerms`, selected country/cities, `jobspyResultsWanted`; `npx playwright install` for fresh environments | Algolia-backed search plus detail-page enrichment via package import; orchestrator maps normalized records and de-duplicates by `jobUrl` |
| [UKVisaJobs](/docs/next/extractors/ukvisajobs) | UK visa sponsorship-focused roles | Requires authenticated session and periodic token/cookie refresh | `UKVISAJOBS_EMAIL`, `UKVISAJOBS_PASSWORD`, `UKVISAJOBS_MAX_JOBS`, `UKVISAJOBS_SEARCH_KEYWORD` | API pagination + dataset output; orchestrator de-dupes and may fetch missing descriptions |
| [SmartRecruiters](/docs/next/extractors/smartrecruiters) | Enterprise employers on SmartRecruiters public boards | No auth; needs configured company identifiers; one HTTP round-trip per posting for apply URLs + descriptions | `SMARTRECRUITERS_COMPANIES`, `SMARTRECRUITERS_MAX_JOBS_PER_COMPANY` | Paginates the public Posting API, filters by pipeline terms, normalizes to `CreateJobInput` |
@@ -25,7 +25,6 @@ Extractor integrations are now registered through manifests and loaded automatic
| [Eluta](/docs/next/extractors/eluta) | Canadian listings aggregated from employer career sites (RSS) | Canada-only source (skipped when search geography is not Canada); RSS `location` strings must be set | `ELUTA_RSS_LOCATIONS`, `ELUTA_MAX_JOBS_PER_TERM` | Fetches one or more `eluta.ca` RSS feeds, filters by terms, de-duplicates by guid/URL |
| [QAJobsBoard](/docs/next/extractors/qajobsboard) | QA / SDET / automation-heavy board (global JSON feed) | No auth; geography skew is manual/filter downstream | `qajobsboardMaxJobsPerTerm` | Fetches JobBoardly JSON, filters by pipeline terms |
| [Arc.dev](/docs/next/extractors/arcdev) | Remote roles from Arc.dev listing pages (tool-tagged paths) | Parses SSR `__NEXT_DATA__`; relies on stable Next payload | `ARC_REMOTE_JOBS_PATHS` (seeds defaults), `arcRemoteJobsPaths`, `arcMaxJobsPerPath` | Merges Arc-managed + external rows; dedupes by URL |
| [Google Jobs](/docs/next/extractors/google-jobs) | Broader open-web coverage via the Google for Jobs widget | Unofficial scrape (no API); CAPTCHA-prone from datacenter IPs; selectors can rot; slower (click-through per job) | `googleJobsMaxJobsPerTerm`, `GOOGLE_JOBS_PROXY`, `GOOGLE_JOBS_HEADLESS` | Browser scrape of `udm=8` Jobs vertical; scrolls list, double-clicks cards, expands description |
| [Manual Import](/docs/next/extractors/manual) | One-off jobs not covered by scrapers | Inference quality depends on model/provider and input quality; some URLs cannot be fetched reliably | App/API endpoints (`/api/manual-jobs/infer`, `/api/manual-jobs/import`) | Accepts text/HTML/URL, runs inference, then saves and scores job after review |
## Which extractor should I use?
@@ -41,7 +40,6 @@ Extractor integrations are now registered through manifests and loaded automatic
- Use **BC T-Net** for British Columbia tech RSS listings (runs only when search geography is Canada).
- Use **Eluta** for Canadian employer-direct listings via RSS (set metro/province `location` strings).
- Use **QAJobsBoard** or **Arc.dev** when you want QA- or remote-stack-focused feeds without extra credentials.
- Use **Google Jobs** when you want broader open-web coverage beyond dedicated job boards (unofficial scrape; best-effort).
- Use **Manual Import** when you already have a specific posting and need direct import.
Many runs combine sources: broad discovery first, then manual import for high-priority jobs that scraping misses.
@@ -84,7 +82,5 @@ JobOps ships **BC T-Net** and **iCIMS tenant HTML** extractors for two cases tha
- [Arc.dev](/docs/next/extractors/arcdev)
- [Canadian / NA QA contracting firms](/docs/next/extractors/qa-contract-staffing-canada)
- [Canadian companies — QA-strong ATS](/docs/next/extractors/canadian-companies-qa-ats)
- [Google Jobs](/docs/next/extractors/google-jobs)
- [Himalayas](/docs/next/extractors/himalayas)
- [Manual Import](/docs/next/extractors/manual)
- [Add an Extractor](/docs/next/workflows/add-an-extractor)
+2 -22
View File
@@ -45,31 +45,11 @@ During a pipeline run, if a new posting matches an existing row by URL, source i
Open jobs that match a prior skip or apply are **hidden** from Discovered, Ready, and All tabs so the queue stays fresh. Skipped and applied rows themselves remain visible in their statuses.
Use **Filters → Employer keywords → Hide roles you already skipped** to toggle this behavior (on by default). When enabled, JobOps compares **company + job title** only:
- Same company after normalization (legal suffixes stripped; short names like `CGI` match longer forms like `CGI IT UK Limited`).
- Same title after normalization (including simple plural variants such as `Engineer` vs `Engineers`).
- Prior skip or apply must be within the **last 90 days** (`updatedAt` on the skipped/applied row).
Uncheck the filter to temporarily review rediscovered rows that match older skips.
### Clean up rediscovered rows already in Discovered
If older pipeline runs left Discovered copies of roles you already skipped or applied:
1. Open **Settings → Danger Zone**.
2. Use **Clear Rediscovered Archive Matches**.
3. Confirm. JobOps permanently deletes matching **Discovered** rows only (Ready / Applied / Skipped stay).
Matching uses the same employer+title (and employer+description) keys as skip/import dedup.
## Defaults and constraints
- Prior-skip hiding uses employer + title only (not description text).
- The 90-day window applies to both the Jobs list filter and pipeline import suppression for skipped/applied rows.
- Description matching during import still requires at least **80 characters** of normalized text; short or empty descriptions fall back to employer+title only.
- Description matching requires at least **80 characters** of normalized text; short or empty descriptions fall back to employer+title only.
- Matching is **per profile** (`ownerProfileId`); different login profiles do not share dedup state.
- Dedup does **not** delete existing rows retroactively when you change skip list or country filters — use **Clear Rediscovered Archive Matches** in Danger Zone, run discovery again, or skip manually for old data.
- Dedup does **not** delete existing rows retroactively when you change skip list or country filters — run discovery again or skip manually for old data.
- Very different titles at the same company (for example `SDET` vs `Product Designer`) are **not** collapsed.
## Common problems
-16
View File
@@ -153,14 +153,6 @@ Readiness requires:
- Adzuna app ID/app key
- Optional basic authentication for write operations
### Job search profiles and local resume
- **Search profiles** store target roles, deal-breakers, locations, and scoring context. Use the profile picker in Settings to create, switch, or activate a profile.
- When **Basic Auth** is enabled, each login user maps to profiles whose `basicAuthUser` field lists that username. Comma-separated aliases are supported (for example `ilia,dobkin` accepts either login name).
- Jobs are scoped per profile (`owner_profile_id`). Pipeline runs and the orchestrator board only show jobs for the profile tied to your current login.
- **Local resume** JSON paths can be set per profile via `resumeLocalPath` in profile data (for example `../data/resumes/ilia-dobkin.json`). JobOps resolves that path relative to the orchestrator working directory when `JOBOPS_LOCAL_RESUME_PATH` is unset.
- Activating a profile on Settings calls `POST /api/profiles/:id/activate` and updates `activeProfileId`, `jobSearchProfile`, and `localResumeProfilePath` when Basic Auth is off.
### Backup
![Backup settings section](/img/features/settings-backup-section.png)
@@ -186,8 +178,6 @@ Readiness requires:
![Danger Zone settings section](/img/features/settings-danger-zone-section.png)
- Clear jobs by selected statuses
- Clear rediscovered archive matches (Discovered jobs that match skipped/applied roles)
- Mark expired listings (probe Discovered URLs; mark 404/410 / “job expired” as expired)
- Clear jobs below a score threshold
- Clear the full database
@@ -270,12 +260,6 @@ curl -X POST "http://localhost:3001/api/backups"
- Ensure `JOBOPS_PUBLIC_BASE_URL` is set for background/pipeline usage.
- Ensure the configured host is publicly reachable and `/health` responds.
### Profile activate returns 403 or jobs list is empty after login
- Confirm your Basic Auth username appears in the profile's `basicAuthUser` list (comma-separated aliases are allowed).
- If you previously ran pipelines before per-profile ownership existed, older jobs may still be under `__default__`. Re-run discovery while logged in, or migrate rows in `jobs.owner_profile_id` to your profile id.
- Ensure the profile has `resumeLocalPath` pointing at an on-disk resume JSON if you rely on per-profile resumes without `JOBOPS_LOCAL_RESUME_PATH`.
## Related pages
- [Company skip list](./company-skip-list)
-189
View File
@@ -1,189 +0,0 @@
/**
* Built In — tech job board (JSON-LD ItemList embedded in SSR HTML).
*
* https://builtin.com/jobs
*/
import type {
ExtractorManifest,
ExtractorRunResult,
} from "@shared/types/extractors";
import type { CreateJobInput } from "@shared/types/jobs";
const ORIGIN = "https://builtin.com";
interface BuiltinListItem {
title: string;
url: string;
description?: string;
}
function asString(value: unknown): string | undefined {
if (typeof value !== "string") return undefined;
const trimmed = value.trim();
return trimmed ? trimmed : undefined;
}
function decodeJsonString(value: string): string {
return value
.replace(/\\"/g, '"')
.replace(/\\n/g, "\n")
.replace(/\\t/g, "\t")
.replace(/\\\\/g, "\\");
}
function readMaxPages(raw: string | undefined): number {
const parsed = raw ? Number.parseInt(raw, 10) : Number.NaN;
if (!Number.isFinite(parsed)) return 3;
return Math.min(Math.max(parsed, 1), 15);
}
function parseListItems(html: string): BuiltinListItem[] {
const items: BuiltinListItem[] = [];
const pattern =
/\{"@type":"ListItem","position":\d+,"name":"((?:\\.|[^"\\])*)","url":"(https:\/\/builtin\.com\/job\/[^"]+)"(?:,"description":"((?:\\.|[^"\\])*)")?\}/g;
for (const match of html.matchAll(pattern)) {
const title = decodeJsonString(match[1] ?? "");
const url = match[2];
if (!title || !url) continue;
items.push({
title,
url,
description: match[3] ? decodeJsonString(match[3]) : undefined,
});
}
return items;
}
function slugToWords(slug: string): string {
return slug
.split("-")
.filter(Boolean)
.map((part) => part.charAt(0).toUpperCase() + part.slice(1))
.join(" ");
}
function employerFromUrl(jobUrl: string): string {
const match = jobUrl.match(/\/job\/[^/]+\/(\d+)/);
if (!match) return "Unknown Employer";
const slugMatch = jobUrl.match(/\/job\/([^/]+)\/\d+/);
const slug = slugMatch?.[1] ?? "";
const parts = slug.split("-");
const maybeId = parts[parts.length - 1];
if (/^\d+$/.test(maybeId ?? "")) parts.pop();
const trimmed = parts.slice(-2);
if (trimmed.length === 0) return slugToWords(slug);
return slugToWords(trimmed.join("-"));
}
function searchPath(term: string | null): string {
if (!term) return "/jobs/remote";
const query = encodeURIComponent(term.trim());
return `/jobs/remote?search=${query}`;
}
function matchesTerm(item: BuiltinListItem, term: string): boolean {
const lower = term.toLowerCase();
if (item.title.toLowerCase().includes(lower)) return true;
if (item.description?.toLowerCase().includes(lower)) return true;
return false;
}
async function fetchPage(path: string, page: number): Promise<string> {
const separator = path.includes("?") ? "&" : "?";
const url =
page <= 1 ? `${ORIGIN}${path}` : `${ORIGIN}${path}${separator}page=${page}`;
const response = await fetch(url, {
headers: { Accept: "text/html", "User-Agent": "JobOps/1.0" },
});
if (!response.ok) {
throw new Error(`Built In request failed (${response.status}) for ${url}`);
}
return response.text();
}
export const manifest: ExtractorManifest = {
id: "builtin",
displayName: "Built In",
providesSources: ["builtin"],
async run(context): Promise<ExtractorRunResult> {
if (context.shouldCancel?.()) return { success: true, jobs: [] };
const maxJobs = context.settings.builtinMaxJobsPerTerm
? Number.parseInt(context.settings.builtinMaxJobsPerTerm, 10)
: 100;
const cap = Number.isFinite(maxJobs)
? Math.min(Math.max(maxJobs, 1), 500)
: 100;
const maxPages = readMaxPages(context.settings.builtinMaxPagesPerTerm);
const terms = context.searchTerms.length > 0 ? context.searchTerms : [null];
const seen = new Set<string>();
const out: CreateJobInput[] = [];
try {
for (let termIndex = 0; termIndex < terms.length; termIndex += 1) {
if (context.shouldCancel?.()) break;
const term = terms[termIndex];
const path = searchPath(term);
for (let page = 1; page <= maxPages; page += 1) {
if (context.shouldCancel?.()) break;
if (out.length >= cap) break;
context.onProgress?.({
phase: "list",
termsProcessed: termIndex,
termsTotal: terms.length,
currentUrl: `${ORIGIN}${path}`,
detail: `Built In: term ${termIndex + 1}/${terms.length}, page ${page}`,
});
const html = await fetchPage(path, page);
const items = parseListItems(html);
if (items.length === 0) break;
for (const item of items) {
if (out.length >= cap) break;
if (term && !matchesTerm(item, term)) continue;
const key = item.url;
if (seen.has(key)) continue;
seen.add(key);
out.push({
source: "builtin",
sourceJobId: item.url.split("/").pop(),
title: item.title,
employer: employerFromUrl(item.url),
jobUrl: item.url,
applicationLink: item.url,
location: "Remote",
isRemote: true,
jobDescription: item.description,
});
}
}
}
context.onProgress?.({
phase: "list",
termsProcessed: terms.length,
termsTotal: terms.length,
currentUrl: `${ORIGIN}/jobs`,
jobPagesProcessed: out.length,
detail: `Built In: ${out.length} jobs`,
});
return { success: true, jobs: out };
} catch (error) {
const message = error instanceof Error ? error.message : "Unknown error";
return { success: false, jobs: out, error: message };
}
},
};
export default manifest;
-17
View File
@@ -1,17 +0,0 @@
{
"name": "builtin-extractor",
"version": "0.0.1",
"type": "module",
"description": "Built In tech job board extractor",
"main": "manifest.ts",
"dependencies": {
"job-ops-shared": "^1.0.0"
},
"devDependencies": {
"@types/node": "^24.0.0",
"typescript": "~5.9.0"
},
"scripts": {
"check:types": "tsc --noEmit"
}
}
-17
View File
@@ -1,17 +0,0 @@
{
"compilerOptions": {
"module": "ESNext",
"moduleResolution": "bundler",
"target": "ES2022",
"outDir": "dist",
"strict": true,
"noUnusedLocals": false,
"lib": ["ES2022", "DOM"],
"types": ["node"],
"baseUrl": ".",
"paths": {
"@shared/*": ["../../shared/src/*"]
}
},
"include": ["./manifest.ts", "./src/**/*"]
}
-89
View File
@@ -1,89 +0,0 @@
/**
* Direct company career pages — auto-detect ATS backends and pull public job feeds.
*/
import type {
ExtractorManifest,
ExtractorRunResult,
} from "@shared/types/extractors";
import type { CreateJobInput } from "@shared/types/jobs";
import {
detectAtsTargets,
fetchCareersPage,
fetchJobsForTarget,
readUrlList,
} from "./src/resolve-ats.js";
function matchesTerm(job: CreateJobInput, term: string): boolean {
const lower = term.toLowerCase();
if (job.title.toLowerCase().includes(lower)) return true;
if (job.employer.toLowerCase().includes(lower)) return true;
if (job.jobDescription?.toLowerCase().includes(lower)) return true;
return false;
}
export const manifest: ExtractorManifest = {
id: "careerspages",
displayName: "Career Pages",
providesSources: ["careerspages"],
async run(context): Promise<ExtractorRunResult> {
if (context.shouldCancel?.()) return { success: true, jobs: [] };
const pageUrls = readUrlList(context.settings.careersPageUrls);
if (pageUrls.length === 0) {
return {
success: true,
jobs: [],
error:
"No career page URLs configured. Set CAREERS_PAGE_URLS or careersPageUrls (JSON array or comma/newline-separated URLs).",
};
}
const terms = context.searchTerms.length > 0 ? context.searchTerms : [];
const seen = new Set<string>();
const out: CreateJobInput[] = [];
try {
for (let i = 0; i < pageUrls.length; i += 1) {
if (context.shouldCancel?.()) break;
const pageUrl = pageUrls[i];
context.onProgress?.({
phase: "list",
termsProcessed: i,
termsTotal: pageUrls.length,
currentUrl: pageUrl,
detail: `Career pages: resolving ATS for ${pageUrl}`,
});
const html = await fetchCareersPage(pageUrl);
const targets = detectAtsTargets(html, pageUrl);
if (targets.length === 0) continue;
for (const target of targets) {
if (context.shouldCancel?.()) break;
const jobs = await fetchJobsForTarget(target);
for (const job of jobs) {
if (
terms.length > 0 &&
!terms.some((term) => matchesTerm(job, term))
) {
continue;
}
const key = job.jobUrl;
if (seen.has(key)) continue;
seen.add(key);
out.push(job);
}
}
}
return { success: true, jobs: out };
} catch (error) {
const message = error instanceof Error ? error.message : "Unknown error";
return { success: false, jobs: out, error: message };
}
},
};
export default manifest;
-17
View File
@@ -1,17 +0,0 @@
{
"name": "careerspages-extractor",
"version": "0.0.1",
"type": "module",
"description": "Direct company career page ATS resolver extractor",
"main": "manifest.ts",
"dependencies": {
"job-ops-shared": "^1.0.0"
},
"devDependencies": {
"@types/node": "^24.0.0",
"typescript": "~5.9.0"
},
"scripts": {
"check:types": "tsc --noEmit"
}
}
-345
View File
@@ -1,345 +0,0 @@
import type { CreateJobInput } from "@shared/types/jobs";
export type AtsKind =
| "ashby"
| "greenhouse"
| "lever"
| "teamtailor"
| "huntflow"
| "factorial";
export interface DetectedAts {
kind: AtsKind;
slug: string;
}
export function readUrlList(raw: string | undefined): string[] {
if (!raw) return [];
try {
const parsed = JSON.parse(raw);
if (Array.isArray(parsed)) {
return parsed
.map((entry) => (typeof entry === "string" ? entry.trim() : ""))
.filter(Boolean);
}
} catch {
// fall through
}
return raw
.split(/[\n,;|]+/)
.map((entry) => entry.trim())
.filter(Boolean);
}
export function detectAtsTargets(html: string, pageUrl: string): DetectedAts[] {
const found: DetectedAts[] = [];
const seen = new Set<string>();
const add = (kind: AtsKind, slug: string) => {
const key = `${kind}:${slug.toLowerCase()}`;
if (!slug || seen.has(key)) return;
seen.add(key);
found.push({ kind, slug });
};
for (const match of html.matchAll(/jobs\.ashbyhq\.com\/([a-zA-Z0-9_-]+)/g)) {
add("ashby", match[1]);
}
for (const match of html.matchAll(
/boards-api\.greenhouse\.io\/v1\/boards\/([a-zA-Z0-9_-]+)/g,
)) {
add("greenhouse", match[1]);
}
for (const match of html.matchAll(
/boards\.greenhouse\.io\/([a-zA-Z0-9_-]+)/g,
)) {
add("greenhouse", match[1]);
}
for (const match of html.matchAll(/jobs\.lever\.co\/([a-zA-Z0-9_-]+)/g)) {
add("lever", match[1]);
}
for (const match of html.matchAll(
/https?:\/\/([a-z0-9-]+)\.teamtailor\.com/gi,
)) {
add("teamtailor", match[1]);
}
for (const match of html.matchAll(
/https?:\/\/([a-z0-9-]+)\.huntflow\.io/gi,
)) {
add("huntflow", match[1]);
}
for (const match of html.matchAll(
/https?:\/\/([a-z0-9-]+)\.factorialhr\.com/gi,
)) {
add("factorial", match[1]);
}
try {
const host = new URL(pageUrl).hostname.toLowerCase();
if (host.endsWith(".huntflow.io")) {
add("huntflow", host.replace(/\.huntflow\.io$/, ""));
}
if (host.endsWith(".factorialhr.com")) {
add("factorial", host.replace(/\.factorialhr\.com$/, ""));
}
if (host.endsWith(".teamtailor.com")) {
add("teamtailor", host.replace(/\.teamtailor\.com$/, ""));
}
} catch {
// ignore invalid pageUrl
}
return found;
}
function asString(value: unknown): string | undefined {
if (typeof value !== "string") return undefined;
const trimmed = value.trim();
return trimmed ? trimmed : undefined;
}
function decodeHtmlEntities(value: string): string {
return value
.replace(/&amp;/g, "&")
.replace(/&lt;/g, "<")
.replace(/&gt;/g, ">")
.replace(/&quot;/g, '"')
.replace(/&#39;/g, "'");
}
function stripHtml(html: string): string {
return decodeHtmlEntities(html)
.replace(/<[^>]+>/g, " ")
.replace(/\s+/g, " ")
.trim();
}
async function fetchAshby(slug: string): Promise<CreateJobInput[]> {
const url = `https://api.ashbyhq.com/posting-api/job-board/${encodeURIComponent(slug)}`;
const response = await fetch(url, {
headers: { Accept: "application/json" },
});
if (!response.ok) return [];
const body = (await response.json()) as {
jobs?: Array<Record<string, unknown>>;
};
return (body.jobs ?? []).flatMap((job) => {
const jobUrl = asString(job.jobUrl);
if (!jobUrl) return [];
return [
{
source: "careerspages" as const,
sourceJobId: asString(job.id),
title: asString(job.title) ?? "Unknown Title",
employer: slug,
jobUrl,
applicationLink: asString(job.applyUrl) ?? jobUrl,
location:
asString(job.location) ?? asString(job.locationName) ?? "Unknown",
isRemote: job.isRemote === true,
datePosted: asString(job.publishedAt),
jobDescription:
asString(job.descriptionPlain) ??
(job.descriptionHtml
? stripHtml(String(job.descriptionHtml))
: undefined),
},
];
});
}
async function fetchGreenhouse(slug: string): Promise<CreateJobInput[]> {
const url = `https://boards-api.greenhouse.io/v1/boards/${encodeURIComponent(slug)}/jobs?content=true`;
const response = await fetch(url, {
headers: { Accept: "application/json" },
});
if (!response.ok) return [];
const body = (await response.json()) as {
jobs?: Array<Record<string, unknown>>;
};
return (body.jobs ?? []).flatMap((job) => {
const jobUrl = asString(job.absolute_url);
if (!jobUrl) return [];
const location = job.location as { name?: string } | undefined;
return [
{
source: "careerspages" as const,
sourceJobId: String(job.id ?? jobUrl),
title: asString(job.title) ?? "Unknown Title",
employer: slug,
jobUrl,
applicationLink: jobUrl,
location: asString(location?.name) ?? "Unknown",
datePosted: asString(job.updated_at),
jobDescription: job.content
? stripHtml(String(job.content))
: undefined,
},
];
});
}
async function fetchLever(slug: string): Promise<CreateJobInput[]> {
const url = `https://api.lever.co/v0/postings/${encodeURIComponent(slug)}?mode=json`;
const response = await fetch(url, {
headers: { Accept: "application/json" },
});
if (!response.ok) return [];
const body = (await response.json()) as Array<Record<string, unknown>>;
if (!Array.isArray(body)) return [];
return body.flatMap((job) => {
const jobUrl = asString(job.hostedUrl) ?? asString(job.applyUrl);
if (!jobUrl) return [];
const categories = job.categories as Record<string, unknown> | undefined;
return [
{
source: "careerspages" as const,
sourceJobId: asString(job.id),
title: asString(job.text) ?? "Unknown Title",
employer: slug,
jobUrl,
applicationLink: asString(job.applyUrl) ?? jobUrl,
location: asString(categories?.location) ?? "Unknown",
datePosted: asString(job.createdAt),
jobDescription: job.descriptionPlain
? String(job.descriptionPlain)
: job.description
? stripHtml(String(job.description))
: undefined,
},
];
});
}
function xmlText(xml: string, tag: string): string | undefined {
const pattern = new RegExp(`<${tag}[^>]*>([\\s\\S]*?)</${tag}>`, "i");
const match = xml.match(pattern);
if (!match?.[1]) return undefined;
return (
match[1].replace(/<!\[CDATA\[([\s\S]*?)\]\]>/g, "$1").trim() || undefined
);
}
async function fetchTeamtailor(slug: string): Promise<CreateJobInput[]> {
const url = `https://${encodeURIComponent(slug)}.teamtailor.com/jobs.rss?per_page=200`;
const response = await fetch(url, {
headers: { Accept: "application/rss+xml, application/xml, text/xml" },
});
if (!response.ok) return [];
const xml = await response.text();
const out: CreateJobInput[] = [];
for (const raw of xml.match(/<item>([\s\S]*?)<\/item>/gi) ?? []) {
const block = raw.replace(/^<item>/i, "").replace(/<\/item>$/i, "");
const jobUrl = xmlText(block, "link");
if (!jobUrl) continue;
out.push({
source: "careerspages",
sourceJobId: xmlText(block, "guid") ?? jobUrl,
title: xmlText(block, "title") ?? "Unknown Title",
employer: xmlText(block, "company_name") ?? slug,
jobUrl,
applicationLink: jobUrl,
location: xmlText(block, "location") ?? "Unknown",
datePosted: xmlText(block, "pubDate"),
jobDescription: xmlText(block, "description")
? stripHtml(xmlText(block, "description") ?? "")
: undefined,
});
}
return out;
}
async function fetchHuntflow(slug: string): Promise<CreateJobInput[]> {
const origin = `https://${slug}.huntflow.io`;
const response = await fetch(`${origin}/`, {
headers: { Accept: "text/html", "User-Agent": "JobOps/1.0" },
});
if (!response.ok) return [];
const html = await response.text();
const out: CreateJobInput[] = [];
const pattern =
/<article class="_item_[^"]*">[\s\S]*?<a href="(\/vacancy\/[^"]+)"[^>]*>([^<]+)<\/a>[\s\S]*?<div class="_info_[^"]*">([^<]*)<\/div>/g;
for (const match of html.matchAll(pattern)) {
const jobUrl = `${origin}${match[1]}`;
out.push({
source: "careerspages",
sourceJobId: match[1]?.replace(/^\/vacancy\//, ""),
title: decodeHtmlEntities(match[2]?.trim() ?? "Unknown Title"),
employer: slug,
jobUrl,
applicationLink: jobUrl,
location: match[3]?.trim() || "Unknown",
isRemote: match[3]?.toLowerCase().includes("remote"),
});
}
return out;
}
async function fetchFactorial(slug: string): Promise<CreateJobInput[]> {
const origin = `https://${slug}.factorialhr.com`;
const response = await fetch(origin, {
headers: { Accept: "text/html", "User-Agent": "JobOps/1.0" },
});
if (!response.ok) return [];
const html = await response.text();
const out: CreateJobInput[] = [];
for (const match of html.matchAll(
/(?:href|data-job-postings-url)=['"]([^'"]*\/job_posting\/[^'"]+)['"]/g,
)) {
const raw = match[1];
const path = raw.startsWith("http")
? raw.replace(origin, "")
: raw.startsWith("/")
? raw
: `/${raw}`;
const jobUrl = `${origin}${path}`;
const slugPart = path.split("/").pop() ?? "";
const title = slugPart
.replace(/-\d+$/, "")
.split("-")
.map((part) => part.charAt(0).toUpperCase() + part.slice(1))
.join(" ");
out.push({
source: "careerspages",
sourceJobId: slugPart,
title,
employer: slug,
jobUrl,
applicationLink: jobUrl,
location: "Unknown",
});
}
return out;
}
export async function fetchJobsForTarget(
target: DetectedAts,
): Promise<CreateJobInput[]> {
switch (target.kind) {
case "ashby":
return fetchAshby(target.slug);
case "greenhouse":
return fetchGreenhouse(target.slug);
case "lever":
return fetchLever(target.slug);
case "teamtailor":
return fetchTeamtailor(target.slug);
case "huntflow":
return fetchHuntflow(target.slug);
case "factorial":
return fetchFactorial(target.slug);
}
}
export async function fetchCareersPage(pageUrl: string): Promise<string> {
const response = await fetch(pageUrl, {
headers: { Accept: "text/html", "User-Agent": "JobOps/1.0" },
redirect: "follow",
});
if (!response.ok) {
throw new Error(
`Careers page request failed (${response.status}) for ${pageUrl}`,
);
}
return response.text();
}
-17
View File
@@ -1,17 +0,0 @@
{
"compilerOptions": {
"module": "ESNext",
"moduleResolution": "bundler",
"target": "ES2022",
"outDir": "dist",
"strict": true,
"noUnusedLocals": false,
"lib": ["ES2022", "DOM"],
"types": ["node"],
"baseUrl": ".",
"paths": {
"@shared/*": ["../../shared/src/*"]
}
},
"include": ["./manifest.ts", "./src/**/*"]
}
-216
View File
@@ -1,216 +0,0 @@
/**
* Factorial HR public career sites — tenant HTML job_posting pages.
*
* https://{tenant}.factorialhr.com/
*/
import type {
ExtractorManifest,
ExtractorRunResult,
} from "@shared/types/extractors";
import type { CreateJobInput } from "@shared/types/jobs";
function asString(value: unknown): string | undefined {
if (typeof value !== "string") return undefined;
const trimmed = value.trim();
return trimmed ? trimmed : undefined;
}
function readTenants(raw: string | undefined): string[] {
if (!raw) return [];
try {
const parsed = JSON.parse(raw);
if (Array.isArray(parsed)) {
return parsed
.map((entry) => (typeof entry === "string" ? entry.trim() : ""))
.filter(Boolean);
}
} catch {
// fall through
}
return raw
.split(/[\n,;|]+/)
.map((entry) => entry.trim())
.filter(Boolean);
}
function decodeHtmlEntities(value: string): string {
return value
.replace(/&amp;/g, "&")
.replace(/&lt;/g, "<")
.replace(/&gt;/g, ">")
.replace(/&quot;/g, '"')
.replace(/&#39;/g, "'");
}
function stripHtml(html: string): string {
return decodeHtmlEntities(html)
.replace(/<[^>]+>/g, " ")
.replace(/\s+/g, " ")
.trim();
}
function parseJobLinks(html: string, origin: string): string[] {
const links = new Set<string>();
for (const match of html.matchAll(
/(?:href|data-job-postings-url)=['"]([^'"]*\/job_posting\/[^'"]+)['"]/g,
)) {
const raw = match[1];
const path = raw.startsWith("http")
? raw.replace(origin, "")
: raw.startsWith("/")
? raw
: `/${raw}`;
links.add(path);
}
return [...links];
}
function parseJobDetail(html: string): {
title?: string;
description?: string;
} {
const titleMatch =
html.match(/property='og:title'[^>]*content='([^']+)'/) ??
html.match(/<h1[^>]*>([^<]+)<\/h1>/);
const bodyMatch = html.match(
/<div class='mb-12'>[\s\S]*?<div class='mb-2 sm:mb-4'>[\s\S]*?<\/h1>([\s\S]*?)<\/div>\s*<\/div>/,
);
return {
title: titleMatch?.[1]?.trim(),
description: bodyMatch?.[1] ? stripHtml(bodyMatch[1]) : undefined,
};
}
function matchesTerm(values: string[], term: string): boolean {
const lower = term.toLowerCase();
return values.some((value) => value.toLowerCase().includes(lower));
}
async function fetchText(url: string): Promise<string> {
const response = await fetch(url, {
headers: { Accept: "text/html", "User-Agent": "JobOps/1.0" },
});
if (!response.ok) {
throw new Error(`Factorial request failed (${response.status}) for ${url}`);
}
return response.text();
}
function tenantOrigin(tenant: string): string {
const host = tenant.includes(".") ? tenant : `${tenant}.factorialhr.com`;
return host.startsWith("http") ? host : `https://${host}`;
}
function employerLabel(tenant: string, pageHtml: string): string {
const ogSiteMatch = pageHtml.match(
/property=['"]og:site_name['"][^>]*content=['"]([^'"]+)['"]/i,
);
if (ogSiteMatch?.[1]) {
const cleaned = ogSiteMatch[1].split(/\s[-|]\s/)[0]?.trim();
if (cleaned) return cleaned;
}
return tenant
.split(".")[0]
.split(/[-_]/)
.map((part) => part.charAt(0).toUpperCase() + part.slice(1))
.join(" ");
}
export const manifest: ExtractorManifest = {
id: "factorial",
displayName: "Factorial (ATS)",
providesSources: ["factorial"],
async run(context): Promise<ExtractorRunResult> {
if (context.shouldCancel?.()) return { success: true, jobs: [] };
const tenants = readTenants(context.settings.factorialTenants);
if (tenants.length === 0) {
return {
success: true,
jobs: [],
error:
"No Factorial tenants configured. Set FACTORIAL_TENANTS or factorialTenants (comma- or newline-separated subdomains, e.g. yourbourse).",
};
}
const terms = context.searchTerms.length > 0 ? context.searchTerms : [];
const seen = new Set<string>();
const out: CreateJobInput[] = [];
try {
for (let i = 0; i < tenants.length; i += 1) {
if (context.shouldCancel?.()) break;
const tenant = tenants[i];
const origin = tenantOrigin(tenant);
context.onProgress?.({
phase: "list",
termsProcessed: i,
termsTotal: tenants.length,
currentUrl: origin,
detail: `Factorial: ${tenant} (${i + 1}/${tenants.length})`,
});
const indexHtml = await fetchText(origin);
const employer = employerLabel(tenant, indexHtml);
const links = parseJobLinks(indexHtml, origin);
for (const link of links) {
if (context.shouldCancel?.()) break;
const jobUrl = `${origin}${link}`;
let title =
link.split("/").pop()?.replace(/-\d+$/, "").replace(/-/g, " ") ??
"Unknown Title";
title = title
.split(" ")
.map((part) => part.charAt(0).toUpperCase() + part.slice(1))
.join(" ");
let jobDescription: string | undefined;
try {
const detailHtml = await fetchText(jobUrl);
const detail = parseJobDetail(detailHtml);
if (detail.title) title = detail.title.replace(/\s+$/, "");
if (detail.description) jobDescription = detail.description;
} catch {
// keep index-derived row when detail fetch fails
}
const haystack = [title, jobDescription ?? ""];
if (
terms.length > 0 &&
!terms.some((term) => matchesTerm(haystack, term))
) {
continue;
}
const key = jobUrl;
if (seen.has(key)) continue;
seen.add(key);
out.push({
source: "factorial",
sourceJobId: link.split("/").pop(),
title,
employer,
jobUrl,
applicationLink: jobUrl,
location: "Unknown",
jobDescription,
});
}
}
return { success: true, jobs: out };
} catch (error) {
const message = error instanceof Error ? error.message : "Unknown error";
return { success: false, jobs: out, error: message };
}
},
};
export default manifest;
-17
View File
@@ -1,17 +0,0 @@
{
"name": "factorial-extractor",
"version": "0.0.1",
"type": "module",
"description": "Factorial HR public career-site extractor",
"main": "manifest.ts",
"dependencies": {
"job-ops-shared": "^1.0.0"
},
"devDependencies": {
"@types/node": "^24.0.0",
"typescript": "~5.9.0"
},
"scripts": {
"check:types": "tsc --noEmit"
}
}
-17
View File
@@ -1,17 +0,0 @@
{
"compilerOptions": {
"module": "ESNext",
"moduleResolution": "bundler",
"target": "ES2022",
"outDir": "dist",
"strict": true,
"noUnusedLocals": false,
"lib": ["ES2022", "DOM"],
"types": ["node"],
"baseUrl": ".",
"paths": {
"@shared/*": ["../../shared/src/*"]
}
},
"include": ["./manifest.ts", "./src/**/*"]
}
-49
View File
@@ -1,49 +0,0 @@
import type {
ExtractorManifest,
ExtractorRunResult,
} from "@shared/types/extractors";
import { runGoogleJobs } from "./src/run.js";
export const manifest: ExtractorManifest = {
id: "google-jobs",
displayName: "Google Jobs",
providesSources: ["google-jobs"],
async run(context): Promise<ExtractorRunResult> {
if (context.shouldCancel?.()) return { success: true, jobs: [] };
const parsedMax = context.settings.googleJobsMaxJobsPerTerm
? Number.parseInt(context.settings.googleJobsMaxJobsPerTerm, 10)
: Number.NaN;
const maxJobs = Number.isFinite(parsedMax) ? Math.max(1, parsedMax) : 30;
context.onProgress?.({
phase: "list",
termsProcessed: 0,
termsTotal: context.searchTerms.length || 1,
currentUrl: "https://www.google.com/search?udm=8",
detail: "Google Jobs: launching browser scrape",
});
const result = await runGoogleJobs({
searchTerms: context.searchTerms,
maxJobs,
});
if (!result.success) {
return { success: false, jobs: [], error: result.error };
}
context.onProgress?.({
phase: "list",
termsProcessed: context.searchTerms.length || 1,
termsTotal: context.searchTerms.length || 1,
currentUrl: "https://www.google.com/search?udm=8",
jobPagesProcessed: result.jobs.length,
detail: `Google Jobs: ${result.jobs.length} jobs`,
});
return { success: true, jobs: result.jobs };
},
};
export default manifest;
-22
View File
@@ -1,22 +0,0 @@
{
"name": "google-jobs-extractor",
"version": "0.0.1",
"type": "module",
"description": "Google Jobs (Google for Jobs) search widget extractor (browser-backed)",
"main": "manifest.ts",
"dependencies": {
"camoufox-js": "^0.8.0",
"job-ops-shared": "^1.0.0",
"playwright": "^1.57.0",
"tsx": "^4.4.0"
},
"devDependencies": {
"@types/node": "^24.0.0",
"typescript": "~5.9.0"
},
"scripts": {
"start": "tsx src/main.ts",
"check:types": "tsc --noEmit",
"get-binaries": "camoufox-js fetch"
}
}
-616
View File
@@ -1,616 +0,0 @@
import { access, mkdir, writeFile } from "node:fs/promises";
import { dirname, join } from "node:path";
import { fileURLToPath } from "node:url";
import { launchOptions } from "camoufox-js";
import {
type Browser,
type BrowserContext,
firefox,
type Page,
} from "playwright";
const __dirname = dirname(fileURLToPath(import.meta.url));
const OUTPUT_PATH = join(__dirname, "../storage/jobs.json");
const STATE_PATH = join(__dirname, "../storage/state.json");
const DEBUG_HTML_PATH = join(__dirname, "../storage/debug-last.html");
const DEBUG_SCREENSHOT_PATH = join(__dirname, "../storage/debug-last.png");
interface ScrapedJob {
title: string;
employer: string;
jobUrl: string;
applicationLink?: string;
location?: string;
jobDescription?: string;
postedVia?: string;
isRemote?: boolean;
}
interface ScrapedCard {
docId?: string;
title: string;
employer: string;
location?: string;
via?: string;
/** Google's own permalink into this specific listing (from `data-share-url`). Always present when the card matched, so it works as a stable jobUrl even if the detail click fails. */
shareUrl?: string;
}
interface ScrapedDetail {
applyLinks: Array<{ href: string; text: string }>;
description?: string;
}
/**
* Confirmed against Google's live markup (2026). Google renders each job as
* a `.EimVGf` card with a `data-share-url` permalink baked in — no `<ul>`/
* `<li>`/`aria-label="Jobs list"` wrapper as older scraping guides assumed.
* Kept as a short list (not a single hardcoded class) since Google rotates
* these obfuscated names periodically.
*/
const CARD_SELECTORS = [".EimVGf", 'li[role="listitem"]', ".iFjolb"] as const;
function parseTerms(raw: string | undefined): string[] {
if (!raw) return ["software engineer"];
try {
const parsed = JSON.parse(raw) as unknown;
if (Array.isArray(parsed)) {
return parsed.map((entry) => String(entry).trim()).filter(Boolean);
}
} catch {
// fall through to delimited parsing
}
return raw
.split(/[\n|,]+/)
.map((entry) => entry.trim())
.filter(Boolean);
}
/**
* `udm=8` is Google's current internal id for the Jobs vertical (the legacy
* `ibp=htl;jobs` deep link now gets silently redirected to it anyway, but
* only when requested directly — combining the two, or adding `num=`/`gl=`
* overrides that don't match the browser's real/proxied IP, has been
* observed to make the vertical come back "no matches" or "can't determine
* location" even though the widget itself loads correctly). `hl=en` keeps
* copy in English so downstream text parsing (e.g. "via LinkedIn") is
* consistent regardless of where the request is geolocated. Location is
* intentionally NOT set via a `gl` param — it's derived from the real (or
* `GOOGLE_JOBS_PROXY`) IP through Camoufox's `geoip` option instead.
*/
function buildSearchUrl(term: string): string {
const params = new URLSearchParams({
q: `${term} jobs`,
udm: "8",
hl: "en",
});
return `https://www.google.com/search?${params.toString()}`;
}
/**
* Launches Camoufox-hardened Firefox. `humanize`/`geoip` reduce (but do not
* eliminate) Google's automated-traffic detection. `GOOGLE_JOBS_PROXY` (e.g.
* a residential proxy URL) is the most reliable way to avoid datacenter-IP
* blocks entirely, if configured.
*/
async function launchBrowser(headless: boolean): Promise<Browser> {
const proxy = process.env.GOOGLE_JOBS_PROXY?.trim();
return firefox.launch(
await launchOptions({
headless,
humanize: true,
geoip: true,
...(proxy ? { proxy } : {}),
}),
);
}
/** Google shows a cookie/consent wall in some locales before rendering results. */
async function dismissConsent(page: Page): Promise<void> {
const labels = ["Reject all", "I agree", "Accept all"];
for (const label of labels) {
try {
const button = page.getByRole("button", { name: label }).first();
if (await button.isVisible({ timeout: 1500 })) {
await button.click({ timeout: 3000 });
await page.waitForTimeout(500);
return;
}
} catch {
// selector not present in this render; try the next label
}
}
}
/** Thrown when Google serves a CAPTCHA / rate-limit challenge instead of results. */
class GoogleBlockedError extends Error {}
/** Google serves an interstitial CAPTCHA page for automated / datacenter traffic. */
async function isBlockedPage(page: Page): Promise<boolean> {
if (/\/sorry\//.test(page.url())) return true;
try {
return await page.evaluate(() => {
const bodyText = document.body?.textContent ?? "";
return (
document.querySelector("#captcha-form") !== null ||
document.querySelector(".g-recaptcha") !== null ||
/unusual traffic/i.test(bodyText)
);
});
} catch {
return false;
}
}
/**
* In headed/interactive mode, give a human at the keyboard a chance to solve
* the CAPTCHA in the visible browser window instead of failing immediately.
* Polls until the challenge clears (Google's form redirects on success) or
* the timeout elapses.
*/
async function waitForHumanToUnblock(
page: Page,
timeoutMs: number,
): Promise<boolean> {
const deadline = Date.now() + timeoutMs;
console.error(
`[google-jobs] Google is showing a CAPTCHA in the open browser window — please solve it there. Waiting up to ${Math.round(timeoutMs / 1000)}s...`,
);
while (Date.now() < deadline) {
await page.waitForTimeout(3_000);
if (!(await isBlockedPage(page))) {
console.error("[google-jobs] CAPTCHA cleared, resuming.");
return true;
}
}
return false;
}
async function waitForJobsList(page: Page): Promise<boolean> {
for (const selector of CARD_SELECTORS) {
try {
await page.waitForSelector(selector, { timeout: 8_000 });
return true;
} catch {
// try the next fallback selector
}
}
return false;
}
/**
* Google sometimes lands correctly on the Jobs vertical/tab but reports no
* matches for the query (distinct from a CAPTCHA block or a selector-rot
* failure) — e.g. "It looks like there aren't any 'Jobs' matches on this topic."
*/
async function isEmptyJobsVertical(page: Page): Promise<boolean> {
try {
return await page.evaluate(() => {
const bodyText = document.body?.textContent ?? "";
return /aren.t any .Jobs. matches/i.test(bodyText);
});
} catch {
return false;
}
}
function countCardsScript(selectors: readonly string[]): number {
for (const selector of selectors) {
const count = document.querySelectorAll(selector).length;
if (count > 0) return count;
}
return 0;
}
/** Google lazy-loads additional cards (via a custom `<infinity-scrolling>` element) as the page is scrolled. */
function scrollPageScript(): number {
const before = window.scrollY;
window.scrollTo(0, document.body.scrollHeight);
return window.scrollY - before;
}
async function growJobsList(
page: Page,
selectors: readonly string[],
targetCount: number,
): Promise<void> {
for (let attempt = 0; attempt < 8; attempt += 1) {
const count = await page.evaluate(countCardsScript, selectors);
if (count >= targetCount) return;
const scrolled = await page.evaluate(scrollPageScript);
if (scrolled <= 0) return;
await page.waitForTimeout(900);
}
}
function extractCardsScript(selectors: readonly string[]): ScrapedCard[] {
// NOTE: this whole function is serialized via Function.prototype.toString()
// and re-run inside the browser page by Playwright — it cannot reference
// anything from the outer module scope (constants, other functions, etc.).
// All helpers must be declared inside this function body.
function clean(value: string | null | undefined): string {
return (value ?? "").replace(/\s+/g, " ").trim();
}
/** Extracts Google's internal `htidocid` from a job's `data-share-url`, matching `data-encoded-docid` in the detail panel's apply-link list. */
function extractDocId(shareUrl: string | null): string | undefined {
if (!shareUrl) return undefined;
try {
const url = new URL(shareUrl, "https://www.google.com");
return url.searchParams.get("htidocid") ?? undefined;
} catch {
return undefined;
}
}
function findCards(): Element[] {
for (const selector of selectors) {
const items = Array.from(document.querySelectorAll(selector));
if (items.length > 0) return items;
}
return [];
}
const cards: ScrapedCard[] = [];
for (const item of findCards()) {
// Known current classes first (title/employer/location+via), falling
// back to a leaf-node text heuristic if Google has rotated them.
const titleEl = item.querySelector(".tNxQIb, h2, [role='heading']");
const title = clean(titleEl?.textContent);
if (!title) continue;
const wHYlTdEls = Array.from(item.querySelectorAll(".wHYlTd"));
let employer = clean(wHYlTdEls[0]?.textContent);
let location: string | undefined;
let via: string | undefined;
const viaBlock = wHYlTdEls.find((el) =>
/\bvia\b/i.test(el.textContent ?? ""),
);
if (viaBlock) {
const text = clean(viaBlock.textContent);
const match = text.match(/^(.*?)\s*(?:•|·)\s*via\s+(.+)$/i);
if (match) {
location = match[1] || undefined;
via = match[2] || undefined;
} else {
via = clean(text.replace(/^via\s*/i, ""));
}
}
if (!employer || !title || employer === title) {
// Selector rot fallback: scan leaf text nodes structurally.
const leafTexts = Array.from(item.querySelectorAll("div, span"))
.filter((el) => el.children.length === 0)
.map((el) => clean(el.textContent))
.filter((text) => text && text !== title && !/^[•·|,]+$/.test(text));
employer = employer || leafTexts[0] || "Unknown Employer";
if (!via) {
const viaText = leafTexts.find((text) => /^via\b/i.test(text));
via = viaText ? clean(viaText.replace(/^via\s*/i, "")) : undefined;
location =
location ?? leafTexts.slice(1).find((text) => text !== viaText);
}
}
const shareUrlRaw = item.getAttribute("data-share-url") ?? undefined;
const docId = extractDocId(shareUrlRaw ?? null);
cards.push({
docId,
title,
employer: employer || "Unknown Employer",
location,
via,
shareUrl: shareUrlRaw,
});
}
return cards;
}
/**
* Scopes apply-link extraction to the clicked job's own `data-encoded-docid`
* block. Google's detail viewer can have adjacent/related jobs' apply links
* present in the DOM simultaneously, so a document-wide "first apply link"
* scan can silently pick up the wrong job's link.
*/
function extractDetailScript(docId: string | undefined): ScrapedDetail {
function clean(value: string | null | undefined): string {
return (value ?? "").replace(/\s+/g, " ").trim();
}
function collectApplyLinks(
root: ParentNode,
): Array<{ href: string; text: string }> {
const links: Array<{ href: string; text: string }> = [];
for (const anchor of Array.from(root.querySelectorAll("a[href]"))) {
const href = (anchor as HTMLAnchorElement).href;
const text =
clean(anchor.textContent) || clean(anchor.getAttribute("aria-label"));
if (!href) continue;
if (
!/apply/i.test(text) &&
!/apply/i.test(anchor.getAttribute("aria-label") ?? "")
) {
continue;
}
if (
href.includes("google.com/search") ||
href.includes("accounts.google.com")
) {
continue;
}
links.push({ href, text });
}
return links;
}
let applyLinks: Array<{ href: string; text: string }> = [];
if (docId) {
const scoped = document.querySelector(`[data-encoded-docid="${docId}"]`);
if (scoped) applyLinks = collectApplyLinks(scoped);
}
if (applyLinks.length === 0) {
applyLinks = collectApplyLinks(document);
}
// `.textContent` also pulls in text from nested <style>/<script> tags
// (Google embeds scoped theme <style> blocks inside content divs) and from
// the job-list sidebar itself (which concatenates every visible card's
// text and easily outsizes a real description) — exclude both.
const blocks = Array.from(document.querySelectorAll("div"))
.filter(
(el) =>
el.children.length <= 3 && !el.querySelector("style, script, .EimVGf"),
)
.map((el) => clean(el.textContent))
.filter((text) => text.length > 200);
blocks.sort((a, b) => b.length - a.length);
return {
applyLinks,
description: blocks[0]?.slice(0, 5_000),
};
}
async function openCardAndExtractDetail(
page: Page,
selectors: readonly string[],
index: number,
docId: string | undefined,
): Promise<ScrapedDetail | null> {
const elementHandle = await page.evaluateHandle(
(args: { selectors: readonly string[]; idx: number }) => {
for (const selector of args.selectors) {
const items = document.querySelectorAll(selector);
if (items.length > 0) return items[args.idx] ?? null;
}
return null;
},
{ selectors, idx: index },
);
const element = elementHandle.asElement();
if (!element) return null;
await element.scrollIntoViewIfNeeded();
// A single click is unreliable here: clicking a card while a *different*
// card's detail pane is already open frequently just closes the open pane
// instead of switching to the new one (confirmed by watching real runs —
// it alternates open/closed rather than opening the newly clicked card).
// A double-click reopens reliably; verified across 6 sequential cards.
await element.dblclick({ timeout: 5_000, delay: 80 });
if (docId) {
try {
await page.waitForSelector(`[data-encoded-docid="${docId}"]`, {
timeout: 6_000,
});
} catch {
// fall through — extractDetailScript falls back to a document-wide
// scan for apply links if the docid-scoped block never shows up.
}
}
await page.waitForTimeout(500);
return page.evaluate(extractDetailScript, docId);
}
async function saveDebugArtifacts(page: Page): Promise<void> {
try {
await mkdir(dirname(DEBUG_HTML_PATH), { recursive: true });
await writeFile(DEBUG_HTML_PATH, await page.content(), "utf-8");
await page.screenshot({ path: DEBUG_SCREENSHOT_PATH, fullPage: true });
} catch (error) {
console.error(
`[google-jobs] failed to save debug artifacts: ${error instanceof Error ? error.message : String(error)}`,
);
}
}
async function scrapeTerm(
context: BrowserContext,
term: string,
remainingCap: number,
options: { headless: boolean; unblockTimeoutMs: number },
): Promise<ScrapedJob[]> {
const page = await context.newPage();
const out: ScrapedJob[] = [];
try {
await page.goto(buildSearchUrl(term), {
waitUntil: "domcontentloaded",
timeout: 45_000,
});
await dismissConsent(page);
await page.waitForTimeout(1_500);
if (await isBlockedPage(page)) {
const solvedByHuman =
!options.headless &&
(await waitForHumanToUnblock(page, options.unblockTimeoutMs));
if (!solvedByHuman) {
throw new GoogleBlockedError(
"Google blocked this request (CAPTCHA / unusual traffic detection). This is common from shared, VPN, or datacenter IPs. Try again later, reduce run frequency, run from a residential network, set GOOGLE_JOBS_PROXY, or set GOOGLE_JOBS_HEADLESS=false to solve it yourself.",
);
}
}
const listReady = await waitForJobsList(page);
if (!listReady) {
if (await isEmptyJobsVertical(page)) {
console.error(
`[google-jobs] Google's Jobs tab reported no matches for "${term}" (not a block or selector issue — try a broader/more common phrasing).`,
);
} else {
console.error(
`[google-jobs] Jobs list did not appear for "${term}" (url: ${page.url()}). Saved a screenshot + HTML dump to ${DEBUG_SCREENSHOT_PATH} for inspection.`,
);
await saveDebugArtifacts(page);
}
return out;
}
await growJobsList(page, CARD_SELECTORS, remainingCap);
const cards = await page.evaluate(extractCardsScript, CARD_SELECTORS);
const limit = Math.min(cards.length, remainingCap);
for (let index = 0; index < limit; index += 1) {
const card = cards[index];
// Google's own permalink into this specific listing — always available
// from the list view, so results stay unique/useful even if the detail
// click below fails for this card.
const shareUrl = card.shareUrl;
try {
const detail = await openCardAndExtractDetail(
page,
CARD_SELECTORS,
index,
card.docId,
);
const applyLink = detail?.applyLinks[0];
const jobUrl = shareUrl ?? applyLink?.href ?? page.url();
out.push({
title: card.title,
employer: card.employer,
jobUrl,
applicationLink: applyLink?.href,
location: card.location,
jobDescription: detail?.description,
postedVia: card.via,
isRemote: /remote/i.test(`${card.location ?? ""} ${card.title}`),
});
} catch (error) {
console.error(
`[google-jobs] card ${index} for "${term}" failed: ${
error instanceof Error ? error.message : String(error)
}`,
);
if (shareUrl) {
// Detail click failed, but we still have a unique, valid link to
// the listing from the card itself — don't drop the job entirely.
out.push({
title: card.title,
employer: card.employer,
jobUrl: shareUrl,
location: card.location,
postedVia: card.via,
isRemote: /remote/i.test(`${card.location ?? ""} ${card.title}`),
});
}
}
await page.waitForTimeout(300 + Math.floor(Math.random() * 400));
}
} finally {
await page.close();
}
return out;
}
async function main(): Promise<void> {
const terms = parseTerms(process.env.GOOGLE_JOBS_SEARCH_TERMS);
const maxJobsEnv = Number.parseInt(
process.env.GOOGLE_JOBS_MAX_JOBS ?? "30",
10,
);
const cap = Number.isFinite(maxJobsEnv)
? Math.max(1, Math.min(maxJobsEnv, 150))
: 30;
// Headed mode lets a human solve a CAPTCHA in the visible window; only
// useful when run directly on a machine with a display, not on a headless
// cron/server pipeline.
const headless = process.env.GOOGLE_JOBS_HEADLESS !== "false";
const unblockTimeoutMs = Number.parseInt(
process.env.GOOGLE_JOBS_UNBLOCK_TIMEOUT_MS ?? "300000",
10,
);
const browser = await launchBrowser(headless);
const hasSavedState = await access(STATE_PATH)
.then(() => true)
.catch(() => false);
const context = await browser.newContext(
hasSavedState ? { storageState: STATE_PATH } : {},
);
// tsx/esbuild compiles this file with `keepNames`, which injects
// `__name(fn, "fn")` calls into any named function it transpiles —
// including ones later serialized into `page.evaluate()` scripts via
// `Function.prototype.toString()`. That helper only exists in our Node
// process, not in the browser page, so without this no-op polyfill any
// evaluate script with a nested named function throws
// `ReferenceError: __name is not defined`.
await context.addInitScript(() => {
(window as unknown as { __name?: (fn: unknown) => unknown }).__name = (
fn: unknown,
) => fn;
});
const all: ScrapedJob[] = [];
const seenUrls = new Set<string>();
try {
for (const term of terms) {
if (all.length >= cap) break;
try {
const rows = await scrapeTerm(context, term, cap - all.length, {
headless,
unblockTimeoutMs,
});
for (const row of rows) {
if (all.length >= cap) break;
if (seenUrls.has(row.jobUrl)) continue;
seenUrls.add(row.jobUrl);
all.push(row);
}
} catch (error) {
if (error instanceof GoogleBlockedError) {
// IP-level block affects the whole run; stop trying further terms
// and let it propagate so the process exits non-zero with a clear message.
throw error;
}
console.error(
`[google-jobs] term "${term}" failed: ${
error instanceof Error ? error.message : String(error)
}`,
);
}
}
} finally {
// Persist cookies/session (even a partially solved one) so future
// headless/automated runs can reuse it and hit fewer CAPTCHA walls.
await mkdir(dirname(STATE_PATH), { recursive: true });
await context.storageState({ path: STATE_PATH }).catch(() => undefined);
await browser.close();
}
await mkdir(dirname(OUTPUT_PATH), { recursive: true });
await writeFile(OUTPUT_PATH, JSON.stringify(all, null, 2), "utf-8");
}
main().catch((error) => {
console.error(error instanceof Error ? error.message : String(error));
process.exit(1);
});
-168
View File
@@ -1,168 +0,0 @@
import { spawn, spawnSync } from "node:child_process";
import { readFile } from "node:fs/promises";
import { createRequire } from "node:module";
import { dirname, join } from "node:path";
import { fileURLToPath } from "node:url";
import { envForExtractorSubprocess } from "@shared/extractor-subprocess-env.js";
import type { CreateJobInput } from "@shared/types/jobs";
const srcDir = dirname(fileURLToPath(import.meta.url));
const EXTRACTOR_DIR = join(srcDir, "..");
const OUTPUT_PATH = join(EXTRACTOR_DIR, "storage/jobs.json");
const require = createRequire(import.meta.url);
const TSX_CLI_PATH = (() => {
try {
return require.resolve("tsx/dist/cli.mjs");
} catch {
return null;
}
})();
/** Browser scrape can take a while (per-term navigation + per-card clicks). */
const SUBPROCESS_TIMEOUT_MS = 4 * 60 * 1000;
interface ScrapedJob {
title?: string;
employer?: string;
jobUrl?: string;
applicationLink?: string;
location?: string;
jobDescription?: string;
postedVia?: string;
isRemote?: boolean;
}
export interface RunGoogleJobsOptions {
searchTerms?: string[];
maxJobs?: number;
}
export interface GoogleJobsResult {
success: boolean;
jobs: CreateJobInput[];
error?: string;
}
function canRunNpmCommand(): boolean {
const result = spawnSync("npm", ["--version"], { stdio: "ignore" });
return !result.error && result.status === 0;
}
function mapJob(row: ScrapedJob): CreateJobInput | null {
const jobUrl = row.jobUrl?.trim();
if (!jobUrl) return null;
return {
source: "google-jobs",
title: row.title?.trim() || "Unknown Title",
employer: row.employer?.trim() || "Unknown Employer",
jobUrl,
applicationLink: row.applicationLink?.trim() || jobUrl,
location: row.location?.trim() || undefined,
jobDescription: row.jobDescription?.trim() || undefined,
isRemote: row.isRemote ?? undefined,
};
}
export async function runGoogleJobs(
options: RunGoogleJobsOptions = {},
): Promise<GoogleJobsResult> {
const searchTerms =
options.searchTerms && options.searchTerms.length > 0
? options.searchTerms
: ["software engineer"];
const maxJobs = options.maxJobs ?? 30;
const useNpmCommand = canRunNpmCommand();
if (!TSX_CLI_PATH && !useNpmCommand) {
return {
success: false,
jobs: [],
error: "Unable to execute Google Jobs extractor (npm/tsx unavailable)",
};
}
try {
await new Promise<void>((resolve, reject) => {
const extractorEnv = envForExtractorSubprocess({
...process.env,
GOOGLE_JOBS_SEARCH_TERMS: JSON.stringify(searchTerms),
GOOGLE_JOBS_MAX_JOBS: String(maxJobs),
});
const child = TSX_CLI_PATH
? spawn(process.execPath, [TSX_CLI_PATH, join(srcDir, "main.ts")], {
cwd: EXTRACTOR_DIR,
env: extractorEnv,
stdio: ["ignore", "pipe", "pipe"],
})
: spawn("npx", ["tsx", join(srcDir, "main.ts")], {
cwd: EXTRACTOR_DIR,
env: extractorEnv,
stdio: ["ignore", "pipe", "pipe"],
});
let stderr = "";
child.stderr?.on("data", (chunk: Buffer) => {
stderr += chunk.toString();
});
const timeout = setTimeout(() => {
child.kill("SIGKILL");
reject(new Error("Google Jobs browser scrape timed out"));
}, SUBPROCESS_TIMEOUT_MS);
child.on("error", (error) => {
clearTimeout(timeout);
reject(error);
});
child.on("close", (code) => {
clearTimeout(timeout);
if (code === 0) {
resolve();
return;
}
reject(
new Error(
stderr.trim() ||
`Google Jobs browser scrape exited with code ${code}`,
),
);
});
});
const raw = await readFile(OUTPUT_PATH, "utf-8");
const parsed = JSON.parse(raw) as unknown;
const rows = Array.isArray(parsed) ? parsed : [];
const jobs: CreateJobInput[] = [];
const seen = new Set<string>();
for (const value of rows) {
if (!value || typeof value !== "object" || Array.isArray(value)) continue;
const mapped = mapJob(value as ScrapedJob);
if (!mapped) continue;
const key = mapped.jobUrl;
if (seen.has(key)) continue;
seen.add(key);
jobs.push(mapped);
}
const { filterExpiredJobListings } = await import(
"@shared/job-listing-probe.js"
);
const { kept, dropped } = await filterExpiredJobListings(jobs, {
concurrency: 4,
timeoutMs: 5_000,
});
if (dropped > 0) {
console.info(
`[google-jobs] Dropped ${dropped} expired listing(s) after URL probe`,
);
}
return { success: true, jobs: kept };
} catch (error) {
const message = error instanceof Error ? error.message : "Unknown error";
return { success: false, jobs: [], error: message };
}
}
-17
View File
@@ -1,17 +0,0 @@
{
"compilerOptions": {
"module": "ESNext",
"moduleResolution": "bundler",
"target": "ES2022",
"outDir": "dist",
"strict": true,
"noUnusedLocals": false,
"lib": ["ES2022", "DOM"],
"types": ["node"],
"baseUrl": ".",
"paths": {
"@shared/*": ["../../shared/src/*"]
}
},
"include": ["./manifest.ts", "./src/**/*"]
}
+5 -26
View File
@@ -116,32 +116,11 @@ router.addHandler(
const title = (await titleLocator.textContent())?.trim() ?? null;
const jobUrl = toAbsolute(await titleLocator.getAttribute("href"));
// Employer logo/link are optional — some cards omit <figure img>, and a
// hard getAttribute timeout used to fail the whole region page (0 jobs)
// even after we had already counted cards.
let employer: string | null = null;
let employerUrl: string | null = null;
try {
const employerImg = article.locator("figure img").first();
if ((await employerImg.count()) > 0) {
employer =
(
await employerImg.getAttribute("alt", { timeout: 2000 })
)?.trim() ?? null;
}
} catch {
// optional
}
try {
const employerAnchor = article.locator("figure a").first();
if ((await employerAnchor.count()) > 0) {
employerUrl = toAbsolute(
await employerAnchor.getAttribute("href", { timeout: 2000 }),
);
}
} catch {
// optional
}
const employerImg = article.locator("figure img");
const employer = (await employerImg.getAttribute("alt"))?.trim() ?? null;
const employerAnchor = article.locator("figure a");
const employerUrl = toAbsolute(await employerAnchor.getAttribute("href"));
let disciplines: string | null = null;
try {
+2 -4
View File
@@ -14,10 +14,8 @@ import type {
import type { CreateJobInput } from "@shared/types/jobs";
const API_URL = "https://himalayas.app/jobs/api";
/** Himalayas silently caps `limit` at 20; requesting more makes us think the
* feed ended early (`raw.length < PAGE_SIZE`) and stop after the first page. */
const PAGE_SIZE = 20;
const MAX_PAGES = 10;
const PAGE_SIZE = 50;
const MAX_PAGES = 5;
interface HimalayasJob {
title?: string;
+52 -71
View File
@@ -111,6 +111,12 @@ function parseWorkplaceTypes(
}
}
function encodeSearchState(searchState: unknown): string {
const json = JSON.stringify(searchState);
const urlEncodedJson = encodeURIComponent(json);
return Buffer.from(urlEncodedJson, "utf-8").toString("base64");
}
function asRecord(value: unknown): Record<string, unknown> | null {
if (!value || typeof value !== "object" || Array.isArray(value)) return null;
return value as Record<string, unknown>;
@@ -224,18 +230,7 @@ function extractResultsBatch(payload: unknown): RawHiringCafeJob[] {
}
const payloadRecord = asRecord(payload);
if (!payloadRecord) return [];
const pageProps = asRecord(payloadRecord.pageProps);
const ssrHits = pageProps?.ssrHits ?? payloadRecord.ssrHits;
if (Array.isArray(ssrHits)) {
return ssrHits.filter(
(item): item is RawHiringCafeJob =>
Boolean(item) && typeof item === "object" && !Array.isArray(item),
);
}
const results = payloadRecord.results;
const results = payloadRecord?.results;
if (!Array.isArray(results)) return [];
return results.filter(
@@ -247,22 +242,7 @@ function extractResultsBatch(payload: unknown): RawHiringCafeJob[] {
function parseTotalCount(payload: unknown): number | null {
const payloadRecord = asRecord(payload);
if (!payloadRecord) return null;
const pageProps = asRecord(payloadRecord.pageProps);
return (
toNumberOrNull(pageProps?.ssrTotalCount) ??
toNumberOrNull(payloadRecord.ssrTotalCount) ??
toNumberOrNull(payloadRecord.total)
);
}
function isLastSsrPage(payload: unknown): boolean {
const payloadRecord = asRecord(payload);
if (!payloadRecord) return true;
const pageProps = asRecord(payloadRecord.pageProps);
const isLastPage = pageProps?.ssrIsLastPage ?? payloadRecord.ssrIsLastPage;
return isLastPage === true;
return toNumberOrNull(payloadRecord.total);
}
function buildCityLocationId(input: string): string {
@@ -521,37 +501,23 @@ function createCitySearchState(args: {
};
}
async function readBuildIdFromPage(page: Page): Promise<string> {
const buildId = await page.evaluate(() => {
const element = document.querySelector("#__NEXT_DATA__");
if (!element?.textContent) return null;
const data = JSON.parse(element.textContent) as { buildId?: string };
return data.buildId ?? null;
});
if (!buildId) {
throw new Error("Hiring Cafe page did not expose Next.js buildId");
}
return buildId;
}
async function fetchSsrSearchPage(
async function callHiringCafeApi(
page: Page,
buildId: string,
searchState: Record<string, unknown>,
pageNo: number,
endpoint: string,
params: Record<string, string>,
): Promise<unknown> {
const response = await page.evaluate(
async ({ buildIdArg, searchStateArg, pageNoArg }) => {
const params = new URLSearchParams();
params.set("searchState", JSON.stringify(searchStateArg));
params.set("page", String(pageNoArg));
const url = `/_next/data/${buildIdArg}/index.json?${params.toString()}`;
async ({ endpointArg, paramsArg }) => {
const url = new URL(endpointArg, window.location.origin);
for (const [key, value] of Object.entries(paramsArg)) {
url.searchParams.set(key, value);
}
const res = await fetch(url, {
const res = await fetch(url.toString(), {
method: "GET",
credentials: "include",
headers: {
Accept: "application/json",
"x-nextjs-data": "1",
Accept: "application/json, text/plain, */*",
},
});
@@ -573,7 +539,7 @@ async function fetchSsrSearchPage(
return output;
},
{ buildIdArg: buildId, searchStateArg: searchState, pageNoArg: pageNo },
{ endpointArg: endpoint, paramsArg: params },
);
const result = response as BrowserApiResponse;
@@ -581,14 +547,14 @@ async function fetchSsrSearchPage(
if (!result.ok) {
const snippet = result.responseText.slice(0, 250);
throw new Error(
`Hiring Cafe SSR search failed (${result.status} ${result.statusText}): ${snippet}`,
`Hiring Cafe API ${endpoint} failed (${result.status} ${result.statusText}): ${snippet}`,
);
}
if (result.data === null) {
const snippet = result.responseText.slice(0, 250);
throw new Error(
`Hiring Cafe SSR search returned non-JSON response: ${snippet}`,
`Hiring Cafe API ${endpoint} returned non-JSON response: ${snippet}`,
);
}
@@ -660,8 +626,6 @@ async function run(): Promise<void> {
await initializePage();
}
const buildId = await readBuildIdFromPage(page);
const countryLocation = resolveHiringCafeCountryLocation(country);
const countryLong =
countryLocation?.address_components[0]?.long_name ?? "United Kingdom";
@@ -700,23 +664,40 @@ async function run(): Promise<void> {
dateFetchedPastNDays,
workplaceTypes,
});
let termTarget = maxJobsPerTerm;
const encodedSearchState = encodeSearchState(searchState);
let totalAvailable: number | null = null;
try {
const countPayload = await callHiringCafeApi(
page,
"/api/search-jobs/get-total-count",
{
s: encodedSearchState,
},
);
totalAvailable = parseTotalCount(countPayload);
} catch (error) {
const message = error instanceof Error ? error.message : String(error);
console.warn(
`Hiring Cafe count request failed for term '${searchTerm}': ${message}`,
);
}
const termTarget =
totalAvailable !== null
? Math.min(maxJobsPerTerm, totalAvailable)
: maxJobsPerTerm;
let pageNo = 0;
let termCollected = 0;
while (termCollected < termTarget && pageNo < PAGE_LIMIT) {
const jobsPayload = await fetchSsrSearchPage(
page,
buildId,
searchState,
pageNo,
);
const totalAvailable = parseTotalCount(jobsPayload);
if (totalAvailable !== null) {
termTarget = Math.min(maxJobsPerTerm, totalAvailable);
}
const size = Math.min(1000, termTarget - termCollected);
const jobsPayload = await callHiringCafeApi(page, "/api/search-jobs", {
size: String(size),
page: String(pageNo),
s: encodedSearchState,
});
const batch = extractResultsBatch(jobsPayload);
if (batch.length === 0) break;
@@ -746,7 +727,7 @@ async function run(): Promise<void> {
totalCollected: termCollected,
});
if (isLastSsrPage(jobsPayload)) break;
if (batch.length < size) break;
pageNo += 1;
}
-221
View File
@@ -1,221 +0,0 @@
/**
* Huntflow public career sites — SSR HTML listing + optional vacancy detail pages.
*
* https://{tenant}.huntflow.io/
*/
import type {
ExtractorManifest,
ExtractorRunResult,
} from "@shared/types/extractors";
import type { CreateJobInput } from "@shared/types/jobs";
interface HuntflowListing {
slug: string;
title: string;
location?: string;
}
function asString(value: unknown): string | undefined {
if (typeof value !== "string") return undefined;
const trimmed = value.trim();
return trimmed ? trimmed : undefined;
}
function readTenants(raw: string | undefined): string[] {
if (!raw) return [];
try {
const parsed = JSON.parse(raw);
if (Array.isArray(parsed)) {
return parsed
.map((entry) => (typeof entry === "string" ? entry.trim() : ""))
.filter(Boolean);
}
} catch {
// fall through
}
return raw
.split(/[\n,;|]+/)
.map((entry) => entry.trim())
.filter(Boolean);
}
function decodeHtmlEntities(value: string): string {
return value
.replace(/&amp;/g, "&")
.replace(/&lt;/g, "<")
.replace(/&gt;/g, ">")
.replace(/&quot;/g, '"')
.replace(/&#39;/g, "'");
}
function stripHtml(html: string): string {
return decodeHtmlEntities(html)
.replace(/<[^>]+>/g, " ")
.replace(/\s+/g, " ")
.trim();
}
function parseListings(html: string): HuntflowListing[] {
const listings: HuntflowListing[] = [];
const pattern =
/<article class="_item_[^"]*">[\s\S]*?<a href="(\/vacancy\/[^"]+)"[^>]*>([^<]+)<\/a>[\s\S]*?<div class="_info_[^"]*">([^<]*)<\/div>/g;
for (const match of html.matchAll(pattern)) {
const slug = match[1]?.replace(/^\/vacancy\//, "").replace(/\/$/, "");
const title = match[2]?.trim();
if (!slug || !title) continue;
listings.push({
slug,
title: decodeHtmlEntities(title),
location: match[3]?.trim() || undefined,
});
}
return listings;
}
function parseVacancyDetail(html: string): {
description?: string;
location?: string;
} {
const positionMatch = html.match(/<h1 class="_position_[^"]*">([^<]+)<\/h1>/);
const infoMatch = html.match(
/<div class="_infoWrapper_[^"]*">[\s\S]*?<div>([^<]+)<\/div>/,
);
const sections: string[] = [];
for (const match of html.matchAll(
/<div class="_content_1phzm_2">[\s\S]*?<!--\[-->([\s\S]*?)<!--\]-->/g,
)) {
const text = stripHtml(match[1] ?? "");
if (text) sections.push(text);
}
return {
location: infoMatch?.[1]?.trim() || undefined,
description:
sections.length > 0
? sections.join("\n\n")
: positionMatch?.[1]?.trim() || undefined,
};
}
function matchesTerm(values: string[], term: string): boolean {
const lower = term.toLowerCase();
return values.some((value) => value.toLowerCase().includes(lower));
}
async function fetchText(url: string): Promise<string> {
const response = await fetch(url, {
headers: { Accept: "text/html", "User-Agent": "JobOps/1.0" },
});
if (!response.ok) {
throw new Error(`Huntflow request failed (${response.status}) for ${url}`);
}
return response.text();
}
function tenantOrigin(tenant: string): string {
const host = tenant.includes(".") ? tenant : `${tenant}.huntflow.io`;
return host.startsWith("http") ? host : `https://${host}`;
}
export const manifest: ExtractorManifest = {
id: "huntflow",
displayName: "Huntflow (ATS)",
providesSources: ["huntflow"],
async run(context): Promise<ExtractorRunResult> {
if (context.shouldCancel?.()) return { success: true, jobs: [] };
const tenants = readTenants(context.settings.huntflowTenants);
if (tenants.length === 0) {
return {
success: true,
jobs: [],
error:
"No Huntflow tenants configured. Set HUNTFLOW_TENANTS or huntflowTenants (comma- or newline-separated subdomains, e.g. apicworld).",
};
}
const terms = context.searchTerms.length > 0 ? context.searchTerms : [];
const enrichDetails = true;
const seen = new Set<string>();
const out: CreateJobInput[] = [];
try {
for (let i = 0; i < tenants.length; i += 1) {
if (context.shouldCancel?.()) break;
const tenant = tenants[i];
const origin = tenantOrigin(tenant);
context.onProgress?.({
phase: "list",
termsProcessed: i,
termsTotal: tenants.length,
currentUrl: origin,
detail: `Huntflow: ${tenant} (${i + 1}/${tenants.length})`,
});
const indexHtml = await fetchText(`${origin}/`);
const employer = tenant
.split(".")[0]
.split(/[-_]/)
.map((part) => part.charAt(0).toUpperCase() + part.slice(1))
.join(" ");
for (const listing of parseListings(indexHtml)) {
if (context.shouldCancel?.()) break;
const haystack = [listing.title, listing.location ?? ""];
if (
terms.length > 0 &&
!terms.some((term) => matchesTerm(haystack, term))
) {
continue;
}
const jobUrl = `${origin}/vacancy/${listing.slug}`;
let location = listing.location ?? "Unknown";
let jobDescription: string | undefined;
let isRemote = location.toLowerCase().includes("remote");
if (enrichDetails) {
try {
const detailHtml = await fetchText(jobUrl);
const detail = parseVacancyDetail(detailHtml);
if (detail.location) location = detail.location;
if (detail.description) jobDescription = detail.description;
isRemote = location.toLowerCase().includes("remote");
} catch {
// keep listing row when detail fetch fails
}
}
const key = jobUrl;
if (seen.has(key)) continue;
seen.add(key);
out.push({
source: "huntflow",
sourceJobId: listing.slug,
title: listing.title,
employer,
jobUrl,
applicationLink: jobUrl,
location,
isRemote,
jobDescription,
});
}
}
return { success: true, jobs: out };
} catch (error) {
const message = error instanceof Error ? error.message : "Unknown error";
return { success: false, jobs: out, error: message };
}
},
};
export default manifest;
-17
View File
@@ -1,17 +0,0 @@
{
"name": "huntflow-extractor",
"version": "0.0.1",
"type": "module",
"description": "Huntflow public career-site extractor",
"main": "manifest.ts",
"dependencies": {
"job-ops-shared": "^1.0.0"
},
"devDependencies": {
"@types/node": "^24.0.0",
"typescript": "~5.9.0"
},
"scripts": {
"check:types": "tsc --noEmit"
}
}
-17
View File
@@ -1,17 +0,0 @@
{
"compilerOptions": {
"module": "ESNext",
"moduleResolution": "bundler",
"target": "ES2022",
"outDir": "dist",
"strict": true,
"noUnusedLocals": false,
"lib": ["ES2022", "DOM"],
"types": ["node"],
"baseUrl": ".",
"paths": {
"@shared/*": ["../../shared/src/*"]
}
},
"include": ["./manifest.ts", "./src/**/*"]
}
+1 -6
View File
@@ -1,8 +1,3 @@
# python-jobspy requires Python 3.10+ (wheels not published for 3.9 and below).
#
# Pinned to 1.1.82: scrape_jobs.py monkeypatches Glassdoor._fetch_jobs_page
# to work around an upstream bug where any partial GraphQL error (even for
# unrelated fields) discards a whole page of valid job data. If bumping this
# version, re-check that method still matches before removing the pin.
python-jobspy==1.1.82
python-jobspy
pandas
-69
View File
@@ -1,81 +1,12 @@
import csv
import json
import logging
import os
from concurrent.futures import ThreadPoolExecutor, as_completed
from pathlib import Path
import pandas as pd
from jobspy import scrape_jobs
from jobspy.exception import GlassdoorException
from jobspy.glassdoor import Glassdoor
from jobspy.glassdoor.util import get_cursor_for_page
PROGRESS_PREFIX = "JOBOPS_PROGRESS "
_glassdoor_log = logging.getLogger("JobSpy:Glassdoor")
def _patched_glassdoor_fetch_jobs_page(
self, scraper_input, location_id, location_type, page_num, cursor
):
"""Replaces python-jobspy's Glassdoor._fetch_jobs_page (as of 1.1.82).
Upstream treats *any* `errors` entry in Glassdoor's GraphQL response as
fatal and discards the whole page, even when the job listings we need
parsed fine. In practice Glassdoor's backend intermittently (but
consistently, as of writing) returns a partial error for an unrelated
field — observed: `jobsPageSeoData` failing with
`SUBREQUEST_HTTP_ERROR` / "dns error" on Glassdoor's own infra — while
`data.jobListings.jobListings` still comes back with real results. The
upstream check throws that good data away, so every Glassdoor run
returns 0 jobs. This patch only bails out when the job listings payload
itself is actually missing.
"""
jobs = []
self.scraper_input = scraper_input
try:
payload = self._add_payload(location_id, location_type, page_num, cursor)
response = self.session.post(
f"{self.base_url}/graph",
timeout_seconds=15,
data=payload,
)
if response.status_code != 200:
raise GlassdoorException(
f"bad response status code: {response.status_code}"
)
res_json = response.json()[0]
job_listings = res_json.get("data", {}).get("jobListings")
if job_listings is None:
raise ValueError("Error encountered in API response")
if "errors" in res_json:
_glassdoor_log.warning(
"Glassdoor GraphQL returned partial errors for unrelated "
"fields; continuing with the job listings that did parse: %s",
res_json["errors"],
)
except (GlassdoorException, ValueError, Exception) as e:
_glassdoor_log.error(f"Glassdoor: {str(e)}")
return jobs, None
jobs_data = job_listings["jobListings"]
with ThreadPoolExecutor(max_workers=self.jobs_per_page) as executor:
future_to_job_data = {
executor.submit(self._process_job, job): job for job in jobs_data
}
for future in as_completed(future_to_job_data):
try:
job_post = future.result()
if job_post:
jobs.append(job_post)
except Exception as exc:
raise GlassdoorException(f"Glassdoor generated an exception: {exc}")
return jobs, get_cursor_for_page(job_listings["paginationCursors"], page_num + 1)
Glassdoor._fetch_jobs_page = _patched_glassdoor_fetch_jobs_page
COUNTRY_ALIASES = {
"uk": "united kingdom",
"united kingdom": "united kingdom",
+1 -1
View File
@@ -6,11 +6,11 @@ import { createInterface } from "node:readline";
import { fileURLToPath } from "node:url";
import { resolveSearchCities } from "@shared/search-cities.js";
import type { CreateJobInput, JobSource } from "@shared/types/jobs";
import { normalizeIsRemote } from "@shared/work-arrangement.js";
import {
toNumberOrNull,
toStringOrNull,
} from "@shared/utils/type-conversion.js";
import { normalizeIsRemote } from "@shared/work-arrangement.js";
const srcDir = dirname(fileURLToPath(import.meta.url));
const EXTRACTOR_DIR = join(srcDir, "..");
-216
View File
@@ -1,216 +0,0 @@
/**
* Teamtailor public career sites — RSS feed per tenant.
*
* https://{company}.teamtailor.com/jobs.rss
*/
import type {
ExtractorManifest,
ExtractorRunResult,
} from "@shared/types/extractors";
import type { CreateJobInput } from "@shared/types/jobs";
interface TeamtailorItem {
title?: string;
link?: string;
guid?: string;
description?: string;
pubDate?: string;
remoteStatus?: string;
companyName?: string;
location?: string;
department?: string;
}
function asString(value: unknown): string | undefined {
if (typeof value !== "string") return undefined;
const trimmed = value.trim();
return trimmed ? trimmed : undefined;
}
function readCompanies(raw: string | undefined): string[] {
if (!raw) return [];
try {
const parsed = JSON.parse(raw);
if (Array.isArray(parsed)) {
return parsed
.map((entry) => (typeof entry === "string" ? entry.trim() : ""))
.filter(Boolean);
}
} catch {
// fall through
}
return raw
.split(/[\n,;|]+/)
.map((entry) => entry.trim())
.filter(Boolean);
}
function xmlText(xml: string, tag: string): string | undefined {
const pattern = new RegExp(`<${tag}[^>]*>([\\s\\S]*?)</${tag}>`, "i");
const match = xml.match(pattern);
if (!match?.[1]) return undefined;
return (
match[1].replace(/<!\[CDATA\[([\s\S]*?)\]\]>/g, "$1").trim() || undefined
);
}
function decodeHtmlEntities(value: string): string {
return value
.replace(/&amp;/g, "&")
.replace(/&lt;/g, "<")
.replace(/&gt;/g, ">")
.replace(/&quot;/g, '"')
.replace(/&#39;/g, "'");
}
function stripHtml(html: string): string {
return decodeHtmlEntities(html)
.replace(/<[^>]+>/g, " ")
.replace(/\s+/g, " ")
.trim();
}
function parseItems(xml: string): TeamtailorItem[] {
const items: TeamtailorItem[] = [];
const blocks = xml.match(/<item>([\s\S]*?)<\/item>/gi) ?? [];
for (const raw of blocks) {
const block = raw.replace(/^<item>/i, "").replace(/<\/item>$/i, "");
items.push({
title: xmlText(block, "title"),
link: xmlText(block, "link"),
guid: xmlText(block, "guid"),
description: xmlText(block, "description"),
pubDate: xmlText(block, "pubDate"),
remoteStatus: xmlText(block, "remoteStatus"),
companyName: xmlText(block, "company_name"),
location: xmlText(block, "location"),
department: xmlText(block, "department"),
});
}
return items;
}
function matchesTerm(item: TeamtailorItem, term: string): boolean {
const lower = term.toLowerCase();
if (item.title?.toLowerCase().includes(lower)) return true;
if (item.department?.toLowerCase().includes(lower)) return true;
const description = item.description ? stripHtml(item.description) : "";
if (description.toLowerCase().includes(lower)) return true;
return false;
}
function mapJob(
item: TeamtailorItem,
companySlug: string,
): CreateJobInput | null {
const jobUrl = asString(item.link);
if (!jobUrl) return null;
const employer =
asString(item.companyName) ??
companySlug
.split(/[-_]/)
.map((part) => part.charAt(0).toUpperCase() + part.slice(1))
.join(" ");
const remoteStatus = asString(item.remoteStatus)?.toLowerCase();
const isRemote =
remoteStatus === "fully" ||
remoteStatus === "hybrid" ||
item.location?.toLowerCase().includes("remote") === true;
return {
source: "teamtailor",
sourceJobId: item.guid ?? jobUrl,
title: asString(item.title) ?? "Unknown Title",
employer,
jobUrl,
applicationLink: jobUrl,
location: asString(item.location) ?? "Unknown",
isRemote,
datePosted: asString(item.pubDate),
jobDescription: item.description ? stripHtml(item.description) : undefined,
jobFunction: asString(item.department),
};
}
async function fetchCompanyFeed(company: string): Promise<TeamtailorItem[]> {
const url = `https://${encodeURIComponent(company)}.teamtailor.com/jobs.rss?per_page=200`;
const response = await fetch(url, {
headers: {
Accept: "application/rss+xml, application/xml, text/xml",
"User-Agent": "JobOps/1.0",
},
});
if (response.status === 404) return [];
if (!response.ok) {
throw new Error(
`Teamtailor RSS for "${company}" failed with status ${response.status}`,
);
}
return parseItems(await response.text());
}
export const manifest: ExtractorManifest = {
id: "teamtailor",
displayName: "Teamtailor (ATS)",
providesSources: ["teamtailor"],
async run(context): Promise<ExtractorRunResult> {
if (context.shouldCancel?.()) return { success: true, jobs: [] };
const companies = readCompanies(context.settings.teamtailorCompanies);
if (companies.length === 0) {
return {
success: true,
jobs: [],
error:
"No Teamtailor companies configured. Set TEAMTAILOR_COMPANIES or teamtailorCompanies (comma- or newline-separated subdomains).",
};
}
const terms = context.searchTerms.length > 0 ? context.searchTerms : [];
const seen = new Set<string>();
const out: CreateJobInput[] = [];
try {
for (let i = 0; i < companies.length; i += 1) {
if (context.shouldCancel?.()) break;
const company = companies[i];
context.onProgress?.({
phase: "list",
termsProcessed: i,
termsTotal: companies.length,
currentUrl: company,
detail: `Teamtailor: ${company} (${i + 1}/${companies.length})`,
});
const items = await fetchCompanyFeed(company);
for (const item of items) {
if (
terms.length > 0 &&
!terms.some((term) => matchesTerm(item, term))
) {
continue;
}
const mapped = mapJob(item, company);
if (!mapped) continue;
const key = mapped.sourceJobId || mapped.jobUrl;
if (seen.has(key)) continue;
seen.add(key);
out.push(mapped);
}
}
return { success: true, jobs: out };
} catch (error) {
const message = error instanceof Error ? error.message : "Unknown error";
return { success: false, jobs: out, error: message };
}
},
};
export default manifest;
-17
View File
@@ -1,17 +0,0 @@
{
"name": "teamtailor-extractor",
"version": "0.0.1",
"type": "module",
"description": "Teamtailor public RSS career-site extractor",
"main": "manifest.ts",
"dependencies": {
"job-ops-shared": "^1.0.0"
},
"devDependencies": {
"@types/node": "^24.0.0",
"typescript": "~5.9.0"
},
"scripts": {
"check:types": "tsc --noEmit"
}
}
-17
View File
@@ -1,17 +0,0 @@
{
"compilerOptions": {
"module": "ESNext",
"moduleResolution": "bundler",
"target": "ES2022",
"outDir": "dist",
"strict": true,
"noUnusedLocals": false,
"lib": ["ES2022", "DOM"],
"types": ["node"],
"baseUrl": ".",
"paths": {
"@shared/*": ["../../shared/src/*"]
}
},
"include": ["./manifest.ts", "./src/**/*"]
}
-259
View File
@@ -1,259 +0,0 @@
/**
* TestDevJobs — QA-focused board (Gridsome SSR HTML + embedded job state).
*
* https://testdevjobs.com/software-testing-jobs/
*/
import type {
ExtractorManifest,
ExtractorRunResult,
} from "@shared/types/extractors";
import type { CreateJobInput } from "@shared/types/jobs";
const ORIGIN = "https://testdevjobs.com";
const LIST_PATH = "/software-testing-jobs/";
interface ListingJob {
path: string;
title: string;
employer: string;
datePosted?: string;
location?: string;
jobType?: string;
}
interface TestDevJobState {
id?: string;
jobTitle?: string;
path?: string;
jobDescription?: string;
joblocation?: string;
jobType?: string;
salary?: string;
isRemote?: boolean;
applyLink?: string;
companyName?: string;
jobPosted?: string;
}
function asString(value: unknown): string | undefined {
if (typeof value !== "string") return undefined;
const trimmed = value.trim();
return trimmed ? trimmed : undefined;
}
function decodeHtmlEntities(value: string): string {
return value
.replace(/&amp;/g, "&")
.replace(/&lt;/g, "<")
.replace(/&gt;/g, ">")
.replace(/&quot;/g, '"')
.replace(/&#39;/g, "'");
}
function stripHtml(html: string): string {
return decodeHtmlEntities(html)
.replace(/<[^>]+>/g, " ")
.replace(/\s+/g, " ")
.trim();
}
function readMaxPages(raw: string | undefined): number {
const parsed = raw ? Number.parseInt(raw, 10) : Number.NaN;
if (!Number.isFinite(parsed)) return 3;
return Math.min(Math.max(parsed, 1), 20);
}
function parseListingJobs(html: string): ListingJob[] {
const jobs: ListingJob[] = [];
const blocks = html.match(
/<div class="job-tile-wrapper[\s\S]*?<\/div>\s*<\/div>\s*<\/div>/g,
);
for (const block of blocks ?? []) {
const pathMatch = block.match(/href="(\/job\/[^"]+)"/);
const titleMatch = block.match(/class="ml-5 jobtitle[^"]*"[^>]*>([^<]+)</);
const employerMatch = block.match(
/class="mb-0 comptitle[^"]*"[^>]*>([^<]+)</,
);
const dateMatch = block.match(/itemprop="datePosted"[^>]*>([^<]+)</);
const locationMatch = block.match(
/itemprop="addressLocality"[^>]*>([^<]+)</,
);
const typeMatch = block.match(/itemprop="employmentType"[^>]*>([^<]+)</);
const path = pathMatch?.[1];
const title = titleMatch?.[1]?.trim();
if (!path || !title) continue;
jobs.push({
path,
title: decodeHtmlEntities(title),
employer: decodeHtmlEntities(
employerMatch?.[1]?.trim() ?? "Unknown Employer",
),
datePosted: dateMatch?.[1]?.trim(),
location: locationMatch?.[1]?.trim(),
jobType: typeMatch?.[1]?.trim(),
});
}
return jobs;
}
function parseInitialState(html: string): TestDevJobState | null {
const match = html.match(/window\.__INITIAL_STATE__=({[\s\S]*?});/);
if (!match?.[1]) return null;
try {
const parsed = JSON.parse(match[1]) as {
data?: { job?: TestDevJobState };
};
return parsed.data?.job ?? null;
} catch {
return null;
}
}
function matchesTerm(values: string[], term: string): boolean {
const lower = term.toLowerCase();
return values.some((value) => value.toLowerCase().includes(lower));
}
async function fetchText(url: string): Promise<string> {
const response = await fetch(url, {
headers: { Accept: "text/html", "User-Agent": "JobOps/1.0" },
});
if (!response.ok) {
throw new Error(
`TestDevJobs request failed (${response.status}) for ${url}`,
);
}
return response.text();
}
function listingUrl(page: number): string {
if (page <= 1) return `${ORIGIN}${LIST_PATH}`;
return `${ORIGIN}${LIST_PATH}${page}/`;
}
export const manifest: ExtractorManifest = {
id: "testdevjobs",
displayName: "TestDevJobs",
providesSources: ["testdevjobs"],
async run(context): Promise<ExtractorRunResult> {
if (context.shouldCancel?.()) return { success: true, jobs: [] };
const maxJobs = context.settings.testdevjobsMaxJobsPerTerm
? Number.parseInt(context.settings.testdevjobsMaxJobsPerTerm, 10)
: 100;
const cap = Number.isFinite(maxJobs)
? Math.min(Math.max(maxJobs, 1), 500)
: 100;
const maxPages = readMaxPages(context.settings.testdevjobsMaxPages);
const terms = context.searchTerms.length > 0 ? context.searchTerms : [];
const enrichDetails = true;
const seen = new Set<string>();
const out: CreateJobInput[] = [];
try {
for (let page = 1; page <= maxPages; page += 1) {
if (context.shouldCancel?.()) break;
if (out.length >= cap) break;
const url = listingUrl(page);
context.onProgress?.({
phase: "list",
termsProcessed: page - 1,
termsTotal: maxPages,
currentUrl: url,
detail: `TestDevJobs: listing page ${page}/${maxPages}`,
});
const html = await fetchText(url);
const listings = parseListingJobs(html);
if (listings.length === 0) break;
for (const listing of listings) {
if (context.shouldCancel?.()) break;
if (out.length >= cap) break;
const haystack = [
listing.title,
listing.employer,
listing.location ?? "",
listing.jobType ?? "",
];
if (
terms.length > 0 &&
!terms.some((term) => matchesTerm(haystack, term))
) {
continue;
}
let jobUrl = `${ORIGIN}${listing.path}`;
let applicationLink = jobUrl;
let jobDescription: string | undefined;
let location = listing.location;
let jobType = listing.jobType;
let datePosted = listing.datePosted;
let isRemote = listing.location?.toLowerCase().includes("remote");
if (enrichDetails) {
try {
const detailHtml = await fetchText(jobUrl);
const state = parseInitialState(detailHtml);
if (state) {
if (state.path) jobUrl = `${ORIGIN}${state.path}`;
if (state.applyLink) applicationLink = state.applyLink;
if (state.jobDescription) jobDescription = state.jobDescription;
if (state.joblocation) location = state.joblocation;
if (state.jobType) jobType = state.jobType;
if (state.jobPosted) datePosted = state.jobPosted;
if (typeof state.isRemote === "boolean")
isRemote = state.isRemote;
}
} catch {
// keep listing row when detail fetch fails
}
}
const key = listing.path;
if (seen.has(key)) continue;
seen.add(key);
out.push({
source: "testdevjobs",
sourceJobId: key.split("/").filter(Boolean).pop(),
title: listing.title,
employer: listing.employer,
jobUrl,
applicationLink,
location: location ?? "Unknown",
isRemote,
datePosted,
jobDescription,
jobType,
});
}
}
context.onProgress?.({
phase: "list",
termsProcessed: maxPages,
termsTotal: maxPages,
currentUrl: `${ORIGIN}${LIST_PATH}`,
jobPagesProcessed: out.length,
detail: `TestDevJobs: ${out.length} jobs`,
});
return { success: true, jobs: out };
} catch (error) {
const message = error instanceof Error ? error.message : "Unknown error";
return { success: false, jobs: out, error: message };
}
},
};
export default manifest;
-17
View File
@@ -1,17 +0,0 @@
{
"name": "testdevjobs-extractor",
"version": "0.0.1",
"type": "module",
"description": "TestDevJobs QA board extractor",
"main": "manifest.ts",
"dependencies": {
"job-ops-shared": "^1.0.0"
},
"devDependencies": {
"@types/node": "^24.0.0",
"typescript": "~5.9.0"
},
"scripts": {
"check:types": "tsc --noEmit"
}
}
-17
View File
@@ -1,17 +0,0 @@
{
"compilerOptions": {
"module": "ESNext",
"moduleResolution": "bundler",
"target": "ES2022",
"outDir": "dist",
"strict": true,
"noUnusedLocals": false,
"lib": ["ES2022", "DOM"],
"types": ["node"],
"baseUrl": ".",
"paths": {
"@shared/*": ["../../shared/src/*"]
}
},
"include": ["./manifest.ts", "./src/**/*"]
}
-49
View File
@@ -1,49 +0,0 @@
import type {
ExtractorManifest,
ExtractorRunResult,
} from "@shared/types/extractors";
import { runWellfound } from "./src/run.js";
export const manifest: ExtractorManifest = {
id: "wellfound",
displayName: "Wellfound",
providesSources: ["wellfound"],
async run(context): Promise<ExtractorRunResult> {
if (context.shouldCancel?.()) return { success: true, jobs: [] };
const parsedMax = context.settings.wellfoundMaxJobsPerTerm
? Number.parseInt(context.settings.wellfoundMaxJobsPerTerm, 10)
: Number.NaN;
const maxJobs = Number.isFinite(parsedMax) ? Math.max(1, parsedMax) : 50;
context.onProgress?.({
phase: "list",
termsProcessed: 0,
termsTotal: 1,
currentUrl: "https://wellfound.com/jobs",
detail: "Wellfound: launching browser scrape",
});
const result = await runWellfound({
searchTerms: context.searchTerms,
maxJobs,
});
if (!result.success) {
return { success: false, jobs: [], error: result.error };
}
context.onProgress?.({
phase: "list",
termsProcessed: 1,
termsTotal: 1,
currentUrl: "https://wellfound.com/jobs",
jobPagesProcessed: result.jobs.length,
detail: `Wellfound: ${result.jobs.length} jobs`,
});
return { success: true, jobs: result.jobs };
},
};
export default manifest;
-22
View File
@@ -1,22 +0,0 @@
{
"name": "wellfound-extractor",
"version": "0.0.1",
"type": "module",
"description": "Wellfound startup job board extractor (browser-backed)",
"main": "manifest.ts",
"dependencies": {
"camoufox-js": "^0.8.0",
"job-ops-shared": "^1.0.0",
"playwright": "^1.57.0",
"tsx": "^4.4.0"
},
"devDependencies": {
"@types/node": "^24.0.0",
"typescript": "~5.9.0"
},
"scripts": {
"start": "tsx src/main.ts",
"check:types": "tsc --noEmit",
"get-binaries": "camoufox-js fetch"
}
}
-171
View File
@@ -1,171 +0,0 @@
import { mkdir, writeFile } from "node:fs/promises";
import { dirname, join } from "node:path";
import { fileURLToPath } from "node:url";
import { launchOptions } from "camoufox-js";
import { firefox } from "playwright";
const __dirname = dirname(fileURLToPath(import.meta.url));
const OUTPUT_PATH = join(__dirname, "../storage/jobs.json");
interface ScrapedJob {
title: string;
employer: string;
jobUrl: string;
location?: string;
isRemote?: boolean;
}
function slugifyTerm(term: string): string {
return term
.trim()
.toLowerCase()
.replace(/[^a-z0-9]+/g, "-")
.replace(/^-+|-+$/g, "");
}
function parseTerms(raw: string | undefined): string[] {
if (!raw) return ["software-engineer"];
try {
const parsed = JSON.parse(raw) as unknown;
if (Array.isArray(parsed)) {
return parsed.map((entry) => slugifyTerm(String(entry))).filter(Boolean);
}
} catch {
// fall through
}
return raw
.split(/[\n|,]+/)
.map((entry) => slugifyTerm(entry))
.filter(Boolean);
}
function buildSearchUrl(roleSlug: string): string {
return `https://wellfound.com/role/l/${encodeURIComponent(roleSlug)}/remote`;
}
async function scrapeRole(
roleSlug: string,
maxJobs: number,
): Promise<ScrapedJob[]> {
const browser = await firefox.launch(await launchOptions({ headless: true }));
const page = await browser.newPage();
try {
await page.goto(buildSearchUrl(roleSlug), {
waitUntil: "domcontentloaded",
timeout: 90_000,
});
await page.waitForTimeout(4_000);
const jobs = await page.evaluate(() => {
const results: Array<{
title: string;
employer: string;
jobUrl: string;
location?: string;
}> = [];
const scripts = Array.from(
document.querySelectorAll('script[type="application/ld+json"]'),
);
for (const script of scripts) {
try {
const data = JSON.parse(script.textContent ?? "null") as unknown;
const rows = Array.isArray(data) ? data : [data];
for (const row of rows) {
if (!row || typeof row !== "object") continue;
const record = row as Record<string, unknown>;
if (record["@type"] !== "JobPosting") continue;
const title = String(record.title ?? "").trim();
const jobUrl = String(record.url ?? record.sameAs ?? "").trim();
const org = record.hiringOrganization as
| Record<string, unknown>
| undefined;
const employer = String(org?.name ?? "Unknown Employer").trim();
const location = String(
(record.jobLocation as Record<string, unknown> | undefined)
?.name ??
record.jobLocation ??
"",
).trim();
if (!title || !jobUrl) continue;
results.push({
title,
employer,
jobUrl,
location: location || undefined,
});
}
} catch {
// ignore malformed JSON-LD blocks
}
}
if (results.length === 0) {
for (const anchor of Array.from(
document.querySelectorAll('a[href*="/job/"], a[href*="/jobs/"]'),
)) {
const href = anchor.getAttribute("href") ?? "";
if (!href.includes("/job")) continue;
const jobUrl = href.startsWith("http")
? href
: `https://wellfound.com${href.startsWith("/") ? href : `/${href}`}`;
const title = (anchor.textContent ?? "").trim();
if (!title || title.length < 4) continue;
results.push({
title,
employer: "Unknown Employer",
jobUrl,
});
}
}
return results;
});
const seen = new Set<string>();
const out: ScrapedJob[] = [];
for (const job of jobs) {
if (out.length >= maxJobs) break;
if (seen.has(job.jobUrl)) continue;
seen.add(job.jobUrl);
out.push({
...job,
isRemote: true,
});
}
return out;
} finally {
await browser.close();
}
}
async function main(): Promise<void> {
const terms = parseTerms(process.env.WELLFOUND_SEARCH_TERMS);
const maxJobs = Number.parseInt(process.env.WELLFOUND_MAX_JOBS ?? "50", 10);
const cap = Number.isFinite(maxJobs)
? Math.max(1, Math.min(maxJobs, 200))
: 50;
const all: ScrapedJob[] = [];
const seen = new Set<string>();
for (const term of terms) {
const rows = await scrapeRole(term, cap);
for (const row of rows) {
if (all.length >= cap) break;
if (seen.has(row.jobUrl)) continue;
seen.add(row.jobUrl);
all.push(row);
}
}
await mkdir(dirname(OUTPUT_PATH), { recursive: true });
await writeFile(OUTPUT_PATH, JSON.stringify(all, null, 2), "utf-8");
}
main().catch((error) => {
console.error(error instanceof Error ? error.message : String(error));
process.exit(1);
});
-139
View File
@@ -1,139 +0,0 @@
import { spawn, spawnSync } from "node:child_process";
import { readFile } from "node:fs/promises";
import { createRequire } from "node:module";
import { dirname, join } from "node:path";
import { fileURLToPath } from "node:url";
import { envForExtractorSubprocess } from "@shared/extractor-subprocess-env.js";
import type { CreateJobInput } from "@shared/types/jobs";
const srcDir = dirname(fileURLToPath(import.meta.url));
const EXTRACTOR_DIR = join(srcDir, "..");
const OUTPUT_PATH = join(EXTRACTOR_DIR, "storage/jobs.json");
const require = createRequire(import.meta.url);
const TSX_CLI_PATH = (() => {
try {
return require.resolve("tsx/dist/cli.mjs");
} catch {
return null;
}
})();
interface ScrapedJob {
title?: string;
employer?: string;
jobUrl?: string;
location?: string;
isRemote?: boolean;
}
export interface RunWellfoundOptions {
searchTerms?: string[];
maxJobs?: number;
}
export interface WellfoundResult {
success: boolean;
jobs: CreateJobInput[];
error?: string;
}
function canRunNpmCommand(): boolean {
const result = spawnSync("npm", ["--version"], { stdio: "ignore" });
return !result.error && result.status === 0;
}
function mapJob(row: ScrapedJob): CreateJobInput | null {
const jobUrl = row.jobUrl?.trim();
if (!jobUrl) return null;
return {
source: "wellfound",
sourceJobId: jobUrl.split("/").filter(Boolean).pop(),
title: row.title?.trim() || "Unknown Title",
employer: row.employer?.trim() || "Unknown Employer",
jobUrl,
applicationLink: jobUrl,
location: row.location?.trim() || "Remote",
isRemote: row.isRemote ?? true,
};
}
export async function runWellfound(
options: RunWellfoundOptions = {},
): Promise<WellfoundResult> {
const searchTerms =
options.searchTerms && options.searchTerms.length > 0
? options.searchTerms
: ["software engineer"];
const maxJobs = options.maxJobs ?? 50;
const useNpmCommand = canRunNpmCommand();
if (!TSX_CLI_PATH && !useNpmCommand) {
return {
success: false,
jobs: [],
error: "Unable to execute Wellfound extractor (npm/tsx unavailable)",
};
}
try {
await new Promise<void>((resolve, reject) => {
const extractorEnv = envForExtractorSubprocess({
...process.env,
WELLFOUND_SEARCH_TERMS: JSON.stringify(searchTerms),
WELLFOUND_MAX_JOBS: String(maxJobs),
});
const child = TSX_CLI_PATH
? spawn(process.execPath, [TSX_CLI_PATH, join(srcDir, "main.ts")], {
cwd: EXTRACTOR_DIR,
env: extractorEnv,
stdio: ["ignore", "pipe", "pipe"],
})
: spawn("npx", ["tsx", join(srcDir, "main.ts")], {
cwd: EXTRACTOR_DIR,
env: extractorEnv,
stdio: ["ignore", "pipe", "pipe"],
});
let stderr = "";
child.stderr?.on("data", (chunk: Buffer) => {
stderr += chunk.toString();
});
child.on("error", reject);
child.on("close", (code) => {
if (code === 0) {
resolve();
return;
}
reject(
new Error(
stderr.trim() ||
`Wellfound browser scrape exited with code ${code}`,
),
);
});
});
const raw = await readFile(OUTPUT_PATH, "utf-8");
const parsed = JSON.parse(raw) as unknown;
const rows = Array.isArray(parsed) ? parsed : [];
const jobs: CreateJobInput[] = [];
const seen = new Set<string>();
for (const value of rows) {
if (!value || typeof value !== "object" || Array.isArray(value)) continue;
const mapped = mapJob(value as ScrapedJob);
if (!mapped) continue;
const key = mapped.jobUrl;
if (seen.has(key)) continue;
seen.add(key);
jobs.push(mapped);
}
return { success: true, jobs };
} catch (error) {
const message = error instanceof Error ? error.message : "Unknown error";
return { success: false, jobs: [], error: message };
}
}
-17
View File
@@ -1,17 +0,0 @@
{
"compilerOptions": {
"module": "ESNext",
"moduleResolution": "bundler",
"target": "ES2022",
"outDir": "dist",
"strict": true,
"noUnusedLocals": false,
"lib": ["ES2022", "DOM"],
"types": ["node"],
"baseUrl": ".",
"paths": {
"@shared/*": ["../../shared/src/*"]
}
},
"include": ["./manifest.ts", "./src/**/*"]
}
-149
View File
@@ -1,149 +0,0 @@
/**
* Working Nomads — public JSON feed.
*
* https://www.workingnomads.com/api/exposed_jobs/
*/
import type {
ExtractorManifest,
ExtractorRunResult,
} from "@shared/types/extractors";
import type { CreateJobInput } from "@shared/types/jobs";
const API_URL = "https://www.workingnomads.com/api/exposed_jobs/";
interface WorkingNomadsJob {
url?: string;
title?: string;
description?: string;
company_name?: string;
category_name?: string;
tags?: string;
location?: string;
pub_date?: string;
}
function asString(value: unknown): string | undefined {
if (typeof value !== "string") return undefined;
const trimmed = value.trim();
return trimmed ? trimmed : undefined;
}
function stripHtml(html: string): string {
return html
.replace(/<[^>]+>/g, " ")
.replace(/&nbsp;/g, " ")
.replace(/&amp;/g, "&")
.replace(/&lt;/g, "<")
.replace(/&gt;/g, ">")
.replace(/\s+/g, " ")
.trim();
}
function matchesTerm(job: WorkingNomadsJob, term: string): boolean {
const lower = term.toLowerCase();
if (job.title?.toLowerCase().includes(lower)) return true;
if (job.company_name?.toLowerCase().includes(lower)) return true;
if (job.category_name?.toLowerCase().includes(lower)) return true;
if (job.tags?.toLowerCase().includes(lower)) return true;
const description = job.description ? stripHtml(job.description) : "";
if (description.toLowerCase().includes(lower)) return true;
return false;
}
function mapJob(raw: WorkingNomadsJob): CreateJobInput | null {
const jobUrl = asString(raw.url);
if (!jobUrl) return null;
const title = asString(raw.title) ?? "Unknown Title";
const employer = asString(raw.company_name) ?? "Unknown Employer";
const location = asString(raw.location) ?? "Remote";
const description = raw.description ? stripHtml(raw.description) : undefined;
return {
source: "workingnomads",
sourceJobId: jobUrl.split("/").filter(Boolean).pop(),
title,
employer,
jobUrl,
applicationLink: jobUrl,
location,
isRemote: true,
datePosted: asString(raw.pub_date),
jobDescription: description,
companyIndustry: asString(raw.category_name),
disciplines: asString(raw.tags),
};
}
export const manifest: ExtractorManifest = {
id: "workingnomads",
displayName: "Working Nomads",
providesSources: ["workingnomads"],
async run(context): Promise<ExtractorRunResult> {
if (context.shouldCancel?.()) return { success: true, jobs: [] };
const maxJobs = context.settings.workingnomadsMaxJobsPerTerm
? Number.parseInt(context.settings.workingnomadsMaxJobsPerTerm, 10)
: 100;
const cap = Number.isFinite(maxJobs)
? Math.min(Math.max(maxJobs, 1), 500)
: 100;
const terms = context.searchTerms.length > 0 ? context.searchTerms : [];
context.onProgress?.({
phase: "list",
termsProcessed: 0,
termsTotal: 1,
currentUrl: API_URL,
detail: "Working Nomads: fetching exposed_jobs API",
});
try {
const response = await fetch(API_URL, {
headers: { Accept: "application/json", "User-Agent": "JobOps/1.0" },
});
if (!response.ok) {
throw new Error(
`Working Nomads request failed with status ${response.status}`,
);
}
const body = (await response.json()) as unknown;
const rows = Array.isArray(body) ? body : [];
const seen = new Set<string>();
const out: CreateJobInput[] = [];
for (const row of rows as WorkingNomadsJob[]) {
if (out.length >= cap) break;
if (terms.length > 0 && !terms.some((term) => matchesTerm(row, term))) {
continue;
}
const mapped = mapJob(row);
if (!mapped) continue;
const key = mapped.sourceJobId || mapped.jobUrl;
if (seen.has(key)) continue;
seen.add(key);
out.push(mapped);
}
context.onProgress?.({
phase: "list",
termsProcessed: 1,
termsTotal: 1,
currentUrl: API_URL,
jobPagesProcessed: out.length,
detail: `Working Nomads: ${out.length} matched (${rows.length} total)`,
});
return { success: true, jobs: out };
} catch (error) {
const message = error instanceof Error ? error.message : "Unknown error";
return { success: false, jobs: [], error: message };
}
},
};
export default manifest;
-17
View File
@@ -1,17 +0,0 @@
{
"name": "workingnomads-extractor",
"version": "0.0.1",
"type": "module",
"description": "Working Nomads public JSON feed extractor",
"main": "manifest.ts",
"dependencies": {
"job-ops-shared": "^1.0.0"
},
"devDependencies": {
"@types/node": "^24.0.0",
"typescript": "~5.9.0"
},
"scripts": {
"check:types": "tsc --noEmit"
}
}
-17
View File
@@ -1,17 +0,0 @@
{
"compilerOptions": {
"module": "ESNext",
"moduleResolution": "bundler",
"target": "ES2022",
"outDir": "dist",
"strict": true,
"noUnusedLocals": false,
"lib": ["ES2022", "DOM"],
"types": ["node"],
"baseUrl": ".",
"paths": {
"@shared/*": ["../../shared/src/*"]
}
},
"include": ["./manifest.ts", "./src/**/*"]
}
+1
View File
@@ -9,6 +9,7 @@
<link rel="preconnect" href="https://fonts.googleapis.com">
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
<link href="https://fonts.googleapis.com/css2?family=Inter:wght@400;500;600;700&display=swap" rel="stylesheet">
<script defer src="https://umami.dakheera47.com/script.js" data-website-id="0dc42ed1-87c3-4ac0-9409-5a9b9588fe66"></script>
</head>
<body>
<div id="root"></div>
+3 -3
View File
@@ -91,7 +91,7 @@ describe("App demo banner", () => {
});
render(
<MemoryRouter initialEntries={["/jobs/discovered"]}>
<MemoryRouter initialEntries={["/overview"]}>
<App />
</MemoryRouter>,
);
@@ -114,7 +114,7 @@ describe("App demo banner", () => {
});
render(
<MemoryRouter initialEntries={["/jobs/discovered"]}>
<MemoryRouter initialEntries={["/overview"]}>
<App />
</MemoryRouter>,
);
@@ -135,7 +135,7 @@ describe("App demo banner", () => {
});
render(
<MemoryRouter initialEntries={["/jobs/discovered"]}>
<MemoryRouter initialEntries={["/overview"]}>
<App />
</MemoryRouter>,
);
+23 -14
View File
@@ -14,31 +14,29 @@ import { BasicAuthPrompt } from "./components/BasicAuthPrompt";
import { OnboardingGate } from "./components/OnboardingGate";
import { useDemoInfo } from "./hooks/useDemoInfo";
import { GmailOauthCallbackPage } from "./pages/GmailOauthCallbackPage";
import { HomePage } from "./pages/HomePage";
import { InProgressBoardPage } from "./pages/InProgressBoardPage";
import { JobPage } from "./pages/JobPage";
import { OrchestratorPage } from "./pages/OrchestratorPage";
import { SettingsPage } from "./pages/SettingsPage";
import { TracerLinksPage } from "./pages/TracerLinksPage";
import { TrackingInboxPage } from "./pages/TrackingInboxPage";
import { VisaSponsorsPage } from "./pages/VisaSponsorsPage";
/** Backwards-compatibility redirects: old URL paths -> new URL paths */
const REDIRECTS: Array<{ from: string; to: string }> = [
{ from: "/", to: "/jobs/discovered" },
{ from: "/home", to: "/jobs/discovered" },
{ from: "/overview", to: "/jobs/discovered" },
{ from: "/", to: "/jobs/ready" },
{ from: "/home", to: "/overview" },
{ from: "/ready", to: "/jobs/ready" },
{ from: "/ready/:jobId", to: "/jobs/ready/:jobId" },
{ from: "/discovered", to: "/jobs/discovered" },
{ from: "/discovered/:jobId", to: "/jobs/discovered/:jobId" },
{ from: "/applied", to: "/jobs/applied" },
{ from: "/applied/:jobId", to: "/jobs/applied/:jobId" },
// Slim product: kanban / inbox / tracer / visa browser redirect into Jobs.
{ from: "/in-progress", to: "/jobs/discovered" },
{ from: "/in-progress/:jobId", to: "/jobs/discovered" },
{ from: "/jobs/in_progress", to: "/jobs/discovered" },
{ from: "/jobs/in_progress/:jobId", to: "/jobs/discovered" },
{ from: "/applications/in-progress", to: "/jobs/discovered" },
{ from: "/applications/in-progress/:jobId", to: "/jobs/discovered" },
{ from: "/tracking-inbox", to: "/jobs/discovered" },
{ from: "/tracer-links", to: "/jobs/discovered" },
{ from: "/visa-sponsors", to: "/jobs/discovered" },
{ from: "/in-progress", to: "/applications/in-progress" },
{ from: "/in-progress/:jobId", to: "/applications/in-progress" },
{ from: "/jobs/in_progress", to: "/applications/in-progress" },
{ from: "/jobs/in_progress/:jobId", to: "/applications/in-progress" },
{ from: "/all", to: "/jobs/all" },
{ from: "/all/:jobId", to: "/jobs/all/:jobId" },
];
@@ -136,13 +134,24 @@ export const App: React.FC = () => {
/>
))}
{/* Application routes — slim product: Jobs + Settings */}
{/* Application routes */}
<Route path="/overview" element={<HomePage />} />
<Route
path="/oauth/gmail/callback"
element={<GmailOauthCallbackPage />}
/>
<Route path="/job/:id" element={<JobPage />} />
<Route
path="/applications/in-progress"
element={<InProgressBoardPage />}
/>
<Route path="/settings" element={<SettingsPage />} />
<Route path="/tracer-links" element={<TracerLinksPage />} />
<Route path="/visa-sponsors" element={<VisaSponsorsPage />} />
<Route
path="/tracking-inbox"
element={<TrackingInboxPage />}
/>
<Route path="/jobs/:tab" element={<OrchestratorPage />} />
<Route
path="/jobs/:tab/:jobId"
+55 -73
View File
@@ -25,7 +25,6 @@ import type {
JobsListResponse,
JobsRevisionResponse,
JobTracerLinksResponse,
KeywordSet,
ManualJobDraft,
ManualJobFetchResponse,
ManualJobInferenceResponse,
@@ -40,6 +39,7 @@ import type {
ProfileStatusResponse,
ResumeProfile,
ResumeProjectCatalogItem,
RxResumeMode,
SearchProfile,
StageEvent,
StageEventMetadata,
@@ -393,16 +393,11 @@ export async function activateSearchProfileForBasicAuthUser(
return;
}
if (!listPayload.ok || !Array.isArray(listPayload.data)) return;
const normalizedUsername = username.trim().toLowerCase();
const matches = listPayload.data.filter((row) => {
const aliases = (row.data?.basicAuthUser ?? "")
.split(",")
.map((part) => part.trim().toLowerCase())
.filter(Boolean);
return aliases.includes(normalizedUsername);
});
if (matches.length !== 1) return;
await fetch(`${API_BASE}/profiles/${matches[0].id}/activate`, {
const match = listPayload.data.find(
(row) => (row.data?.basicAuthUser ?? "").trim() === username.trim(),
);
if (!match) return;
await fetch(`${API_BASE}/profiles/${match.id}/activate`, {
method: "POST",
headers: {
"Content-Type": "application/json",
@@ -1538,6 +1533,19 @@ export async function getProfileProjects(): Promise<
export async function getResumeProjectsCatalog(): Promise<
ResumeProjectCatalogItem[]
> {
try {
const settings = await getSettings();
if (settings.rxresumeBaseResumeId) {
return await getRxResumeProjects(
settings.rxresumeBaseResumeId,
undefined,
settings.rxresumeMode?.value,
);
}
} catch {
// fall through to profile-based projects
}
return getProfileProjects();
}
@@ -1578,6 +1586,19 @@ export async function getLlmModels(input?: {
return data.models;
}
export async function validateRxresume(input?: {
mode?: "v4" | "v5";
email?: string;
password?: string;
apiKey?: string;
baseUrl?: string;
}): Promise<ValidationResult> {
return fetchApi<ValidationResult>("/onboarding/validate/rxresume", {
method: "POST",
body: JSON.stringify(input ?? {}),
});
}
export async function validateResumeConfig(): Promise<ValidationResult> {
return fetchApi<ValidationResult>("/onboarding/validate/resume");
}
@@ -1591,6 +1612,29 @@ export async function updateSettings(
});
}
export async function getRxResumes(
mode?: RxResumeMode,
): Promise<{ id: string; name: string }[]> {
const query = mode ? `?mode=${encodeURIComponent(mode)}` : "";
const data = await fetchApi<{ resumes: { id: string; name: string }[] }>(
`/settings/rx-resumes${query}`,
);
return data.resumes;
}
export async function getRxResumeProjects(
resumeId: string,
signal?: AbortSignal,
mode?: RxResumeMode,
): Promise<ResumeProjectCatalogItem[]> {
const query = mode ? `?mode=${encodeURIComponent(mode)}` : "";
const data = await fetchApi<{ projects: ResumeProjectCatalogItem[] }>(
`/settings/rx-resumes/${encodeURIComponent(resumeId)}/projects${query}`,
{ signal },
);
return data.projects;
}
// Database API
export async function clearDatabase(): Promise<{
message: string;
@@ -1618,30 +1662,6 @@ export async function deleteJobsByStatus(status: string): Promise<{
});
}
export async function deleteDiscoveredMatchingArchived(): Promise<{
message: string;
count: number;
}> {
return fetchApi<{
message: string;
count: number;
}>("/jobs/duplicates/discovered", {
method: "DELETE",
});
}
export async function expireDeadListings(): Promise<{
message: string;
count: number;
}> {
return fetchApi<{
message: string;
count: number;
}>("/jobs/expire-dead-listings", {
method: "POST",
});
}
export async function deleteJobsBelowScore(threshold: number): Promise<{
message: string;
count: number;
@@ -1770,41 +1790,3 @@ export async function generateProfileFromResume(): Promise<JobSearchProfile> {
method: "POST",
});
}
// Keyword sets API
export async function listKeywordSets(): Promise<KeywordSet[]> {
return fetchApi<KeywordSet[]>("/keyword-sets");
}
export async function createKeywordSet(input: {
name: string;
terms?: string[];
}): Promise<KeywordSet> {
return fetchApi<KeywordSet>("/keyword-sets", {
method: "POST",
body: JSON.stringify(input),
});
}
export async function updateKeywordSet(
id: string,
input: { name?: string; terms?: string[] },
): Promise<KeywordSet> {
return fetchApi<KeywordSet>(`/keyword-sets/${encodeURIComponent(id)}`, {
method: "PATCH",
body: JSON.stringify(input),
});
}
export async function deleteKeywordSet(id: string): Promise<void> {
await fetchApi<void>(`/keyword-sets/${encodeURIComponent(id)}`, {
method: "DELETE",
});
}
export async function activateKeywordSet(id: string): Promise<KeywordSet> {
return fetchApi<KeywordSet>(
`/keyword-sets/${encodeURIComponent(id)}/activate`,
{ method: "POST" },
);
}
@@ -20,7 +20,6 @@ import {
} from "@/components/ui/tooltip";
import { cn, formatDate, sourceLabel } from "@/lib/utils";
import { useSettings } from "../hooks/useSettings";
import { SponsorshipSignalsPills } from "./SponsorshipSignalsPills";
import {
getJobStatusIndicator,
getTracerStatusIndicator,
@@ -256,10 +255,6 @@ export const JobHeader: React.FC<JobHeaderProps> = ({
onCheck={onCheckSponsor}
/>
)}
<SponsorshipSignalsPills
sponsorshipSignals={job.sponsorshipSignals}
size="xs"
/>
</div>
<ScoreMeter score={job.suitabilityScore} />
</div>
@@ -1,6 +1,6 @@
import * as api from "@client/api";
import { useSettings } from "@client/hooks/useSettings";
import { screen, waitFor } from "@testing-library/react";
import { fireEvent, screen, waitFor } from "@testing-library/react";
import type React from "react";
import { beforeEach, describe, expect, it, vi } from "vitest";
import { renderWithQueryClient } from "../test/renderWithQueryClient";
@@ -12,6 +12,8 @@ const render = (ui: Parameters<typeof renderWithQueryClient>[0]) =>
vi.mock("@client/api", () => ({
getDemoInfo: vi.fn(),
validateLlm: vi.fn(),
validateRxresume: vi.fn(),
validateResumeConfig: vi.fn(),
updateSettings: vi.fn(),
}));
@@ -34,6 +36,10 @@ vi.mock("@client/pages/settings/components/SettingsInput", () => ({
),
}));
vi.mock("@client/pages/settings/components/BaseResumeSelection", () => ({
BaseResumeSelection: () => <div>Base resume selection</div>,
}));
vi.mock("@/components/ui/alert-dialog", () => ({
AlertDialog: ({ children }: { children: React.ReactNode }) => (
<div>{children}</div>
@@ -52,6 +58,19 @@ vi.mock("@/components/ui/alert-dialog", () => ({
),
}));
vi.mock("@/components/ui/tabs", () => ({
Tabs: ({ children }: { children: React.ReactNode }) => <div>{children}</div>,
TabsContent: ({ children }: { children: React.ReactNode }) => (
<div>{children}</div>
),
TabsList: ({ children }: { children: React.ReactNode }) => (
<div>{children}</div>
),
TabsTrigger: ({ children }: { children: React.ReactNode }) => (
<button type="button">{children}</button>
),
}));
vi.mock("@/components/ui/select", () => ({
Select: ({ children }: { children: React.ReactNode }) => (
<div>{children}</div>
@@ -86,6 +105,11 @@ const settingsResponse = {
settings: {
llmProvider: { value: "openrouter", default: "openrouter", override: null },
llmApiKeyHint: null,
rxresumeEmail: "",
rxresumeUrl: "",
rxresumeApiKeyHint: null,
rxresumePasswordHint: null,
rxresumeBaseResumeId: null,
localResumeProfilePath: null,
localResumeFileConfigured: false,
},
@@ -107,11 +131,19 @@ describe("OnboardingGate", () => {
vi.mocked(useSettings).mockReturnValue(settingsResponse as any);
});
it("renders the gate when LLM validation fails", async () => {
it("renders the gate once validations complete and any fail", async () => {
vi.mocked(api.validateLlm).mockResolvedValue({
valid: false,
message: "Invalid",
});
vi.mocked(api.validateRxresume).mockResolvedValue({
valid: true,
message: null,
});
vi.mocked(api.validateResumeConfig).mockResolvedValue({
valid: true,
message: null,
});
render(<OnboardingGate />);
@@ -119,17 +151,28 @@ describe("OnboardingGate", () => {
await waitFor(() => {
expect(screen.getByText("Welcome to Job Ops")).toBeInTheDocument();
});
expect(
screen.queryByLabelText("Local resume path"),
).not.toBeInTheDocument();
expect(screen.queryByText("Resume file")).not.toBeInTheDocument();
});
it("hides the gate when LLM validation succeeds", async () => {
it("hides the gate when all validations succeed", async () => {
vi.mocked(useSettings).mockReturnValue({
...settingsResponse,
settings: {
...settingsResponse.settings,
rxresumeApiKeyHint: "abcd1234",
},
} as any);
vi.mocked(api.validateLlm).mockResolvedValue({
valid: true,
message: null,
});
vi.mocked(api.validateRxresume).mockResolvedValue({
valid: true,
message: null,
});
vi.mocked(api.validateResumeConfig).mockResolvedValue({
valid: true,
message: null,
});
render(<OnboardingGate />);
@@ -137,12 +180,13 @@ describe("OnboardingGate", () => {
expect(screen.queryByText("Welcome to Job Ops")).not.toBeInTheDocument();
});
it("hides the gate for providers without API keys", async () => {
it("hides the gate for Ollama when local resume file is configured on the server", async () => {
vi.mocked(useSettings).mockReturnValue({
...settingsResponse,
settings: {
...settingsResponse.settings,
llmProvider: { value: "ollama", default: "ollama", override: null },
localResumeFileConfigured: true,
},
} as any);
@@ -151,6 +195,79 @@ describe("OnboardingGate", () => {
await waitFor(() => {
expect(screen.queryByText("Welcome to Job Ops")).not.toBeInTheDocument();
});
expect(api.validateRxresume).not.toHaveBeenCalled();
expect(api.validateResumeConfig).not.toHaveBeenCalled();
});
it("skips LLM key validation for providers without API keys", async () => {
vi.mocked(useSettings).mockReturnValue({
...settingsResponse,
settings: {
...settingsResponse.settings,
llmProvider: { value: "ollama", default: "ollama", override: null },
},
} as any);
vi.mocked(api.validateRxresume).mockResolvedValue({
valid: false,
message: "Missing",
});
vi.mocked(api.validateResumeConfig).mockResolvedValue({
valid: true,
message: null,
});
render(<OnboardingGate />);
await waitFor(() => expect(api.validateResumeConfig).toHaveBeenCalled());
expect(api.validateLlm).not.toHaveBeenCalled();
expect(api.validateRxresume).not.toHaveBeenCalled();
await waitFor(() => {
expect(screen.getByText("Welcome to Job Ops")).toBeInTheDocument();
});
expect(screen.queryByText("LLM API key")).not.toBeInTheDocument();
});
it("renders the RxResume URL field and includes it in validation", async () => {
vi.mocked(useSettings).mockReturnValue({
...settingsResponse,
settings: {
...settingsResponse.settings,
rxresumeUrl: "https://resume.example.com",
rxresumeApiKeyHint: "abcd1234",
},
} as any);
vi.mocked(api.validateLlm).mockResolvedValue({
valid: false,
message: "Invalid",
});
vi.mocked(api.validateRxresume).mockResolvedValue({
valid: true,
message: null,
});
vi.mocked(api.validateResumeConfig).mockResolvedValue({
valid: true,
message: null,
});
render(<OnboardingGate />);
await waitFor(() =>
expect(screen.getByLabelText("RxResume URL")).toBeInTheDocument(),
);
await waitFor(() =>
expect(api.validateRxresume).toHaveBeenCalledWith(
expect.objectContaining({
baseUrl: "https://resume.example.com",
}),
),
);
fireEvent.change(screen.getByLabelText("RxResume URL"), {
target: { value: "https://self-hosted.example.com" },
});
expect(
screen.getByDisplayValue("https://self-hosted.example.com"),
).toBeInTheDocument();
});
});
@@ -1,6 +1,15 @@
import * as api from "@client/api";
import { ReactiveResumeConfigPanel } from "@client/components/ReactiveResumeConfigPanel";
import { useDemoInfo } from "@client/hooks/useDemoInfo";
import { useRxResumeConfigState } from "@client/hooks/useRxResumeConfigState";
import { useSettings } from "@client/hooks/useSettings";
import {
getInitialRxResumeMode,
getRxResumeCredentialDrafts,
getRxResumeMissingCredentialLabels,
validateAndMaybePersistRxResumeMode,
} from "@client/lib/rxresume-config";
import { BaseResumeSelection } from "@client/pages/settings/components/BaseResumeSelection";
import { SettingsInput } from "@client/pages/settings/components/SettingsInput";
import {
getLlmProviderConfig,
@@ -10,10 +19,10 @@ import {
} from "@client/pages/settings/utils";
import { getDefaultModelForProvider } from "@shared/settings-registry";
import type { UpdateSettingsInput } from "@shared/settings-schema.js";
import type { ValidationResult } from "@shared/types.js";
import type { RxResumeMode, ValidationResult } from "@shared/types.js";
import { Check } from "lucide-react";
import type React from "react";
import { useCallback, useEffect, useState } from "react";
import { useCallback, useEffect, useMemo, useState } from "react";
import { Controller, useForm } from "react-hook-form";
import { toast } from "sonner";
import {
@@ -39,14 +48,22 @@ import {
SelectTrigger,
SelectValue,
} from "@/components/ui/select";
import { Tabs, TabsContent, TabsList, TabsTrigger } from "@/components/ui/tabs";
import { cn } from "@/lib/utils";
type ValidationState = ValidationResult & { checked: boolean };
type TimestampedValidationState = ValidationState & { testedAt: number | null };
type OnboardingFormData = {
llmProvider: string;
llmBaseUrl: string;
llmApiKey: string;
rxresumeMode: RxResumeMode;
rxresumeEmail: string;
rxresumeUrl: string;
rxresumePassword: string;
rxresumeApiKey: string;
rxresumeBaseResumeId: string | null;
};
const EMPTY_VALIDATION_STATE: ValidationState = {
@@ -55,6 +72,27 @@ const EMPTY_VALIDATION_STATE: ValidationState = {
checked: false,
};
const EMPTY_TIMESTAMPED_VALIDATION_STATE: TimestampedValidationState = {
...EMPTY_VALIDATION_STATE,
testedAt: null,
};
function getStepPrimaryLabel(input: {
currentStep: string | null;
llmValidated: boolean;
rxresumeValidated: boolean;
baseResumeValidated: boolean;
}): string {
const toLabel = (isValidated: boolean): string =>
isValidated ? "Revalidate" : "Validate";
if (input.currentStep === "llm") return toLabel(input.llmValidated);
if (input.currentStep === "rxresume") return toLabel(input.rxresumeValidated);
if (input.currentStep === "baseresume")
return toLabel(input.baseResumeValidated);
return "Validate";
}
export const OnboardingGate: React.FC = () => {
const {
settings,
@@ -62,11 +100,41 @@ export const OnboardingGate: React.FC = () => {
refreshSettings,
} = useSettings();
/** Skip RxResume onboarding when Vite flag is set, server reports a local resume file, or Settings has a local path. */
const skipRxResumeOnboarding = useMemo(
() =>
import.meta.env.VITE_SKIP_RXRESUME_ONBOARDING === "true" ||
Boolean(settings?.localResumeFileConfigured) ||
Boolean(settings?.localResumeProfilePath?.trim()),
[settings?.localResumeFileConfigured, settings?.localResumeProfilePath],
);
const {
storedRxResume,
getBaseResumeIdForMode,
setBaseResumeIdForMode,
syncBaseResumeIdsForMode,
} = useRxResumeConfigState(settings);
const [isSavingEnv, setIsSavingEnv] = useState(false);
const [isValidatingLlm, setIsValidatingLlm] = useState(false);
const [isValidatingRxresume, setIsValidatingRxresume] = useState(false);
const [isValidatingBaseResume, setIsValidatingBaseResume] = useState(false);
const [llmValidation, setLlmValidation] = useState<ValidationState>(
EMPTY_VALIDATION_STATE,
);
const [rxresumeValidation, setRxresumeValidation] = useState<ValidationState>(
EMPTY_VALIDATION_STATE,
);
const [rxresumeVersionValidations, setRxresumeVersionValidations] = useState<{
v4: TimestampedValidationState;
v5: TimestampedValidationState;
}>({
v4: EMPTY_TIMESTAMPED_VALIDATION_STATE,
v5: EMPTY_TIMESTAMPED_VALIDATION_STATE,
});
const [baseResumeValidation, setBaseResumeValidation] =
useState<ValidationState>(EMPTY_VALIDATION_STATE);
const [currentStep, setCurrentStep] = useState<string | null>(null);
const demoInfo = useDemoInfo();
const demoMode = demoInfo?.demoMode ?? false;
@@ -76,6 +144,12 @@ export const OnboardingGate: React.FC = () => {
llmProvider: "",
llmBaseUrl: "",
llmApiKey: "",
rxresumeMode: "v5",
rxresumeEmail: "",
rxresumeUrl: "",
rxresumePassword: "",
rxresumeApiKey: "",
rxresumeBaseResumeId: null,
},
});
@@ -113,6 +187,26 @@ export const OnboardingGate: React.FC = () => {
}
}, [getValues, settings?.llmProvider]);
const validateBaseResume = useCallback(async () => {
setIsValidatingBaseResume(true);
try {
const result = await api.validateResumeConfig();
setBaseResumeValidation({ ...result, checked: true });
return result;
} catch (error) {
const message =
error instanceof Error
? error.message
: "Base resume validation failed";
const result = { valid: false, message };
setBaseResumeValidation({ ...result, checked: true });
return result;
} finally {
setIsValidatingBaseResume(false);
}
}, []);
const rxresumeModeValue = watch("rxresumeMode");
const selectedProvider = normalizeLlmProvider(
llmProvider || settings?.llmProvider?.value || "openrouter",
);
@@ -126,68 +220,234 @@ export const OnboardingGate: React.FC = () => {
const llmKeyHint = settings?.llmApiKeyHint ?? null;
const hasLlmKey = Boolean(llmKeyHint);
const rxresumeModeCurrent = (rxresumeModeValue ||
settings?.rxresumeMode?.value ||
"v5") as RxResumeMode;
const hasCheckedValidations =
(requiresLlmKey ? llmValidation.checked : true) &&
(skipRxResumeOnboarding
? true
: rxresumeValidation.checked && baseResumeValidation.checked);
const llmValidated = requiresLlmKey ? llmValidation.valid : true;
const hasCheckedValidations = requiresLlmKey ? llmValidation.checked : true;
const shouldOpen =
!demoMode &&
Boolean(settings && !settingsLoading) &&
hasCheckedValidations &&
!llmValidated;
!(
llmValidated &&
(skipRxResumeOnboarding
? true
: rxresumeValidation.valid && baseResumeValidation.valid)
);
const validateRxresumeVersion = useCallback(
async (
version: "v4" | "v5",
): Promise<ValidationResult & { checked: true; testedAt: number }> => {
const values = getValues();
const draftCredentials = getRxResumeCredentialDrafts(values);
const testedAt = Date.now();
const result = await validateAndMaybePersistRxResumeMode({
mode: version,
stored: storedRxResume,
draft: draftCredentials,
validate: api.validateRxresume,
getPrecheckMessage: (failure) =>
failure === "missing-v5-api-key"
? "v5 API key required. Add a v5 API key, then test again."
: "v4 email and password required. Add both credentials, then test again.",
getValidationErrorMessage: (error, mode) =>
error instanceof Error
? error.message
: `RxResume ${mode} validation failed`,
});
return { ...result.validation, checked: true, testedAt };
},
[getValues, storedRxResume],
);
const validateRxresume = useCallback(async () => {
const values = getValues();
const selectedMode = values.rxresumeMode;
setIsValidatingRxresume(true);
try {
const versionResult = await validateRxresumeVersion(selectedMode);
setRxresumeVersionValidations((current) => ({
...current,
[selectedMode]: versionResult,
}));
const result: ValidationResult = {
valid: versionResult.valid,
message: versionResult.message,
};
setRxresumeValidation({ ...result, checked: true });
return result;
} finally {
setIsValidatingRxresume(false);
}
}, [getValues, validateRxresumeVersion]);
// Initialize form values from settings
useEffect(() => {
if (settings) {
const initialMode = getInitialRxResumeMode({
savedMode: (settings.rxresumeMode?.value ??
null) as RxResumeMode | null,
hasV4: storedRxResume.hasV4,
hasV5: storedRxResume.hasV5,
});
const selectedId = syncBaseResumeIdsForMode(initialMode);
reset({
llmProvider: settings.llmProvider?.value || "",
llmBaseUrl: settings.llmBaseUrl?.value || "",
llmApiKey: "",
rxresumeMode: initialMode,
rxresumeEmail: "",
rxresumeUrl: settings.rxresumeUrl ?? "",
rxresumePassword: "",
rxresumeApiKey: "",
rxresumeBaseResumeId: selectedId,
});
}
}, [settings, reset]);
}, [
settings,
reset,
storedRxResume.hasV4,
storedRxResume.hasV5,
syncBaseResumeIdsForMode,
]);
// Clear base URL when provider doesn't require it
useEffect(() => {
if (!showBaseUrl) {
setValue("llmBaseUrl", "");
}
}, [showBaseUrl, setValue]);
// Reset LLM validation when provider changes
useEffect(() => {
if (!selectedProvider) return;
setLlmValidation({ valid: false, message: null, checked: false });
}, [selectedProvider]);
const runLlmValidation = useCallback(async () => {
const steps = useMemo(
() =>
skipRxResumeOnboarding
? [
{
id: "llm",
label: "LLM Provider",
subtitle: "Provider + credentials",
complete: llmValidated,
disabled: false,
},
]
: [
{
id: "llm",
label: "LLM Provider",
subtitle: "Provider + credentials",
complete: llmValidated,
disabled: false,
},
{
id: "rxresume",
label: "Connect Reactive Resume",
subtitle: "Version + credentials",
complete: rxresumeValidation.valid,
disabled: false,
},
{
id: "baseresume",
label: "Select Template Resume",
subtitle: "Template selection",
complete: baseResumeValidation.valid,
disabled: !rxresumeValidation.valid,
},
],
[
skipRxResumeOnboarding,
llmValidated,
rxresumeValidation.valid,
baseResumeValidation.valid,
],
);
const defaultStep = steps.find((step) => !step.complete)?.id ?? steps[0]?.id;
useEffect(() => {
if (!shouldOpen) return;
if (!currentStep && defaultStep) {
setCurrentStep(defaultStep);
}
}, [currentStep, defaultStep, shouldOpen]);
const runAllValidations = useCallback(async () => {
if (!settings) return;
const validations: Promise<ValidationResult>[] = [];
if (requiresLlmKey) {
await validateLlm();
validations.push(validateLlm());
} else {
setLlmValidation({ valid: true, message: null, checked: true });
}
}, [settings, requiresLlmKey, validateLlm]);
if (!skipRxResumeOnboarding) {
validations.push(validateRxresume(), validateBaseResume());
} else {
setRxresumeValidation({ valid: true, message: null, checked: true });
setBaseResumeValidation({ valid: true, message: null, checked: true });
}
const results = await Promise.allSettled(validations);
const failed = results.find((result) => result.status === "rejected");
if (failed) {
const reason = failed.status === "rejected" ? failed.reason : null;
const message =
reason instanceof Error ? reason.message : "Validation checks failed";
toast.error(message);
}
}, [
settings,
requiresLlmKey,
skipRxResumeOnboarding,
validateLlm,
validateRxresume,
validateBaseResume,
]);
// Run validations on mount when needed
useEffect(() => {
if (demoMode) return;
if (!settings || settingsLoading) return;
if (requiresLlmKey ? llmValidation.checked : true) return;
void runLlmValidation();
const needsValidation =
(requiresLlmKey ? !llmValidation.checked : false) ||
(skipRxResumeOnboarding
? false
: !rxresumeValidation.checked || !baseResumeValidation.checked);
if (!needsValidation) return;
void runAllValidations();
}, [
settings,
settingsLoading,
requiresLlmKey,
llmValidation.checked,
runLlmValidation,
rxresumeValidation.checked,
baseResumeValidation.checked,
runAllValidations,
demoMode,
skipRxResumeOnboarding,
]);
const handleSaveLlm = async () => {
const handleSaveLlm = async (): Promise<boolean> => {
const values = getValues();
const apiKeyValue = values.llmApiKey.trim();
const baseUrlValue = values.llmBaseUrl.trim();
if (requiresLlmKey && !apiKeyValue && !hasLlmKey) {
toast.info("Add your LLM API key to continue");
return;
return false;
}
try {
@@ -197,7 +457,7 @@ export const OnboardingGate: React.FC = () => {
if (!validation.valid) {
toast.error(validation.message || "LLM validation failed");
return;
return false;
}
const update: Partial<UpdateSettingsInput> = {
@@ -224,19 +484,165 @@ export const OnboardingGate: React.FC = () => {
? `Default for ${providerConfig.label}: ${defaultModel}.`
: "Select the model manually in Settings > Model.",
});
return true;
} catch (error) {
const message =
error instanceof Error ? error.message : "Failed to save LLM settings";
toast.error(message);
return false;
} finally {
setIsSavingEnv(false);
}
};
const isBusy = isSavingEnv || settingsLoading || isValidatingLlm;
const progressValue = llmValidated ? 100 : 0;
const handleSaveRxresume = async (): Promise<boolean> => {
const values = getValues();
const modeValue = values.rxresumeMode;
const draftCredentials = getRxResumeCredentialDrafts(values);
const missing = getRxResumeMissingCredentialLabels({
mode: modeValue,
stored: storedRxResume,
draft: draftCredentials,
});
if (!shouldOpen) return null;
if (missing.length > 0) {
toast.info("Almost there", {
description: `Missing: ${missing.join(", ")}`,
});
return false;
}
try {
setIsValidatingRxresume(true);
const result = await validateAndMaybePersistRxResumeMode({
mode: modeValue,
stored: storedRxResume,
draft: draftCredentials,
validate: api.validateRxresume,
persist: async (update) => {
setIsSavingEnv(true);
try {
await api.updateSettings(update);
await refreshSettings();
} finally {
setIsSavingEnv(false);
}
},
persistOnSuccess: true,
getPrecheckMessage: (failure) =>
failure === "missing-v5-api-key"
? "v5 API key required. Add a v5 API key, then test again."
: "v4 email and password required. Add both credentials, then test again.",
getValidationErrorMessage: (error) =>
error instanceof Error ? error.message : "RxResume validation failed",
getPersistErrorMessage: (error) =>
error instanceof Error
? error.message
: "Failed to save RxResume credentials",
});
setRxresumeVersionValidations((current) => ({
...current,
[modeValue]: {
...result.validation,
checked: true,
testedAt: Date.now(),
},
}));
setRxresumeValidation({ ...result.validation, checked: true });
if (!result.validation.valid) {
toast.error(result.validation.message || "RxResume validation failed");
return false;
}
setValue("rxresumePassword", "");
setValue("rxresumeApiKey", "");
toast.success("RxResume connected");
return true;
} catch (error) {
const message =
error instanceof Error
? error.message
: "Failed to save RxResume credentials";
toast.error(message);
return false;
} finally {
setIsValidatingRxresume(false);
setIsSavingEnv(false);
}
};
const handleSaveBaseResume = async (): Promise<boolean> => {
const values = getValues();
if (!values.rxresumeBaseResumeId) {
toast.info("Select a base resume to continue");
return false;
}
try {
setIsSavingEnv(true);
await api.updateSettings({
rxresumeMode: values.rxresumeMode,
rxresumeBaseResumeId: values.rxresumeBaseResumeId,
});
const validation = await validateBaseResume();
if (!validation.valid) {
toast.error(validation.message || "Base resume validation failed");
return false;
}
await refreshSettings();
toast.success("Base resume set");
return true;
} catch (error) {
const message =
error instanceof Error ? error.message : "Failed to save base resume";
toast.error(message);
return false;
} finally {
setIsSavingEnv(false);
}
};
const resolvedStepIndex = currentStep
? steps.findIndex((step) => step.id === currentStep)
: 0;
const stepIndex = resolvedStepIndex >= 0 ? resolvedStepIndex : 0;
const completedSteps = steps.filter((step) => step.complete).length;
const progressValue =
steps.length > 0 ? Math.round((completedSteps / steps.length) * 100) : 0;
const isBusy =
isSavingEnv ||
settingsLoading ||
isValidatingLlm ||
isValidatingRxresume ||
isValidatingBaseResume;
const canGoBack = stepIndex > 0;
const handlePrimaryAction = async () => {
if (!currentStep) return;
if (currentStep === "llm") {
await handleSaveLlm();
return;
}
if (currentStep === "rxresume") {
await handleSaveRxresume();
return;
}
if (currentStep === "baseresume") {
await handleSaveBaseResume();
return;
}
};
const handleBack = () => {
if (!canGoBack) return;
setCurrentStep(steps[stepIndex - 1]?.id ?? currentStep);
};
if (!shouldOpen || !currentStep) return null;
return (
<AlertDialog open>
@@ -248,134 +654,241 @@ export const OnboardingGate: React.FC = () => {
<AlertDialogHeader>
<AlertDialogTitle>Welcome to Job Ops</AlertDialogTitle>
<AlertDialogDescription>
Connect your LLM provider to run job scoring, summaries, and
tailoring. You can add a resume file later in Settings if you need
PDF export or resume-based scoring.
Let's get your workspace ready. Add your keys and resume once,
then the pipeline can run end-to-end.
</AlertDialogDescription>
</AlertDialogHeader>
<FieldLabel className="w-full [&>[data-slot=field]]:border-0 [&>[data-slot=field]]:p-0 [&>[data-slot=field]]:rounded-none">
<div className="w-full rounded-md border-b-2 border-primary bg-muted/60 px-3 py-4 text-left">
<Field orientation="horizontal" className="items-start">
<FieldContent>
<FieldTitle>LLM Provider</FieldTitle>
<FieldDescription>Provider + credentials</FieldDescription>
</FieldContent>
<span
className={cn(
"mt-0.5 flex h-6 w-6 items-center justify-center rounded-md text-xs font-semibold",
llmValidated
? "bg-primary text-primary-foreground"
: "bg-muted text-muted-foreground",
)}
>
{llmValidated ? <Check className="h-3.5 w-3.5" /> : "1"}
</span>
</Field>
</div>
</FieldLabel>
<Tabs value={currentStep} onValueChange={setCurrentStep}>
<TabsList className="grid h-auto w-full grid-cols-1 gap-2 border-b border-border/60 bg-transparent p-0 text-left sm:grid-cols-3">
{steps.map((step, index) => {
const isActive = step.id === currentStep;
const isComplete = step.complete;
<div className="space-y-4 pt-2">
<div>
<p className="text-sm font-semibold">Connect LLM provider</p>
<p className="text-xs text-muted-foreground">
Used for job scoring, summaries, and tailoring.
</p>
</div>
<div className="grid gap-4 md:grid-cols-2">
<div className="space-y-2">
<label htmlFor="llmProvider" className="text-sm font-medium">
Provider
</label>
<Controller
name="llmProvider"
control={control}
render={({ field }) => (
<Select
value={selectedProvider}
onValueChange={(value) => {
field.onChange(value);
}}
disabled={isSavingEnv}
return (
<FieldLabel
key={step.id}
className={cn(
"w-full [&>[data-slot=field]]:border-0 [&>[data-slot=field]]:p-0 [&>[data-slot=field]]:rounded-none",
step.disabled && "opacity-50 cursor-not-allowed",
)}
>
<TabsTrigger
value={step.id}
disabled={step.disabled}
className={cn(
"w-full rounded-md hover:bg-muted/60 border-b-2 border-transparent px-3 py-4 text-left shadow-none",
isActive
? "border-primary !bg-muted/60 text-foreground"
: "text-muted-foreground",
)}
>
<SelectTrigger id="llmProvider">
<SelectValue placeholder="Select provider" />
</SelectTrigger>
<SelectContent>
{LLM_PROVIDERS.map((provider) => (
<SelectItem key={provider} value={provider}>
{LLM_PROVIDER_LABELS[provider]}
</SelectItem>
))}
</SelectContent>
</Select>
)}
/>
<Field orientation="horizontal" className="items-start">
<FieldContent>
<FieldTitle>{step.label}</FieldTitle>
<FieldDescription>{step.subtitle}</FieldDescription>
</FieldContent>
<span
className={cn(
"mt-0.5 flex h-6 w-6 items-center justify-center rounded-md text-xs font-semibold",
isComplete
? "bg-primary text-primary-foreground"
: "bg-muted text-muted-foreground",
)}
>
{isComplete ? (
<Check className="h-3.5 w-3.5" />
) : (
index + 1
)}
</span>
</Field>
</TabsTrigger>
</FieldLabel>
);
})}
</TabsList>
<TabsContent value="llm" className="space-y-4 pt-6">
<div>
<p className="text-sm font-semibold">Connect LLM provider</p>
<p className="text-xs text-muted-foreground">
{providerConfig.providerHint}
Used for job scoring, summaries, and tailoring.
</p>
</div>
{showBaseUrl && (
<Controller
name="llmBaseUrl"
control={control}
render={({ field }) => (
<SettingsInput
label="LLM base URL"
inputProps={{
name: "llmBaseUrl",
value: field.value,
onChange: field.onChange,
}}
placeholder={providerConfig.baseUrlPlaceholder}
helper={providerConfig.baseUrlHelper}
current={settings?.llmBaseUrl?.value || "—"}
disabled={isSavingEnv}
/>
)}
/>
)}
{showApiKey && (
<Controller
name="llmApiKey"
control={control}
render={({ field }) => (
<SettingsInput
label="LLM API key"
inputProps={{
name: "llmApiKey",
value: field.value,
onChange: field.onChange,
}}
type="password"
placeholder="Enter key"
helper={
llmKeyHint
? `${providerConfig.keyHelper}. Leave blank to use the saved key.`
: providerConfig.keyHelper
}
disabled={isSavingEnv}
/>
)}
/>
)}
<div className="grid gap-4 md:grid-cols-2">
<div className="space-y-2">
<label htmlFor="llmProvider" className="text-sm font-medium">
Provider
</label>
<Controller
name="llmProvider"
control={control}
render={({ field }) => (
<Select
value={selectedProvider}
onValueChange={(value) => {
field.onChange(value);
}}
disabled={isSavingEnv}
>
<SelectTrigger id="llmProvider">
<SelectValue placeholder="Select provider" />
</SelectTrigger>
<SelectContent>
{LLM_PROVIDERS.map((provider) => (
<SelectItem key={provider} value={provider}>
{LLM_PROVIDER_LABELS[provider]}
</SelectItem>
))}
</SelectContent>
</Select>
)}
/>
<p className="text-xs text-muted-foreground">
{providerConfig.providerHint}
</p>
</div>
{showBaseUrl && (
<Controller
name="llmBaseUrl"
control={control}
render={({ field }) => (
<SettingsInput
label="LLM base URL"
inputProps={{
name: "llmBaseUrl",
value: field.value,
onChange: field.onChange,
}}
placeholder={providerConfig.baseUrlPlaceholder}
helper={providerConfig.baseUrlHelper}
current={settings?.llmBaseUrl?.value || "—"}
disabled={isSavingEnv}
/>
)}
/>
)}
{showApiKey && (
<Controller
name="llmApiKey"
control={control}
render={({ field }) => (
<SettingsInput
label="LLM API key"
inputProps={{
name: "llmApiKey",
value: field.value,
onChange: field.onChange,
}}
type="password"
placeholder="Enter key"
helper={
llmKeyHint
? `${providerConfig.keyHelper}. Leave blank to use the saved key.`
: providerConfig.keyHelper
}
disabled={isSavingEnv}
/>
)}
/>
)}
</div>
</TabsContent>
<TabsContent value="rxresume" className="space-y-4 pt-6">
<ReactiveResumeConfigPanel
mode={rxresumeModeCurrent}
onModeChange={(mode) => {
setValue("rxresumeMode", mode);
setValue(
"rxresumeBaseResumeId",
getBaseResumeIdForMode(mode),
);
setRxresumeValidation((previous) => ({
...EMPTY_VALIDATION_STATE,
checked: previous.checked,
}));
}}
disabled={isSavingEnv}
showValidationStatus
validationStatuses={rxresumeVersionValidations}
intro={{
title: "Link your RxResume account",
description:
"Used to export tailored PDFs. Choose between Reactive Resume version 4 and 5, and provide the credentials.",
}}
v5={{
apiKey: watch("rxresumeApiKey"),
onApiKeyChange: (value) => setValue("rxresumeApiKey", value),
}}
shared={{
baseUrl: watch("rxresumeUrl"),
onBaseUrlChange: (value) => setValue("rxresumeUrl", value),
}}
v4={{
email: watch("rxresumeEmail"),
onEmailChange: (value) => setValue("rxresumeEmail", value),
password: watch("rxresumePassword"),
onPasswordChange: (value) =>
setValue("rxresumePassword", value),
}}
/>
</TabsContent>
<TabsContent value="baseresume" className="space-y-4 pt-6">
<div>
<p className="text-sm font-semibold">
Select your template resume
</p>
<p className="text-xs text-muted-foreground">
Choose the resume you want to use as a template. The selected
resume will be used as a template for tailoring.
</p>
</div>
<Controller
name="rxresumeBaseResumeId"
control={control}
render={({ field }) => (
<BaseResumeSelection
value={field.value}
onValueChange={(value) => {
const mode = (getValues("rxresumeMode") ??
"v5") as RxResumeMode;
setBaseResumeIdForMode(mode, value);
field.onChange(value);
}}
hasRxResumeAccess={rxresumeValidation.valid}
rxresumeMode={rxresumeModeCurrent}
disabled={isSavingEnv}
/>
)}
/>
</TabsContent>
</Tabs>
<div className="flex items-center justify-between">
<Button
variant="outline"
onClick={handleBack}
disabled={!canGoBack || isBusy}
>
Back
</Button>
<div className="flex items-center gap-2">
<Button onClick={handlePrimaryAction} disabled={isBusy}>
{isBusy
? "Validating..."
: getStepPrimaryLabel({
currentStep,
llmValidated,
rxresumeValidated: rxresumeValidation.valid,
baseResumeValidated: baseResumeValidation.valid,
})}
</Button>
</div>
{llmValidation.checked && !llmValidation.valid ? (
<p className="text-xs text-destructive">
{llmValidation.message ?? "LLM validation failed."}
</p>
) : null}
</div>
<div className="flex items-center justify-end">
<Button onClick={handleSaveLlm} disabled={isBusy}>
{isBusy
? "Validating..."
: llmValidated
? "Revalidate"
: "Connect"}
</Button>
</div>
<Progress value={progressValue} className="h-2" />
</div>
</AlertDialogContent>
@@ -0,0 +1,453 @@
import { BaseResumeSelection } from "@client/pages/settings/components/BaseResumeSelection";
import { SettingsInput } from "@client/pages/settings/components/SettingsInput";
import {
toggleAiSelectable,
toggleMustInclude,
} from "@client/pages/settings/resume-projects-state";
import type { ResumeProjectsSettingsInput } from "@shared/settings-schema.js";
import type { ResumeProjectCatalogItem, RxResumeMode } from "@shared/types.js";
import { AlertCircle, AlertTriangle } from "lucide-react";
import type React from "react";
import { Alert, AlertDescription, AlertTitle } from "@/components/ui/alert";
import { Checkbox } from "@/components/ui/checkbox";
import { Input } from "@/components/ui/input";
import { Separator } from "@/components/ui/separator";
import {
Table,
TableBody,
TableCell,
TableHead,
TableHeader,
TableRow,
} from "@/components/ui/table";
import { Tabs, TabsList, TabsTrigger } from "@/components/ui/tabs";
import { clampInt } from "@/lib/utils";
import { StatusIndicator } from "./StatusIndicator";
type VersionValidationState = {
checked: boolean;
valid: boolean;
message?: string | null;
status?: number | null;
};
type ProjectSelectionConfig = {
baseResumeId: string | null;
onBaseResumeIdChange: (value: string | null) => void;
projects: ResumeProjectCatalogItem[];
value: ResumeProjectsSettingsInput | null | undefined;
onChange: (next: ResumeProjectsSettingsInput) => void;
lockedCount: number;
maxProjectsTotal: number;
isProjectsLoading: boolean;
disabled: boolean;
maxProjectsError?: string;
};
type ReactiveResumeConfigPanelProps = {
mode: RxResumeMode;
onModeChange: (mode: RxResumeMode) => void;
disabled?: boolean;
hasRxResumeAccess?: boolean;
showValidationStatus?: boolean;
validationStatuses?: {
v4: VersionValidationState;
v5: VersionValidationState;
};
intro?: {
title: string;
description?: string;
};
v5: {
apiKey: string;
onApiKeyChange: (value: string) => void;
error?: string;
helper?: string;
placeholder?: string;
};
shared: {
baseUrl: string;
onBaseUrlChange: (value: string) => void;
baseUrlError?: string;
baseUrlHelper?: string;
baseUrlPlaceholder?: string;
};
v4: {
email: string;
onEmailChange: (value: string) => void;
emailError?: string;
password: string;
onPasswordChange: (value: string) => void;
passwordError?: string;
emailPlaceholder?: string;
passwordPlaceholder?: string;
};
projectSelection?: ProjectSelectionConfig;
};
function renderStatusPill(label: string, state: VersionValidationState) {
const statusLabel = state.checked
? state.valid
? "Connected"
: "Failed"
: "Not tested";
const dotColor = state.checked
? state.valid
? "bg-emerald-500"
: "bg-destructive"
: "bg-muted-foreground";
return (
<StatusIndicator
label={`${label}: ${statusLabel}`}
dotColor={dotColor}
tooltip={
state.checked && !state.valid && state.message
? state.message
: undefined
}
/>
);
}
function isAvailabilityWarning(state?: VersionValidationState): boolean {
const status = state?.status ?? null;
return status === 0 || (typeof status === "number" && status >= 500);
}
export const ReactiveResumeConfigPanel: React.FC<
ReactiveResumeConfigPanelProps
> = ({
mode,
onModeChange,
disabled = false,
hasRxResumeAccess = false,
showValidationStatus = false,
validationStatuses,
intro,
shared,
v5,
v4,
projectSelection,
}) => {
const canShowProjectSelection = Boolean(
projectSelection && hasRxResumeAccess,
);
const selectedValidationStatus = validationStatuses?.[mode];
const showInlineValidationAlert = Boolean(
selectedValidationStatus?.checked &&
!selectedValidationStatus.valid &&
selectedValidationStatus.message,
);
const selectedValidationIsWarning =
showInlineValidationAlert &&
isAvailabilityWarning(selectedValidationStatus);
const handleModeChange = (value: string) =>
onModeChange(value === "v4" ? "v4" : "v5");
return (
<div className="space-y-4">
{intro ? (
<div>
<p className="text-sm font-semibold">{intro.title}</p>
{intro.description ? (
<p className="text-xs text-muted-foreground">{intro.description}</p>
) : null}
</div>
) : null}
<Tabs value={mode} onValueChange={handleModeChange}>
<TabsList className="grid h-auto w-full grid-cols-2">
<TabsTrigger value="v5" disabled={disabled}>
v5 (API key)
</TabsTrigger>
<TabsTrigger value="v4" disabled={disabled}>
v4 (Email + Password)
</TabsTrigger>
</TabsList>
</Tabs>
{showValidationStatus && selectedValidationStatus ? (
<div className="flex flex-wrap items-center gap-2 text-xs w-full justify-between">
{renderStatusPill(`${mode} status`, selectedValidationStatus)}
</div>
) : null}
{showInlineValidationAlert && selectedValidationStatus?.message ? (
<Alert
variant={selectedValidationIsWarning ? "default" : "destructive"}
className={
selectedValidationIsWarning
? "border-amber-200 bg-amber-50 text-amber-950 [&>svg]:text-amber-700"
: undefined
}
>
{selectedValidationIsWarning ? (
<AlertTriangle className="h-4 w-4" />
) : (
<AlertCircle className="h-4 w-4" />
)}
<AlertTitle>
Reactive Resume {mode.toUpperCase()}{" "}
{selectedValidationIsWarning ? "warning" : "error"}
</AlertTitle>
<AlertDescription>
{selectedValidationStatus.message}
</AlertDescription>
</Alert>
) : null}
{mode === "v5" ? (
<div className="grid gap-4">
<SettingsInput
label="RxResume URL"
inputProps={{
name: "rxresumeUrl",
value: shared.baseUrl,
onChange: (event) =>
shared.onBaseUrlChange(event.currentTarget.value),
}}
type="url"
placeholder={
shared.baseUrlPlaceholder ?? "https://resume.example.com"
}
helper={
shared.baseUrlHelper ??
"Leave blank to use the default for the selected mode (or the RXRESUME_URL environment override, if set)."
}
disabled={disabled}
error={shared.baseUrlError}
/>
<SettingsInput
label="v5 API key"
inputProps={{
name: "rxresumeApiKey",
value: v5.apiKey,
onChange: (event) => v5.onApiKeyChange(event.currentTarget.value),
}}
type="password"
placeholder={v5.placeholder ?? "Enter v5 API key"}
helper={v5.helper}
disabled={disabled}
error={v5.error}
/>
</div>
) : (
<div className="grid gap-4 md:grid-cols-2">
<div className="md:col-span-2">
<SettingsInput
label="RxResume URL"
inputProps={{
name: "rxresumeUrl",
value: shared.baseUrl,
onChange: (event) =>
shared.onBaseUrlChange(event.currentTarget.value),
}}
type="url"
placeholder={
shared.baseUrlPlaceholder ?? "https://resume.example.com"
}
helper={
shared.baseUrlHelper ??
"Leave blank to use the public cloud default for the selected mode."
}
disabled={disabled}
error={shared.baseUrlError}
/>
</div>
<SettingsInput
label="v4 Email"
inputProps={{
name: "rxresumeEmail",
value: v4.email,
onChange: (event) => v4.onEmailChange(event.currentTarget.value),
}}
placeholder={v4.emailPlaceholder ?? "you@example.com"}
disabled={disabled}
error={v4.emailError}
/>
<SettingsInput
label="v4 Password"
inputProps={{
name: "rxresumePassword",
value: v4.password,
onChange: (event) =>
v4.onPasswordChange(event.currentTarget.value),
}}
type="password"
placeholder={v4.passwordPlaceholder ?? "Enter v4 password"}
disabled={disabled}
error={v4.passwordError}
/>
</div>
)}
{projectSelection ? (
<>
<Separator />
{!canShowProjectSelection ? (
<div className="rounded-md border border-dashed border-muted-foreground/40 bg-muted/30 px-4 py-3 text-sm text-muted-foreground">
Connect Reactive Resume and choose a template resume to configure
resume projects.
</div>
) : (
<div className="space-y-4">
<BaseResumeSelection
value={projectSelection.baseResumeId}
onValueChange={projectSelection.onBaseResumeIdChange}
hasRxResumeAccess={hasRxResumeAccess}
rxresumeMode={mode}
disabled={projectSelection.disabled}
/>
{!projectSelection.baseResumeId ? (
<div className="rounded-md border border-dashed border-muted-foreground/40 bg-muted/30 px-4 py-3 text-sm text-muted-foreground">
Choose a PDF to configure resume projects.
</div>
) : (
<>
<div className="space-y-2">
<div className="text-sm font-medium">
Max projects to choose
</div>
<Input
type="number"
inputMode="numeric"
min={projectSelection.lockedCount}
max={projectSelection.maxProjectsTotal}
value={projectSelection.value?.maxProjects ?? 0}
onChange={(event) => {
if (!projectSelection.value) return;
const next = Number(event.target.value);
const clamped = clampInt(
next,
projectSelection.lockedCount,
projectSelection.maxProjectsTotal,
);
projectSelection.onChange({
...projectSelection.value,
maxProjects: clamped,
});
}}
disabled={
projectSelection.disabled ||
projectSelection.isProjectsLoading ||
!projectSelection.value
}
/>
{projectSelection.maxProjectsError ? (
<p className="text-xs text-destructive">
{projectSelection.maxProjectsError}
</p>
) : null}
</div>
<Table>
<TableHeader>
<TableRow>
<TableHead className="text-xs whitespace-wrap sm:whitespace-nowrap">
Project
</TableHead>
<TableHead className="text-xs whitespace-wrap sm:whitespace-nowrap">
Visible in template
</TableHead>
<TableHead className="text-xs whitespace-wrap sm:whitespace-nowrap">
Must Include
</TableHead>
<TableHead className="text-xs whitespace-wrap sm:whitespace-nowrap">
AI selectable
</TableHead>
</TableRow>
</TableHeader>
<TableBody>
{projectSelection.projects.map((project) => {
const value = projectSelection.value;
const locked = Boolean(
value?.lockedProjectIds.includes(project.id),
);
const aiSelectable = Boolean(
value?.aiSelectableProjectIds.includes(project.id),
);
const projectMeta =
mode === "v5"
? project.date
: [project.description, project.date]
.filter(Boolean)
.join(" - ");
return (
<TableRow key={project.id}>
<TableCell>
<div className="space-y-0.5">
<div className="font-medium">
{project.name}
</div>
{projectMeta ? (
<div className="text-xs text-muted-foreground">
{projectMeta}
</div>
) : null}
</div>
</TableCell>
<TableCell>
{project.isVisibleInBase ? "Yes" : "No"}
</TableCell>
<TableCell>
<Checkbox
checked={locked}
onCheckedChange={() => {
if (!value) return;
projectSelection.onChange(
toggleMustInclude({
settings: value,
projectId: project.id,
checked: !locked,
maxProjectsTotal:
projectSelection.maxProjectsTotal,
}),
);
}}
disabled={
projectSelection.disabled ||
projectSelection.isProjectsLoading ||
!value
}
/>
</TableCell>
<TableCell>
<Checkbox
checked={locked ? true : aiSelectable}
onCheckedChange={() => {
if (!value) return;
projectSelection.onChange(
toggleAiSelectable({
settings: value,
projectId: project.id,
checked: !aiSelectable,
}),
);
}}
disabled={
projectSelection.disabled ||
projectSelection.isProjectsLoading ||
locked ||
!value
}
/>
</TableCell>
</TableRow>
);
})}
</TableBody>
</Table>
</>
)}
</div>
)}
</>
) : null}
</div>
);
};
@@ -51,6 +51,7 @@ import { useRescoreJob } from "../hooks/useRescoreJob";
import { FitAssessment, JobHeader, JobNotes, TailoredSummary } from ".";
import { CoverLetterDisplay } from "./CoverLetterDisplay";
import { TailorMode } from "./discovered-panel/TailorMode";
import { GhostwriterDrawer } from "./ghostwriter/GhostwriterDrawer";
import { JobDetailsEditDrawer } from "./JobDetailsEditDrawer";
import { KbdHint } from "./KbdHint";
import { OpenJobListingButton } from "./OpenJobListingButton";
@@ -379,6 +380,11 @@ export const ReadyPanel: React.FC<ReadyPanelProps> = ({
*/}
<div className="pb-4 border-b border-border/40">
<div className="grid gap-2 sm:grid-cols-2">
<GhostwriterDrawer
job={job}
triggerClassName="h-9 w-full justify-center gap-1 px-2 text-xs"
/>
{/* Download PDF - primary artifact action */}
<Button
asChild
@@ -1,74 +0,0 @@
import {
parseSponsorshipSignals,
SPONSORSHIP_GREEN_FLAG_IDS,
SPONSORSHIP_RED_FLAG_IDS,
SPONSORSHIP_SIGNAL_META,
SPONSORSHIP_YELLOW_FLAG_IDS,
type SponsorshipFlagTier,
} from "@shared/sponsorship-signals";
import type React from "react";
import { Badge } from "@/components/ui/badge";
import {
Tooltip,
TooltipContent,
TooltipProvider,
TooltipTrigger,
} from "@/components/ui/tooltip";
import { cn } from "@/lib/utils";
const flagTierClass: Record<SponsorshipFlagTier, string> = {
green: "border-emerald-500/30 bg-emerald-500/10 text-emerald-200",
yellow: "border-amber-500/40 bg-amber-500/10 text-amber-200",
red: "border-rose-500/40 bg-rose-500/10 text-rose-200",
};
interface SponsorshipSignalsPillsProps {
sponsorshipSignals: string | null | undefined;
className?: string;
size?: "xs" | "sm";
}
export const SponsorshipSignalsPills: React.FC<
SponsorshipSignalsPillsProps
> = ({ sponsorshipSignals, className, size = "sm" }) => {
const signals = parseSponsorshipSignals(sponsorshipSignals);
if (signals.length === 0) return null;
const ordered = [
...SPONSORSHIP_RED_FLAG_IDS.filter((id) => signals.includes(id)),
...SPONSORSHIP_YELLOW_FLAG_IDS.filter((id) => signals.includes(id)),
...SPONSORSHIP_GREEN_FLAG_IDS.filter((id) => signals.includes(id)),
];
return (
<TooltipProvider>
<div className={cn("flex flex-wrap items-center gap-1", className)}>
{ordered.map((signal) => {
const meta = SPONSORSHIP_SIGNAL_META[signal];
return (
<Tooltip key={signal} delayDuration={0}>
<TooltipTrigger asChild>
<Badge
variant="outline"
className={cn(
"font-medium",
size === "xs" ? "px-1 py-0 text-[10px]" : "text-xs",
flagTierClass[meta.flagTier],
)}
>
{meta.shortLabel}
</Badge>
</TooltipTrigger>
<TooltipContent side="top" className="max-w-xs">
<p className="text-xs font-medium">{meta.label}</p>
<p className="text-xs text-muted-foreground">
{meta.description}
</p>
</TooltipContent>
</Tooltip>
);
})}
</div>
</TooltipProvider>
);
};
@@ -1,20 +1,24 @@
import { LayoutDashboard, Settings } from "lucide-react";
import {
Columns3,
Home,
Inbox,
LayoutDashboard,
Link2,
Settings,
Shield,
} from "lucide-react";
export type NavLink = {
to: string;
label: string;
icon: typeof LayoutDashboard;
icon: typeof Home;
activePaths?: string[];
};
/**
* Slim product nav: pipeline + scoring only.
* Overview / kanban / inbox / tracer / visa browser stay routable for
* bookmarks but are intentionally omitted from the chrome.
*/
export const NAV_LINKS: NavLink[] = [
{ to: "/overview", label: "Overview", icon: Home },
{
to: "/jobs/discovered",
to: "/jobs/ready",
label: "Jobs",
icon: LayoutDashboard,
activePaths: [
@@ -24,6 +28,20 @@ export const NAV_LINKS: NavLink[] = [
"/jobs/all",
],
},
{
to: "/applications/in-progress",
label: "In Progress",
icon: Columns3,
activePaths: ["/applications/in-progress"],
},
{ to: "/tracking-inbox", label: "Tracking Inbox", icon: Inbox },
{
to: "/tracer-links",
label: "Tracer Links",
icon: Link2,
activePaths: ["/tracer-links"],
},
{ to: "/visa-sponsors", label: "Visa Sponsors", icon: Shield },
{ to: "/settings", label: "Settings", icon: Settings },
];
@@ -0,0 +1,55 @@
import {
getRxResumeBaseResumeSelection,
getStoredRxResumeCredentialAvailability,
type RxResumeSettingsLike,
} from "@client/lib/rxresume-config";
import type { RxResumeMode } from "@shared/types.js";
import { useCallback, useMemo, useState } from "react";
const EMPTY_IDS_BY_MODE: Record<RxResumeMode, string | null> = {
v4: null,
v5: null,
};
export function useRxResumeConfigState(settings: RxResumeSettingsLike) {
const storedRxResume = useMemo(
() => getStoredRxResumeCredentialAvailability(settings),
[settings],
);
const [baseResumeIdsByMode, setBaseResumeIdsByMode] =
useState<Record<RxResumeMode, string | null>>(EMPTY_IDS_BY_MODE);
const syncBaseResumeIdsForMode = useCallback(
(mode: RxResumeMode) => {
const { idsByMode, selectedId } = getRxResumeBaseResumeSelection(
settings,
mode,
);
setBaseResumeIdsByMode(idsByMode);
return selectedId;
},
[settings],
);
const getBaseResumeIdForMode = useCallback(
(mode: RxResumeMode) => baseResumeIdsByMode[mode] ?? null,
[baseResumeIdsByMode],
);
const setBaseResumeIdForMode = useCallback(
(mode: RxResumeMode, value: string | null) => {
setBaseResumeIdsByMode((prev) =>
prev[mode] === value ? prev : { ...prev, [mode]: value },
);
},
[],
);
return {
storedRxResume,
baseResumeIdsByMode,
syncBaseResumeIdsForMode,
getBaseResumeIdForMode,
setBaseResumeIdForMode,
};
}
+1 -74
View File
@@ -1,17 +1,8 @@
import { PRIOR_SKIP_DISMISS_LOOKBACK_MS } from "@shared/job-fingerprint";
import { createJob } from "@shared/testing/factories";
import { describe, expect, it } from "vitest";
import { buildDuplicateDismissHints } from "./job-dedup";
describe("buildDuplicateDismissHints", () => {
const now = Date.parse("2026-06-10T12:00:00.000Z");
const recentUpdatedAt = new Date(
now - PRIOR_SKIP_DISMISS_LOOKBACK_MS + 86_400_000,
).toISOString();
const staleUpdatedAt = new Date(
now - PRIOR_SKIP_DISMISS_LOOKBACK_MS - 86_400_000,
).toISOString();
it("flags open jobs that match a skipped posting", () => {
const jobs = [
createJob({
@@ -19,7 +10,6 @@ describe("buildDuplicateDismissHints", () => {
employer: "Acme",
title: "SDET",
status: "skipped",
updatedAt: recentUpdatedAt,
}),
createJob({
id: "open-1",
@@ -35,71 +25,8 @@ describe("buildDuplicateDismissHints", () => {
}),
];
const hints = buildDuplicateDismissHints(jobs, now);
const hints = buildDuplicateDismissHints(jobs);
expect(hints.get("open-1")).toBe("skipped");
expect(hints.has("open-2")).toBe(false);
});
it("matches employer prefix and plural title variants", () => {
const jobs = [
createJob({
id: "skipped-1",
employer: "CGI",
title: "Automation Test Engineer",
status: "skipped",
updatedAt: recentUpdatedAt,
}),
createJob({
id: "open-1",
employer: ". CGI IT UK Limited",
title: "Automation Test Engineers",
status: "discovered",
}),
];
const hints = buildDuplicateDismissHints(jobs, now);
expect(hints.get("open-1")).toBe("skipped");
});
it("ignores skipped jobs older than the lookback window", () => {
const jobs = [
createJob({
id: "skipped-1",
employer: "Acme",
title: "SDET",
status: "skipped",
updatedAt: staleUpdatedAt,
}),
createJob({
id: "open-1",
employer: "Acme",
title: "SDET",
status: "discovered",
}),
];
const hints = buildDuplicateDismissHints(jobs, now);
expect(hints.has("open-1")).toBe(false);
});
it("does not match different titles at the same employer", () => {
const jobs = [
createJob({
id: "skipped-1",
employer: "Cognizant",
title: "Automation Test Engineer",
status: "skipped",
updatedAt: recentUpdatedAt,
}),
createJob({
id: "open-1",
employer: "Cognizant",
title: "AI Automation Test Engineer",
status: "discovered",
}),
];
const hints = buildDuplicateDismissHints(jobs, now);
expect(hints.has("open-1")).toBe(false);
});
});
+25 -39
View File
@@ -1,60 +1,44 @@
import {
jobsMatchEmployerAndTitleDismiss,
PRIOR_SKIP_DISMISS_LOOKBACK_MS,
} from "@shared/job-fingerprint";
import { collectJobDedupKeys } from "@shared/job-fingerprint";
import type { JobListItem, JobStatus } from "@shared/types";
export type DuplicateDismissReason = "skipped" | "applied";
const OPEN_JOB_STATUSES = new Set<JobStatus>([
"discovered",
"ready",
"processing",
]);
function isWithinDismissLookback(updatedAt: string, nowMs: number): boolean {
const updatedMs = Date.parse(updatedAt);
if (!Number.isFinite(updatedMs)) return false;
return nowMs - updatedMs <= PRIOR_SKIP_DISMISS_LOOKBACK_MS;
}
/**
* Map open jobs to a prior skip/apply when the same company and title match
* within the last 90 days (see {@link PRIOR_SKIP_DISMISS_LOOKBACK_MS}).
* Map open jobs to a prior skip/apply when employer+title or description matches.
*/
export function buildDuplicateDismissHints(
jobs: readonly JobListItem[],
nowMs: number = Date.now(),
): Map<string, DuplicateDismissReason> {
const dismissed: Array<{
employer: string;
title: string;
reason: DuplicateDismissReason;
}> = [];
const dismissedKeys = new Map<string, DuplicateDismissReason>();
for (const job of jobs) {
if (job.status !== "skipped" && job.status !== "applied") continue;
if (!isWithinDismissLookback(job.updatedAt, nowMs)) continue;
dismissed.push({
const reason: DuplicateDismissReason =
job.status === "applied" ? "applied" : "skipped";
for (const key of collectJobDedupKeys({
employer: job.employer,
title: job.title,
reason: job.status === "applied" ? "applied" : "skipped",
});
})) {
if (!dismissedKeys.has(key)) dismissedKeys.set(key, reason);
}
}
const hints = new Map<string, DuplicateDismissReason>();
const openStatuses = new Set<JobStatus>([
"discovered",
"ready",
"processing",
]);
for (const job of jobs) {
if (!OPEN_JOB_STATUSES.has(job.status)) continue;
for (const prior of dismissed) {
if (
jobsMatchEmployerAndTitleDismiss({
employerA: job.employer,
titleA: job.title,
employerB: prior.employer,
titleB: prior.title,
})
) {
hints.set(job.id, prior.reason);
if (!openStatuses.has(job.status)) continue;
for (const key of collectJobDedupKeys({
employer: job.employer,
title: job.title,
})) {
const reason = dismissedKeys.get(key);
if (reason) {
hints.set(job.id, reason);
break;
}
}
@@ -62,3 +46,5 @@ export function buildDuplicateDismissHints(
return hints;
}
export { collectJobDedupKeys };
@@ -0,0 +1,288 @@
import type { UpdateSettingsInput } from "@shared/settings-schema.js";
import type { RxResumeMode, ValidationResult } from "@shared/types.js";
export type RxResumeSettingsLike =
| {
rxresumeMode?: { value?: string | null } | null;
rxresumeEmail?: string | null;
rxresumeUrl?: string | null;
rxresumePasswordHint?: string | null;
rxresumeApiKeyHint?: string | null;
rxresumeBaseResumeId?: string | null;
rxresumeBaseResumeIdV4?: string | null;
rxresumeBaseResumeIdV5?: string | null;
}
| null
| undefined;
export const RXRESUME_MODES = ["v4", "v5"] as const;
export const RXRESUME_PRECHECK_MESSAGES = {
"missing-v4-email-password": "Add v4 email and password, then test again.",
"missing-v5-api-key": "Add a v5 API key, then test again.",
} as const;
export const coerceRxResumeMode = (
value: unknown,
fallback: RxResumeMode = "v5",
): RxResumeMode => (value === "v4" || value === "v5" ? value : fallback);
export const getStoredRxResumeCredentialAvailability = (
settings: RxResumeSettingsLike,
) => {
const email = Boolean(settings?.rxresumeEmail?.trim());
const password = Boolean(settings?.rxresumePasswordHint);
const apiKey = Boolean(settings?.rxresumeApiKeyHint);
return { email, password, apiKey, hasV4: email && password, hasV5: apiKey };
};
export const getInitialRxResumeMode = (input: {
savedMode: RxResumeMode | null | undefined;
hasV4: boolean;
hasV5: boolean;
}): RxResumeMode =>
coerceRxResumeMode(
input.savedMode ?? (input.hasV4 && !input.hasV5 ? "v4" : "v5"),
);
export const getRxResumeBaseResumeSelection = (
settings: RxResumeSettingsLike,
mode: RxResumeMode,
) => {
const idsByMode = {
v4:
settings?.rxresumeBaseResumeIdV4 ??
(mode === "v4" ? (settings?.rxresumeBaseResumeId ?? null) : null),
v5:
settings?.rxresumeBaseResumeIdV5 ??
(mode === "v5" ? (settings?.rxresumeBaseResumeId ?? null) : null),
} satisfies Record<RxResumeMode, string | null>;
return { idsByMode, selectedId: idsByMode[mode] ?? null };
};
export const getRxResumeCredentialDrafts = (input: {
rxresumeEmail?: string | null;
rxresumeUrl?: string | null;
rxresumePassword?: string | null;
rxresumeApiKey?: string | null;
}) => ({
email: input.rxresumeEmail?.trim() ?? "",
baseUrl: input.rxresumeUrl?.trim() ?? "",
password: input.rxresumePassword?.trim() ?? "",
apiKey: input.rxresumeApiKey?.trim() ?? "",
});
export type RxResumeCredentialDrafts = ReturnType<
typeof getRxResumeCredentialDrafts
>;
export type RxResumeStoredCredentialAvailability = Pick<
ReturnType<typeof getStoredRxResumeCredentialAvailability>,
"email" | "password" | "apiKey"
>;
export const getRxResumeCredentialPrecheckFailure = (input: {
mode: RxResumeMode;
stored: RxResumeStoredCredentialAvailability;
draft: RxResumeCredentialDrafts;
}) => {
const hasV4 =
(input.stored.email || Boolean(input.draft.email)) &&
(input.stored.password || Boolean(input.draft.password));
const hasV5 = input.stored.apiKey || Boolean(input.draft.apiKey);
if (input.mode === "v5" && !hasV5) return "missing-v5-api-key" as const;
if (input.mode === "v4" && !hasV4)
return "missing-v4-email-password" as const;
return null;
};
export type RxResumeCredentialPrecheckFailure = ReturnType<
typeof getRxResumeCredentialPrecheckFailure
>;
export const getRxResumeMissingCredentialLabels = (input: {
mode: RxResumeMode;
stored: RxResumeStoredCredentialAvailability;
draft: RxResumeCredentialDrafts;
}) =>
input.mode === "v5"
? input.stored.apiKey || input.draft.apiKey
? []
: ["RxResume v5 API key"]
: [
...(input.stored.email || input.draft.email ? [] : ["RxResume email"]),
...(input.stored.password || input.draft.password
? []
: ["RxResume password"]),
];
export const toRxResumeValidationPayload = (
draft: RxResumeCredentialDrafts,
options?: {
preserveBlankFields?: Array<keyof RxResumeCredentialDrafts>;
},
) => {
const preserveBlankFields = new Set(options?.preserveBlankFields ?? []);
return {
email: preserveBlankFields.has("email")
? draft.email
: draft.email || undefined,
baseUrl: preserveBlankFields.has("baseUrl")
? draft.baseUrl
: draft.baseUrl || undefined,
password: preserveBlankFields.has("password")
? draft.password
: draft.password || undefined,
apiKey: preserveBlankFields.has("apiKey")
? draft.apiKey
: draft.apiKey || undefined,
};
};
export const isRxResumeBlockingValidationFailure = (
validation: ValidationResult,
): boolean =>
!validation.valid &&
typeof validation.status === "number" &&
validation.status >= 400 &&
validation.status < 500;
export const isRxResumeAvailabilityValidationFailure = (
validation: ValidationResult,
): boolean =>
!validation.valid &&
(validation.status === 0 ||
(typeof validation.status === "number" && validation.status >= 500));
export const buildRxResumeSettingsUpdate = (
mode: RxResumeMode,
draft: RxResumeCredentialDrafts,
): Partial<UpdateSettingsInput> => {
const update: Partial<UpdateSettingsInput> = {
rxresumeMode: mode,
rxresumeUrl: draft.baseUrl || null,
};
if (draft.email) update.rxresumeEmail = draft.email;
if (draft.password) update.rxresumePassword = draft.password;
if (draft.apiKey) update.rxresumeApiKey = draft.apiKey;
return update;
};
type ValidateAndMaybePersistRxResumeModeInput<TSettings> = {
mode: RxResumeMode;
stored: RxResumeStoredCredentialAvailability;
draft: RxResumeCredentialDrafts;
validate: (
payload: { mode: RxResumeMode } & ReturnType<
typeof toRxResumeValidationPayload
>,
) => Promise<ValidationResult>;
persist?: (update: Partial<UpdateSettingsInput>) => Promise<TSettings>;
persistOnSuccess?: boolean;
skipPrecheck?: boolean;
getPrecheckMessage?: (
failure: Exclude<RxResumeCredentialPrecheckFailure, null>,
) => string;
getValidationErrorMessage?: (error: unknown, mode: RxResumeMode) => string;
getPersistErrorMessage?: (error: unknown, mode: RxResumeMode) => string;
};
export type ValidateAndMaybePersistRxResumeModeResult<TSettings> = {
validation: ValidationResult;
precheckFailure: RxResumeCredentialPrecheckFailure;
updatedSettings: TSettings | null;
};
export const validateAndMaybePersistRxResumeMode = async <TSettings>(
input: ValidateAndMaybePersistRxResumeModeInput<TSettings>,
): Promise<ValidateAndMaybePersistRxResumeModeResult<TSettings>> => {
const {
mode,
stored,
draft,
validate,
persist,
persistOnSuccess = false,
skipPrecheck = false,
getPrecheckMessage = (failure) => RXRESUME_PRECHECK_MESSAGES[failure],
getValidationErrorMessage = (error) =>
error instanceof Error ? error.message : "RxResume validation failed",
getPersistErrorMessage = (error) =>
error instanceof Error
? error.message
: "Failed to save RxResume settings",
} = input;
const precheckFailure = skipPrecheck
? null
: getRxResumeCredentialPrecheckFailure({
mode,
stored,
draft,
});
if (precheckFailure !== null) {
return {
validation: {
valid: false,
message: getPrecheckMessage(precheckFailure),
status: 400,
},
precheckFailure,
updatedSettings: null,
};
}
let validation: ValidationResult;
try {
validation = await validate({
mode,
...toRxResumeValidationPayload(draft),
});
} catch (error) {
return {
validation: {
valid: false,
message: getValidationErrorMessage(error, mode),
status: 0,
},
precheckFailure: null,
updatedSettings: null,
};
}
if (!validation.valid || !persistOnSuccess || !persist) {
return {
validation: {
valid: validation.valid,
message: validation.valid ? null : (validation.message ?? null),
status: validation.valid ? null : (validation.status ?? null),
},
precheckFailure: null,
updatedSettings: null,
};
}
try {
const updatedSettings = await persist(
buildRxResumeSettingsUpdate(mode, draft),
);
return {
validation: {
valid: true,
message: null,
status: null,
},
precheckFailure: null,
updatedSettings,
};
} catch (error) {
return {
validation: {
valid: false,
message: getPersistErrorMessage(error, mode),
status: 0,
},
precheckFailure: null,
updatedSettings: null,
};
}
};
@@ -39,7 +39,6 @@ const makeJob = (overrides: Partial<JobListItem>): JobListItem => ({
closedAt: null,
suitabilityScore: null,
sponsorMatchScore: null,
sponsorshipSignals: null,
jobType: null,
jobFunction: null,
salaryMinAmount: null,
@@ -55,6 +55,7 @@ import {
} from "@/lib/utils";
import * as api from "../api";
import { ConfirmDelete } from "../components/ConfirmDelete";
import { GhostwriterDrawer } from "../components/ghostwriter/GhostwriterDrawer";
import { JobDetailsEditDrawer } from "../components/JobDetailsEditDrawer";
import { JobHeader } from "../components/JobHeader";
import {
@@ -586,6 +587,7 @@ export const JobPage: React.FC = () => {
<CalendarClock className="h-4 w-4" />
Application details
</CardTitle>
<GhostwriterDrawer job={job} />
</div>
</CardHeader>
<CardContent className="space-y-4">
@@ -60,7 +60,6 @@ let mockAutomaticRunValues: AutomaticRunValues = {
topN: 12,
minSuitabilityScore: 55,
searchTerms: ["backend"],
activeKeywordSetId: null,
runBudget: 150,
country: "united kingdom",
cityLocations: [],
@@ -408,7 +407,6 @@ describe("OrchestratorPage", () => {
topN: 12,
minSuitabilityScore: 55,
searchTerms: ["backend"],
activeKeywordSetId: null,
runBudget: 150,
country: "united kingdom",
cityLocations: [],
@@ -761,21 +759,17 @@ describe("OrchestratorPage", () => {
fireEvent.click(screen.getByTestId("run-automatic"));
await waitFor(() => {
expect(api.updateSettings).toHaveBeenCalledWith(
expect.objectContaining({
searchTerms: ["backend"],
workplaceTypes: ["remote", "hybrid", "onsite"],
jobspyResultsWanted: 150,
gradcrackerMaxJobsPerTerm: 150,
ukvisajobsMaxJobs: 150,
adzunaMaxJobsPerTerm: 150,
startupjobsMaxJobsPerTerm: 150,
workingnomadsMaxJobsPerTerm: 150,
testdevjobsMaxPages: 10,
jobspyCountryIndeed: "united kingdom",
searchCities: "United Kingdom",
}),
);
expect(api.updateSettings).toHaveBeenCalledWith({
searchTerms: ["backend"],
workplaceTypes: ["remote", "hybrid", "onsite"],
jobspyResultsWanted: 150,
gradcrackerMaxJobsPerTerm: 150,
ukvisajobsMaxJobs: 150,
adzunaMaxJobsPerTerm: 150,
startupjobsMaxJobsPerTerm: 150,
jobspyCountryIndeed: "united kingdom",
searchCities: "United Kingdom",
});
});
expect(api.runPipeline).toHaveBeenCalledWith({
topN: 12,
@@ -796,7 +790,6 @@ describe("OrchestratorPage", () => {
topN: 12,
minSuitabilityScore: 55,
searchTerms: ["backend"],
activeKeywordSetId: null,
runBudget: 150,
country: "united kingdom",
cityLocations: ["London", "Manchester"],
@@ -832,7 +825,6 @@ describe("OrchestratorPage", () => {
topN: 12,
minSuitabilityScore: 55,
searchTerms: ["backend"],
activeKeywordSetId: null,
runBudget: 150,
country: "united kingdom",
cityLocations: ["Leeds", "Manchester"],
@@ -868,7 +860,6 @@ describe("OrchestratorPage", () => {
topN: 12,
minSuitabilityScore: 55,
searchTerms: ["backend"],
activeKeywordSetId: null,
runBudget: 150,
country: "united kingdom",
cityLocations: ["Leeds", "Manchester"],
@@ -976,7 +967,6 @@ describe("OrchestratorPage", () => {
topN: 12,
minSuitabilityScore: 55,
searchTerms: ["backend"],
activeKeywordSetId: null,
runBudget: 150,
country: "united states",
cityLocations: [],
@@ -47,10 +47,6 @@ export const OrchestratorPage: React.FC = () => {
setCountrySelection,
sponsorFilter,
setSponsorFilter,
sponsorshipSignalsFilter,
setSponsorshipSignalsFilter,
hideSponsorBlockers,
setHideSponsorBlockers,
workplaceFilter,
setWorkplaceFilter,
salaryFilter,
@@ -63,8 +59,6 @@ export const OrchestratorPage: React.FC = () => {
setEmployerFilterTokens,
applySettingsCompanySkipList,
setApplySettingsCompanySkipList,
hidePriorSkips,
setHidePriorSkips,
sort,
setSort,
resetFilters,
@@ -75,7 +69,7 @@ export const OrchestratorPage: React.FC = () => {
if (tab && validTabs.includes(tab as FilterTab)) {
return tab as FilterTab;
}
return "discovered";
return "ready";
}, [tab]);
// Helper to change URL while preserving search params
@@ -96,7 +90,7 @@ export const OrchestratorPage: React.FC = () => {
// Effect to sync URL if it was invalid
useEffect(() => {
if (tab === "in_progress") {
navigate("/jobs/discovered", { replace: true });
navigate("/applications/in-progress", { replace: true });
return;
}
const validTabs: FilterTab[] = ["ready", "discovered", "applied", "all"];
@@ -175,8 +169,8 @@ export const OrchestratorPage: React.FC = () => {
);
const duplicateDismissHints = useMemo(
() => (hidePriorSkips ? buildDuplicateDismissHints(jobs) : undefined),
[hidePriorSkips, jobs],
() => buildDuplicateDismissHints(jobs),
[jobs],
);
const jobListFilterExtras = useMemo(
@@ -211,8 +205,6 @@ export const OrchestratorPage: React.FC = () => {
countriesFilter,
countriesExcludeFilter,
sponsorFilter,
sponsorshipSignalsFilter,
hideSponsorBlockers,
workplaceFilter,
salaryFilter,
sort,
@@ -395,8 +387,6 @@ export const OrchestratorPage: React.FC = () => {
"countries",
"countriesExclude",
"sponsor",
"sponsorSignal",
"hideSponsorBlockers",
"salaryMode",
"salaryMin",
"salaryMax",
@@ -533,14 +523,8 @@ export const OrchestratorPage: React.FC = () => {
onApplySettingsCompanySkipListChange={
setApplySettingsCompanySkipList
}
hidePriorSkips={hidePriorSkips}
onHidePriorSkipsChange={setHidePriorSkips}
sponsorFilter={sponsorFilter}
onSponsorFilterChange={setSponsorFilter}
sponsorshipSignalsFilter={sponsorshipSignalsFilter}
onSponsorshipSignalsFilterChange={setSponsorshipSignalsFilter}
hideSponsorBlockers={hideSponsorBlockers}
onHideSponsorBlockersChange={setHideSponsorBlockers}
workplaceFilter={workplaceFilter}
onWorkplaceFilterChange={setWorkplaceFilter}
salaryFilter={salaryFilter}
@@ -2,8 +2,9 @@ import { createAppSettings } from "@shared/testing/factories.js";
import { fireEvent, screen, waitFor } from "@testing-library/react";
import { MemoryRouter } from "react-router-dom";
import { toast } from "sonner";
import { afterAll, beforeEach, describe, expect, it, vi } from "vitest";
import { beforeEach, describe, expect, it, vi } from "vitest";
import * as api from "../api";
import { _resetTracerReadinessCache } from "../hooks/useTracerReadiness";
import { renderWithQueryClient } from "../test/renderWithQueryClient";
import { SettingsPage } from "./SettingsPage";
@@ -16,8 +17,14 @@ vi.mock("../api", () => ({
getSettings: vi.fn(),
getLlmModels: vi.fn().mockResolvedValue([]),
updateSettings: vi.fn(),
validateRxresume: vi.fn(),
getRxResumeProjects: vi.fn(),
clearDatabase: vi.fn(),
deleteJobsByStatus: vi.fn(),
getTracerReadiness: vi.fn(),
getBackups: vi.fn().mockResolvedValue({ backups: [], nextScheduled: null }),
createManualBackup: vi.fn(),
deleteBackup: vi.fn(),
}));
vi.mock("sonner", () => ({
@@ -60,6 +67,20 @@ const openModelSection = async () => {
fireEvent.click(modelTrigger);
};
const openWritingStyleSection = async () => {
const chatTrigger = await screen.findByRole("button", {
name: /writing style & language/i,
});
fireEvent.click(chatTrigger);
};
const openReactiveResumeSection = async () => {
const trigger = await screen.findByRole("button", {
name: /reactive resume/i,
});
fireEvent.click(trigger);
};
describe("SettingsPage", () => {
beforeEach(() => {
vi.clearAllMocks();
@@ -67,6 +88,21 @@ describe("SettingsPage", () => {
configurable: true,
value: vi.fn(),
});
_resetTracerReadinessCache();
vi.mocked(api.getTracerReadiness).mockResolvedValue({
status: "ready",
canEnable: true,
publicBaseUrl: "https://my-jobops.example.com",
healthUrl: "https://my-jobops.example.com/health",
checkedAt: Date.now(),
lastSuccessAt: Date.now(),
reason: null,
});
vi.mocked(api.validateRxresume).mockResolvedValue({
valid: false,
message: "Missing credentials",
status: 400,
});
});
afterAll(() => {
@@ -226,7 +262,7 @@ describe("SettingsPage", () => {
);
});
it("hides pipeline tuning and unused settings sections", async () => {
it("hides pipeline tuning sections that moved to run modal", async () => {
vi.mocked(api.getSettings).mockResolvedValue(baseSettings);
renderPage();
@@ -243,21 +279,243 @@ describe("SettingsPage", () => {
expect(
screen.queryByRole("button", { name: /jobspy scraper/i }),
).not.toBeInTheDocument();
});
it("enables save button when display setting is changed", async () => {
vi.mocked(api.getSettings).mockResolvedValue(baseSettings);
renderPage();
const saveButton = screen.getByRole("button", { name: /^save$/i });
const displayTrigger = await screen.findByRole("button", {
name: /display settings/i,
});
fireEvent.click(displayTrigger);
const sponsorCheckbox = screen.getByLabelText(
/show visa sponsor information/i,
);
fireEvent.click(sponsorCheckbox);
await waitFor(() => expect(saveButton).toBeEnabled());
});
it("allows saving when both Reactive Resume v4 and v5 credentials are present", async () => {
const settingsWithBothRxResumeAuth = createAppSettings({
rxresumeEmail: "resume@example.com",
rxresumePasswordHint: "pass",
rxresumeApiKeyHint: "api_",
});
vi.mocked(api.getSettings).mockResolvedValue(settingsWithBothRxResumeAuth);
vi.mocked(api.updateSettings).mockResolvedValue(
settingsWithBothRxResumeAuth,
);
renderPage();
const displayTrigger = await screen.findByRole("button", {
name: /display settings/i,
});
fireEvent.click(displayTrigger);
const sponsorCheckbox = screen.getByLabelText(
/show visa sponsor information/i,
);
fireEvent.click(sponsorCheckbox);
const saveButton = screen.getByRole("button", { name: /^save$/i });
await waitFor(() => expect(saveButton).toBeEnabled());
fireEvent.click(saveButton);
await waitFor(() => expect(api.updateSettings).toHaveBeenCalled());
expect(toast.error).not.toHaveBeenCalledWith(
"Choose one Reactive Resume auth method",
expect.anything(),
);
});
it("saves a shared RxResume URL from the Reactive Resume section", async () => {
vi.mocked(api.getSettings).mockResolvedValue(baseSettings);
vi.mocked(api.updateSettings).mockResolvedValue({
...baseSettings,
rxresumeUrl: "https://resume.example.com",
});
renderPage();
const reactiveResumeTrigger = await screen.findByRole("button", {
name: /reactive resume/i,
});
fireEvent.click(reactiveResumeTrigger);
const urlInput = screen.getByLabelText(/rxresume url/i);
await waitFor(() => expect(urlInput).toBeEnabled());
fireEvent.change(urlInput, {
target: { value: "https://resume.example.com" },
});
const saveButton = screen.getByRole("button", { name: /^save$/i });
await waitFor(() => expect(saveButton).toBeEnabled());
fireEvent.click(saveButton);
await waitFor(() => expect(api.updateSettings).toHaveBeenCalled());
expect(api.updateSettings).toHaveBeenCalledWith(
expect.objectContaining({
rxresumeUrl: "https://resume.example.com",
}),
);
});
it("blocks save and renders an inline alert when the v5 API key is invalid", async () => {
vi.mocked(api.getSettings).mockResolvedValue(baseSettings);
renderPage();
await openReactiveResumeSection();
await waitFor(() => expect(api.validateRxresume).toHaveBeenCalled());
vi.mocked(api.validateRxresume).mockClear();
vi.mocked(api.validateRxresume).mockResolvedValue({
valid: false,
message:
"Reactive Resume v5 API key is invalid. Update the API key and try again.",
status: 401,
});
fireEvent.change(screen.getByLabelText(/v5 api key/i), {
target: { value: "invalid-v5-key" },
});
const saveButton = screen.getByRole("button", { name: /^save$/i });
await waitFor(() => expect(saveButton).toBeEnabled());
fireEvent.click(saveButton);
expect(
screen.queryByRole("button", { name: /display settings/i }),
).not.toBeInTheDocument();
await screen.findByText(/Reactive Resume v5 API key is invalid/i),
).toBeInTheDocument();
expect(api.updateSettings).not.toHaveBeenCalled();
});
it("allows saving on RxResume availability warnings and keeps the inline warning visible", async () => {
vi.mocked(api.getSettings).mockResolvedValue(baseSettings);
vi.mocked(api.updateSettings).mockResolvedValue({
...baseSettings,
rxresumeApiKeyHint: "rr-v",
});
renderPage();
await openReactiveResumeSection();
await waitFor(() => expect(api.validateRxresume).toHaveBeenCalled());
vi.mocked(api.validateRxresume).mockClear();
vi.mocked(api.validateRxresume).mockResolvedValue({
valid: false,
message:
"JobOps could not verify Reactive Resume because the instance is unavailable right now.",
status: 0,
});
fireEvent.change(screen.getByLabelText(/v5 api key/i), {
target: { value: "rr-v5-warning-key" },
});
const saveButton = screen.getByRole("button", { name: /^save$/i });
await waitFor(() => expect(saveButton).toBeEnabled());
fireEvent.click(saveButton);
await waitFor(() => expect(api.updateSettings).toHaveBeenCalled());
expect(
screen.queryByRole("button", { name: /writing style & language/i }),
).not.toBeInTheDocument();
await screen.findByText(/instance is unavailable right now/i),
).toBeInTheDocument();
expect(toast.success).toHaveBeenCalledWith("Settings saved");
expect(toast.info).toHaveBeenCalledWith(
"Settings saved, but JobOps could not verify Reactive Resume because the instance is unavailable.",
);
});
it("does not run RxResume validation for unrelated settings saves", async () => {
vi.mocked(api.getSettings).mockResolvedValue(baseSettings);
vi.mocked(api.updateSettings).mockResolvedValue({
...baseSettings,
model: {
value: "new-model",
default: baseSettings.model.default,
override: "new-model",
},
});
renderPage();
await openModelSection();
await waitFor(() => expect(api.validateRxresume).toHaveBeenCalled());
vi.mocked(api.validateRxresume).mockClear();
fireEvent.change(screen.getByLabelText(/default model/i), {
target: { value: "new-model" },
});
const saveButton = screen.getByRole("button", { name: /^save$/i });
await waitFor(() => expect(saveButton).toBeEnabled());
fireEvent.click(saveButton);
await waitFor(() => expect(api.updateSettings).toHaveBeenCalled());
expect(api.validateRxresume).not.toHaveBeenCalled();
});
it("clears the previous RxResume warning when the key or URL changes", async () => {
vi.mocked(api.getSettings).mockResolvedValue(baseSettings);
vi.mocked(api.validateRxresume).mockResolvedValue({
valid: false,
message:
"JobOps could not verify Reactive Resume because the instance is unavailable right now.",
status: 0,
});
renderPage();
await openReactiveResumeSection();
expect(
screen.queryByRole("button", { name: /backup/i }),
).not.toBeInTheDocument();
await screen.findByText(/instance is unavailable right now/i),
).toBeInTheDocument();
fireEvent.change(screen.getByLabelText(/rxresume url/i), {
target: { value: "https://resume.example.com" },
});
await waitFor(() =>
expect(
screen.queryByText(/instance is unavailable right now/i),
).not.toBeInTheDocument(),
);
});
it("saves the writing language mode through the settings page", async () => {
vi.mocked(api.getSettings).mockResolvedValue(baseSettings);
vi.mocked(api.updateSettings).mockResolvedValue(
createAppSettings({
chatStyleLanguageMode: {
value: "match-resume",
default: "manual",
override: "match-resume",
},
}),
);
renderPage();
await openWritingStyleSection();
fireEvent.click(screen.getByRole("combobox", { name: /output language/i }));
fireEvent.click(await screen.findByText("Match current resume language"));
expect(
screen.queryByRole("button", { name: /tracer/i }),
).not.toBeInTheDocument();
expect(
screen.queryByRole("button", { name: /resume projects/i }),
screen.queryByRole("combobox", { name: /specific language/i }),
).not.toBeInTheDocument();
const saveButton = screen.getByRole("button", { name: /^save$/i });
await waitFor(() => expect(saveButton).toBeEnabled());
fireEvent.click(saveButton);
await waitFor(() => expect(api.updateSettings).toHaveBeenCalled());
expect(api.updateSettings).toHaveBeenCalledWith(
expect.objectContaining({
chatStyleLanguageMode: "match-resume",
chatStyleManualLanguage: null,
}),
);
});
it("enables save button when basic auth toggle is changed", async () => {
+581 -243
View File
@@ -1,13 +1,29 @@
import * as api from "@client/api";
import { PageHeader } from "@client/components/layout";
import { useUpdateSettingsMutation } from "@client/hooks/queries/useSettingsMutation";
import { useRxResumeConfigState } from "@client/hooks/useRxResumeConfigState";
import { useTracerReadiness } from "@client/hooks/useTracerReadiness";
import {
coerceRxResumeMode,
getRxResumeCredentialDrafts,
getRxResumeCredentialPrecheckFailure,
isRxResumeAvailabilityValidationFailure,
isRxResumeBlockingValidationFailure,
RXRESUME_MODES,
RXRESUME_PRECHECK_MESSAGES,
toRxResumeValidationPayload,
validateAndMaybePersistRxResumeMode,
} from "@client/lib/rxresume-config";
import { BackupSettingsSection } from "@client/pages/settings/components/BackupSettingsSection";
import { ChatSettingsSection } from "@client/pages/settings/components/ChatSettingsSection";
import { DangerZoneSection } from "@client/pages/settings/components/DangerZoneSection";
import { DisplaySettingsSection } from "@client/pages/settings/components/DisplaySettingsSection";
import { EnvironmentSettingsSection } from "@client/pages/settings/components/EnvironmentSettingsSection";
import { JobSearchProfileSection } from "@client/pages/settings/components/JobSearchProfileSection";
import { JobSourcesSettingsSection } from "@client/pages/settings/components/JobSourcesSettingsSection";
import { KeywordSetsSettingsSection } from "@client/pages/settings/components/KeywordSetsSettingsSection";
import { ModelSettingsSection } from "@client/pages/settings/components/ModelSettingsSection";
import { ReactiveResumeSection } from "@client/pages/settings/components/ReactiveResumeSection";
import { ScoringSettingsSection } from "@client/pages/settings/components/ScoringSettingsSection";
import { TracerLinksSettingsSection } from "@client/pages/settings/components/TracerLinksSettingsSection";
import { WebhooksSection } from "@client/pages/settings/components/WebhooksSection";
import {
type LlmProviderId,
@@ -21,12 +37,24 @@ import {
type UpdateSettingsInput,
updateSettingsSchema,
} from "@shared/settings-schema.js";
import type { AppSettings, JobStatus } from "@shared/types.js";
import { useQuery } from "@tanstack/react-query";
import type {
AppSettings,
JobStatus,
ResumeProjectCatalogItem,
ResumeProjectsSettings,
RxResumeMode,
ValidationResult,
} from "@shared/types.js";
import { useQuery, useQueryClient } from "@tanstack/react-query";
import { Settings } from "lucide-react";
import type React from "react";
import { useEffect, useState } from "react";
import { FormProvider, type Resolver, useForm } from "react-hook-form";
import { useCallback, useEffect, useState } from "react";
import {
FormProvider,
type Resolver,
useForm,
useWatch,
} from "react-hook-form";
import { toast } from "sonner";
import { useQueryErrorToast } from "@/client/hooks/useQueryErrorToast";
import { queryKeys } from "@/client/lib/queryKeys";
@@ -77,20 +105,37 @@ const DEFAULT_FORM_VALUES: UpdateSettingsInput = {
blockedCompanyKeywords: [],
blockedCountries: [],
scoringInstructions: "",
ashbyCompanies: [],
greenhouseCompanies: [],
leverCompanies: [],
smartrecruitersCompanies: [],
teamtailorCompanies: [],
huntflowTenants: [],
factorialTenants: [],
workdayTenants: [],
careersPageUrls: [],
icimsTenants: [],
elutaRssLocations: [],
};
type LlmProviderValue = LlmProviderId | null;
type RxResumeValidationBadgeState = {
checked: boolean;
valid: boolean;
message: string | null;
status: number | null;
};
const EMPTY_RXRESUME_VALIDATION_BADGE_STATE: RxResumeValidationBadgeState = {
checked: false,
valid: false,
message: null,
status: null,
};
const getRxResumeValidationFieldsForMode = (
mode: RxResumeMode,
): Array<keyof UpdateSettingsInput> =>
mode === "v5"
? ["rxresumeApiKey", "rxresumeUrl"]
: ["rxresumeEmail", "rxresumePassword", "rxresumeUrl"];
const toRxResumeValidationBadgeState = (
validation: ValidationResult,
): RxResumeValidationBadgeState => ({
checked: true,
valid: validation.valid,
message: validation.valid ? null : (validation.message ?? null),
status: validation.valid ? null : (validation.status ?? null),
});
const normalizeLlmProviderValue = (
value: string | null | undefined,
@@ -188,17 +233,6 @@ const mapSettingsToForm = (data: AppSettings): UpdateSettingsInput => ({
blockedCompanyKeywords: data.blockedCompanyKeywords.override ?? [],
blockedCountries: data.blockedCountries.override ?? [],
scoringInstructions: data.scoringInstructions.override ?? "",
ashbyCompanies: data.ashbyCompanies.value,
greenhouseCompanies: data.greenhouseCompanies.value,
leverCompanies: data.leverCompanies.value,
smartrecruitersCompanies: data.smartrecruitersCompanies.value,
teamtailorCompanies: data.teamtailorCompanies.value,
huntflowTenants: data.huntflowTenants.value,
factorialTenants: data.factorialTenants.value,
workdayTenants: data.workdayTenants.value,
careersPageUrls: data.careersPageUrls.value,
icimsTenants: data.icimsTenants.value,
elutaRssLocations: data.elutaRssLocations.value,
});
const normalizeString = (value: string | null | undefined) => {
@@ -220,7 +254,43 @@ const stringArraysEqual = (left: string[], right: string[]): boolean => {
const nullIfSame = <T,>(value: T | null | undefined, defaultValue: T) =>
value === defaultValue ? null : (value ?? null);
const normalizeResumeProjectsForCatalog = (
catalog: ResumeProjectCatalogItem[],
current: ResumeProjectsSettings | null,
): ResumeProjectsSettings | null => {
const allowed = new Set(catalog.map((project) => project.id));
const base = current ?? {
maxProjects: 0,
lockedProjectIds: catalog
.filter((project) => project.isVisibleInBase)
.map((project) => project.id),
aiSelectableProjectIds: [],
};
const lockedProjectIds = base.lockedProjectIds.filter((id) =>
allowed.has(id),
);
const lockedSet = new Set(lockedProjectIds);
const aiSelectableProjectIds = (
current ? base.aiSelectableProjectIds : catalog.map((project) => project.id)
)
.filter((id) => allowed.has(id))
.filter((id) => !lockedSet.has(id));
const maxProjectsRaw = Number.isFinite(base.maxProjects)
? base.maxProjects
: 0;
const maxProjectsInt = Math.max(0, Math.floor(maxProjectsRaw));
const maxProjects = Math.min(
catalog.length,
Math.max(lockedProjectIds.length, maxProjectsInt, 3),
);
return { maxProjects, lockedProjectIds, aiSelectableProjectIds };
};
const getDerivedSettings = (settings: AppSettings | null) => {
const profileProjects = settings?.profileProjects ?? [];
return {
model: {
effective: settings?.model?.value ?? "",
@@ -250,13 +320,41 @@ const getDerivedSettings = (settings: AppSettings | null) => {
default: settings?.renderMarkdownInJobDescriptions?.default ?? true,
},
},
chat: {
tone: {
effective: settings?.chatStyleTone?.value ?? "professional",
default: settings?.chatStyleTone?.default ?? "professional",
},
formality: {
effective: settings?.chatStyleFormality?.value ?? "medium",
default: settings?.chatStyleFormality?.default ?? "medium",
},
constraints: {
effective: settings?.chatStyleConstraints?.value ?? "",
default: settings?.chatStyleConstraints?.default ?? "",
},
doNotUse: {
effective: settings?.chatStyleDoNotUse?.value ?? "",
default: settings?.chatStyleDoNotUse?.default ?? "",
},
languageMode: {
effective: settings?.chatStyleLanguageMode?.value ?? "manual",
default: settings?.chatStyleLanguageMode?.default ?? "manual",
},
manualLanguage: {
effective: settings?.chatStyleManualLanguage?.value ?? "english",
default: settings?.chatStyleManualLanguage?.default ?? "english",
},
},
envSettings: {
readable: {
rxresumeEmail: settings?.rxresumeEmail ?? "",
ukvisajobsEmail: settings?.ukvisajobsEmail ?? "",
adzunaAppId: settings?.adzunaAppId ?? "",
basicAuthUser: settings?.basicAuthUser ?? "",
},
private: {
rxresumePasswordHint: settings?.rxresumePasswordHint ?? null,
ukvisajobsPasswordHint: settings?.ukvisajobsPasswordHint ?? null,
adzunaAppKeyHint: settings?.adzunaAppKeyHint ?? null,
basicAuthPasswordHint: settings?.basicAuthPasswordHint ?? null,
@@ -265,6 +363,10 @@ const getDerivedSettings = (settings: AppSettings | null) => {
basicAuthActive: settings?.basicAuthActive ?? false,
},
defaultResumeProjects: settings?.resumeProjects?.default ?? null,
profileProjects,
maxProjectsTotal: profileProjects.length,
backup: {
backupEnabled: {
effective: settings?.backupEnabled?.value ?? false,
@@ -309,61 +411,41 @@ const getDerivedSettings = (settings: AppSettings | null) => {
default: settings?.scoringInstructions?.default ?? "",
},
},
jobSources: {
ashbyCompanies: {
effective: settings?.ashbyCompanies?.value ?? [],
default: settings?.ashbyCompanies?.default ?? [],
},
greenhouseCompanies: {
effective: settings?.greenhouseCompanies?.value ?? [],
default: settings?.greenhouseCompanies?.default ?? [],
},
leverCompanies: {
effective: settings?.leverCompanies?.value ?? [],
default: settings?.leverCompanies?.default ?? [],
},
smartrecruitersCompanies: {
effective: settings?.smartrecruitersCompanies?.value ?? [],
default: settings?.smartrecruitersCompanies?.default ?? [],
},
teamtailorCompanies: {
effective: settings?.teamtailorCompanies?.value ?? [],
default: settings?.teamtailorCompanies?.default ?? [],
},
huntflowTenants: {
effective: settings?.huntflowTenants?.value ?? [],
default: settings?.huntflowTenants?.default ?? [],
},
factorialTenants: {
effective: settings?.factorialTenants?.value ?? [],
default: settings?.factorialTenants?.default ?? [],
},
workdayTenants: {
effective: settings?.workdayTenants?.value ?? [],
default: settings?.workdayTenants?.default ?? [],
},
careersPageUrls: {
effective: settings?.careersPageUrls?.value ?? [],
default: settings?.careersPageUrls?.default ?? [],
},
icimsTenants: {
effective: settings?.icimsTenants?.value ?? [],
default: settings?.icimsTenants?.default ?? [],
},
elutaRssLocations: {
effective: settings?.elutaRssLocations?.value ?? [],
default: settings?.elutaRssLocations?.default ?? [],
},
},
};
};
export const SettingsPage: React.FC = () => {
const queryClient = useQueryClient();
const [settings, setSettings] = useState<AppSettings | null>(null);
const [isSaving, setIsSaving] = useState(false);
const [rxresumeValidationStatuses, setRxresumeValidationStatuses] = useState<{
v4: RxResumeValidationBadgeState;
v5: RxResumeValidationBadgeState;
}>({
v4: EMPTY_RXRESUME_VALIDATION_BADGE_STATE,
v5: EMPTY_RXRESUME_VALIDATION_BADGE_STATE,
});
const [statusesToClear, setStatusesToClear] = useState<JobStatus[]>([
"discovered",
]);
const [rxResumeBaseResumeIdDraft, setRxResumeBaseResumeIdDraft] = useState<
string | null
>(null);
const [rxResumeProjectsOverride, setRxResumeProjectsOverride] = useState<
ResumeProjectCatalogItem[] | null
>(null);
const [isFetchingRxResumeProjects, setIsFetchingRxResumeProjects] =
useState(false);
// Backup state
const [isCreatingBackup, setIsCreatingBackup] = useState(false);
const [isDeletingBackup, setIsDeletingBackup] = useState(false);
const {
readiness: tracerReadiness,
isLoading: isTracerReadinessLoading,
isChecking: isTracerReadinessChecking,
refreshReadiness,
} = useTracerReadiness();
const methods = useForm<UpdateSettingsInput>({
resolver: zodResolver(
@@ -374,17 +456,49 @@ export const SettingsPage: React.FC = () => {
});
const {
clearErrors,
handleSubmit,
reset,
setError,
setValue,
getValues,
control,
formState: { isDirty, errors, isValid, dirtyFields },
} = methods;
const {
storedRxResume,
getBaseResumeIdForMode,
setBaseResumeIdForMode,
syncBaseResumeIdsForMode,
} = useRxResumeConfigState(settings);
const settingsQuery = useQuery({
queryKey: queryKeys.settings.current(),
queryFn: api.getSettings,
});
const backupsQuery = useQuery({
queryKey: queryKeys.backups.list(),
queryFn: api.getBackups,
});
const updateSettingsMutation = useUpdateSettingsMutation();
const isLoading = settingsQuery.isLoading;
const backups = backupsQuery.data?.backups ?? [];
const nextScheduled = backupsQuery.data?.nextScheduled ?? null;
const isLoadingBackups = backupsQuery.isLoading;
useQueryErrorToast(backupsQuery.error, "Failed to load backups");
const rxresumeMode = (settings?.rxresumeMode?.value ?? "v5") as RxResumeMode;
const selectedRxresumeMode = (useWatch({
control,
name: "rxresumeMode",
}) ?? rxresumeMode) as RxResumeMode;
const resumeProjectsValue = useWatch({
control,
name: "resumeProjects",
});
const hasRxResumeAccess = Boolean(
rxresumeValidationStatuses[selectedRxresumeMode].valid,
);
useEffect(() => {
if (!settingsQuery.data) return;
@@ -394,20 +508,251 @@ export const SettingsPage: React.FC = () => {
useQueryErrorToast(settingsQuery.error, "Failed to load settings");
useEffect(() => {
if (!settings) return;
const effectiveMode = coerceRxResumeMode(settings.rxresumeMode?.value);
const storedId = syncBaseResumeIdsForMode(effectiveMode);
setRxResumeBaseResumeIdDraft(storedId);
setValue("rxresumeBaseResumeId", storedId, { shouldDirty: false });
setRxResumeProjectsOverride(null);
}, [settings, setValue, syncBaseResumeIdsForMode]);
useEffect(() => {
let isMounted = true;
const controller = new AbortController();
if (!rxResumeBaseResumeIdDraft) {
setRxResumeProjectsOverride(null);
return () => {
isMounted = false;
controller.abort();
};
}
if (!hasRxResumeAccess)
return () => {
isMounted = false;
controller.abort();
};
setIsFetchingRxResumeProjects(true);
api
.getRxResumeProjects(
rxResumeBaseResumeIdDraft,
controller.signal,
selectedRxresumeMode,
)
.then((projects) => {
if (!isMounted) return;
setRxResumeProjectsOverride(projects);
const normalized = normalizeResumeProjectsForCatalog(
projects,
getValues("resumeProjects") ?? null,
);
if (normalized) {
setValue("resumeProjects", normalized, { shouldDirty: true });
}
})
.catch((error) => {
if (!isMounted || error.name === "AbortError") return;
const message =
error instanceof Error
? error.message
: "Failed to load RxResume projects";
toast.error(message);
setRxResumeProjectsOverride(null);
})
.finally(() => {
if (!isMounted) return;
setIsFetchingRxResumeProjects(false);
});
return () => {
isMounted = false;
controller.abort();
};
}, [
rxResumeBaseResumeIdDraft,
hasRxResumeAccess,
selectedRxresumeMode,
getValues,
setValue,
]);
const derived = getDerivedSettings(settings);
const {
model,
pipelineWebhook,
jobCompleteWebhook,
display,
chat,
envSettings,
defaultResumeProjects,
profileProjects,
backup,
scoring,
jobSearchProfile,
jobSources,
} = derived;
const handleCreateBackup = async () => {
setIsCreatingBackup(true);
try {
await api.createManualBackup();
toast.success("Backup created successfully");
await queryClient.invalidateQueries({ queryKey: queryKeys.backups.all });
} catch (error) {
const message =
error instanceof Error ? error.message : "Failed to create backup";
toast.error(message);
} finally {
setIsCreatingBackup(false);
}
};
const handleDeleteBackup = async (filename: string) => {
const confirmed = window.confirm(
`Delete backup "${filename}"? This action cannot be undone.`,
);
if (!confirmed) {
return;
}
setIsDeletingBackup(true);
try {
await api.deleteBackup(filename);
toast.success("Backup deleted successfully");
await queryClient.invalidateQueries({ queryKey: queryKeys.backups.all });
} catch (error) {
const message =
error instanceof Error ? error.message : "Failed to delete backup";
toast.error(message);
} finally {
setIsDeletingBackup(false);
}
};
const handleVerifyTracerReadiness = useCallback(async () => {
try {
const readiness = await refreshReadiness(true);
if (!readiness) {
toast.error("Tracer links are unavailable. Verify your public URL.");
} else if (readiness.canEnable) {
toast.success("Tracer links are ready");
} else {
toast.error(
readiness.reason ??
"Tracer links are unavailable. Verify your public URL.",
);
}
} catch (error) {
const message =
error instanceof Error
? error.message
: "Failed to verify tracer-link readiness";
toast.error(message);
}
}, [refreshReadiness]);
const setRxResumeValidationStatus = useCallback(
(mode: RxResumeMode, validation: ValidationResult) => {
setRxresumeValidationStatuses((current) => ({
...current,
[mode]: toRxResumeValidationBadgeState(validation),
}));
},
[],
);
const clearRxResumeValidationFeedback = useCallback(
(mode: RxResumeMode) => {
setRxresumeValidationStatuses((current) => ({
...current,
[mode]: EMPTY_RXRESUME_VALIDATION_BADGE_STATE,
}));
clearErrors(getRxResumeValidationFieldsForMode(mode));
},
[clearErrors],
);
const validateRxresumeMode = useCallback(
async (
mode: RxResumeMode,
options?: { silent?: boolean; persistOnSuccess?: boolean },
) => {
const { silent = false, persistOnSuccess = true } = options ?? {};
const notify = !silent;
const values = getValues();
const draftCredentials = getRxResumeCredentialDrafts(values);
const result = await validateAndMaybePersistRxResumeMode({
mode,
stored: storedRxResume,
draft: draftCredentials,
validate: api.validateRxresume,
persist: api.updateSettings,
persistOnSuccess,
skipPrecheck: silent,
getPrecheckMessage: (failure) => RXRESUME_PRECHECK_MESSAGES[failure],
getValidationErrorMessage: (error) =>
error instanceof Error ? error.message : "RxResume validation failed",
getPersistErrorMessage: (error) =>
error instanceof Error ? error.message : "RxResume validation failed",
});
setRxResumeValidationStatus(mode, result.validation);
if (result.updatedSettings) {
setSettings(result.updatedSettings);
queryClient.setQueryData(
queryKeys.settings.current(),
result.updatedSettings,
);
if (notify) {
toast.success(`Reactive Resume ${mode} validation passed`);
}
return;
}
if (!notify || result.validation.valid) {
return;
}
if (result.precheckFailure) {
toast.info(
result.validation.message ??
RXRESUME_PRECHECK_MESSAGES[result.precheckFailure],
);
return;
}
toast.error(
result.validation.message ||
`Reactive Resume ${mode} validation failed`,
);
},
[getValues, queryClient, setRxResumeValidationStatus, storedRxResume],
);
useEffect(() => {
if (!settings) return;
const modesToCheck = RXRESUME_MODES.filter(
(mode) => !rxresumeValidationStatuses[mode].checked,
);
if (modesToCheck.length === 0) return;
void Promise.all(
modesToCheck.map((mode) =>
validateRxresumeMode(mode, { silent: true, persistOnSuccess: false }),
),
);
}, [rxresumeValidationStatuses, settings, validateRxresumeMode]);
const effectiveProfileProjects =
rxResumeProjectsOverride ??
(selectedRxresumeMode === rxresumeMode ? profileProjects : []);
const effectiveMaxProjectsTotal = effectiveProfileProjects.length;
const lockedCount = resumeProjectsValue?.lockedProjectIds.length ?? 0;
const canSave = isDirty && isValid;
const onSave = async (data: UpdateSettingsInput) => {
@@ -436,6 +781,14 @@ export const SettingsPage: React.FC = () => {
const envPayload: Partial<UpdateSettingsInput> = {};
if (dirtyFields.rxresumeEmail || dirtyFields.rxresumePassword) {
envPayload.rxresumeEmail = normalizeString(data.rxresumeEmail);
}
if (dirtyFields.rxresumeUrl) {
envPayload.rxresumeUrl = normalizeString(data.rxresumeUrl);
}
if (dirtyFields.localResumeProfilePath) {
envPayload.localResumeProfilePath = normalizeString(
data.localResumeProfilePath,
@@ -481,6 +834,16 @@ export const SettingsPage: React.FC = () => {
if (value !== undefined) envPayload.llmApiKey = value;
}
if (dirtyFields.rxresumePassword) {
const value = normalizePrivateInput(data.rxresumePassword);
if (value !== undefined) envPayload.rxresumePassword = value;
}
if (dirtyFields.rxresumeApiKey) {
const value = normalizePrivateInput(data.rxresumeApiKey);
if (value !== undefined) envPayload.rxresumeApiKey = value;
}
if (dirtyFields.ukvisajobsPassword) {
const value = normalizePrivateInput(data.ukvisajobsPassword);
if (value !== undefined) envPayload.ukvisajobsPassword = value;
@@ -527,6 +890,12 @@ export const SettingsPage: React.FC = () => {
pipelineWebhookUrl: normalizeString(data.pipelineWebhookUrl),
jobCompleteWebhookUrl: normalizeString(data.jobCompleteWebhookUrl),
resumeProjects: resumeProjectsOverride,
...(dirtyFields.rxresumeMode
? { rxresumeMode: data.rxresumeMode ?? "v5" }
: {}),
...(dirtyFields.rxresumeBaseResumeId
? { rxresumeBaseResumeId: normalizeString(data.rxresumeBaseResumeId) }
: {}),
showSponsorInfo: nullIfSame(
data.showSponsorInfo,
display.showSponsorInfo.default,
@@ -582,107 +951,84 @@ export const SettingsPage: React.FC = () => {
normalizeString(data.scoringInstructions),
scoring.scoringInstructions.default,
),
ashbyCompanies: (() => {
const normalized = normalizeStringArray(data.ashbyCompanies);
return stringArraysEqual(
normalized,
jobSources.ashbyCompanies.default,
)
? null
: normalized;
})(),
greenhouseCompanies: (() => {
const normalized = normalizeStringArray(data.greenhouseCompanies);
return stringArraysEqual(
normalized,
jobSources.greenhouseCompanies.default,
)
? null
: normalized;
})(),
leverCompanies: (() => {
const normalized = normalizeStringArray(data.leverCompanies);
return stringArraysEqual(
normalized,
jobSources.leverCompanies.default,
)
? null
: normalized;
})(),
smartrecruitersCompanies: (() => {
const normalized = normalizeStringArray(
data.smartrecruitersCompanies,
);
return stringArraysEqual(
normalized,
jobSources.smartrecruitersCompanies.default,
)
? null
: normalized;
})(),
teamtailorCompanies: (() => {
const normalized = normalizeStringArray(data.teamtailorCompanies);
return stringArraysEqual(
normalized,
jobSources.teamtailorCompanies.default,
)
? null
: normalized;
})(),
huntflowTenants: (() => {
const normalized = normalizeStringArray(data.huntflowTenants);
return stringArraysEqual(
normalized,
jobSources.huntflowTenants.default,
)
? null
: normalized;
})(),
factorialTenants: (() => {
const normalized = normalizeStringArray(data.factorialTenants);
return stringArraysEqual(
normalized,
jobSources.factorialTenants.default,
)
? null
: normalized;
})(),
workdayTenants: (() => {
const normalized = normalizeStringArray(data.workdayTenants);
return stringArraysEqual(
normalized,
jobSources.workdayTenants.default,
)
? null
: normalized;
})(),
careersPageUrls: (() => {
const normalized = normalizeStringArray(data.careersPageUrls);
return stringArraysEqual(
normalized,
jobSources.careersPageUrls.default,
)
? null
: normalized;
})(),
icimsTenants: (() => {
const normalized = normalizeStringArray(data.icimsTenants);
return stringArraysEqual(normalized, jobSources.icimsTenants.default)
? null
: normalized;
})(),
elutaRssLocations: (() => {
const normalized = normalizeStringArray(data.elutaRssLocations);
return stringArraysEqual(
normalized,
jobSources.elutaRssLocations.default,
)
? null
: normalized;
})(),
...envPayload,
};
const shouldValidateRxResumeBeforeSave = Boolean(
dirtyFields.rxresumeMode ||
dirtyFields.rxresumeUrl ||
dirtyFields.rxresumeApiKey ||
dirtyFields.rxresumeEmail ||
dirtyFields.rxresumePassword,
);
const rxResumeValidationMode = (data.rxresumeMode ??
rxresumeMode) as RxResumeMode;
let rxResumeSaveWarningMessage: string | null = null;
if (shouldValidateRxResumeBeforeSave) {
const validationDraft = getRxResumeCredentialDrafts(data);
const precheckFailure = getRxResumeCredentialPrecheckFailure({
mode: rxResumeValidationMode,
stored: storedRxResume,
draft: validationDraft,
});
if (!precheckFailure) {
const preserveBlankFields = [
...(dirtyFields.rxresumeEmail ? (["email"] as const) : []),
...(dirtyFields.rxresumePassword ? (["password"] as const) : []),
...(dirtyFields.rxresumeApiKey ? (["apiKey"] as const) : []),
...(dirtyFields.rxresumeUrl ? (["baseUrl"] as const) : []),
];
const validation = await api.validateRxresume({
mode: rxResumeValidationMode,
...toRxResumeValidationPayload(validationDraft, {
preserveBlankFields: preserveBlankFields as Array<
keyof ReturnType<typeof getRxResumeCredentialDrafts>
>,
}),
});
setRxResumeValidationStatus(rxResumeValidationMode, validation);
if (isRxResumeBlockingValidationFailure(validation)) {
clearErrors(
getRxResumeValidationFieldsForMode(rxResumeValidationMode),
);
if (rxResumeValidationMode === "v5") {
setError("rxresumeApiKey", {
type: "manual",
message:
validation.message ??
"Reactive Resume v5 API key is invalid.",
});
} else {
setError("rxresumeEmail", {
type: "manual",
message:
validation.message ??
"Reactive Resume v4 email/password is invalid.",
});
setError("rxresumePassword", {
type: "manual",
message:
validation.message ??
"Reactive Resume v4 email/password is invalid.",
});
}
return;
}
clearErrors(
getRxResumeValidationFieldsForMode(rxResumeValidationMode),
);
if (isRxResumeAvailabilityValidationFailure(validation)) {
rxResumeSaveWarningMessage =
"Settings saved, but JobOps could not verify Reactive Resume because the instance is unavailable.";
}
}
}
const updated = await updateSettingsMutation.mutateAsync(payload);
if (
@@ -702,6 +1048,9 @@ export const SettingsPage: React.FC = () => {
setSettings(updated);
reset(mapSettingsToForm(updated));
toast.success("Settings saved");
if (rxResumeSaveWarningMessage) {
toast.info(rxResumeSaveWarningMessage);
}
} catch (error) {
const message =
error instanceof Error ? error.message : "Failed to save settings";
@@ -788,57 +1137,6 @@ export const SettingsPage: React.FC = () => {
}
};
const handleClearDiscoveredMatchingArchived = async () => {
try {
setIsSaving(true);
const result = await api.deleteDiscoveredMatchingArchived();
if (result.count > 0) {
toast.success("Rediscovered matches cleared", {
description: `Deleted ${result.count} discovered job${result.count === 1 ? "" : "s"} matching skipped or applied roles.`,
});
} else {
toast.info("No matches found", {
description:
"No discovered jobs matched skipped or applied archive roles.",
});
}
} catch (error) {
const message =
error instanceof Error
? error.message
: "Failed to clear rediscovered archive matches";
toast.error(message);
} finally {
setIsSaving(false);
}
};
const handleExpireDeadListings = async () => {
try {
setIsSaving(true);
const result = await api.expireDeadListings();
if (result.count > 0) {
toast.success("Expired listings marked", {
description: `Marked ${result.count} discovered job${result.count === 1 ? "" : "s"} as expired.`,
});
} else {
toast.info("No expired listings found", {
description: "All probed discovered job links still look live.",
});
}
} catch (error) {
const message =
error instanceof Error
? error.message
: "Failed to mark expired listings";
toast.error(message);
} finally {
setIsSaving(false);
}
};
const toggleStatusToClear = (status: JobStatus) => {
setStatusesToClear((prev) =>
prev.includes(status)
@@ -879,8 +1177,52 @@ export const SettingsPage: React.FC = () => {
isLoading={isLoading}
isSaving={isSaving}
/>
<ScoringSettingsSection
values={scoring}
<WebhooksSection
pipelineWebhook={pipelineWebhook}
jobCompleteWebhook={jobCompleteWebhook}
webhookSecretHint={envSettings.private.webhookSecretHint}
isLoading={isLoading}
isSaving={isSaving}
/>
<ReactiveResumeSection
rxResumeBaseResumeIdDraft={rxResumeBaseResumeIdDraft}
onRxresumeModeChange={(mode) => {
const nextId = getBaseResumeIdForMode(mode);
setRxResumeBaseResumeIdDraft(nextId);
setValue("rxresumeBaseResumeId", nextId, { shouldDirty: true });
setRxResumeProjectsOverride(null);
}}
setRxResumeBaseResumeIdDraft={(value) => {
const mode = (getValues("rxresumeMode") ??
rxresumeMode) as RxResumeMode;
setBaseResumeIdForMode(mode, value);
setRxResumeBaseResumeIdDraft(value);
setValue("rxresumeBaseResumeId", value, { shouldDirty: true });
}}
hasRxResumeAccess={hasRxResumeAccess}
rxresumeMode={rxresumeMode}
onCredentialFieldEdit={clearRxResumeValidationFeedback}
validationStatuses={rxresumeValidationStatuses}
profileProjects={effectiveProfileProjects}
lockedCount={lockedCount}
maxProjectsTotal={effectiveMaxProjectsTotal}
isProjectsLoading={isFetchingRxResumeProjects}
isLoading={isLoading}
isSaving={isSaving}
/>
<TracerLinksSettingsSection
readiness={tracerReadiness}
isLoading={isLoading || isTracerReadinessLoading}
isChecking={isTracerReadinessChecking}
onVerifyNow={handleVerifyTracerReadiness}
/>
<DisplaySettingsSection
values={display}
isLoading={isLoading}
isSaving={isSaving}
/>
<ChatSettingsSection
values={chat}
isLoading={isLoading}
isSaving={isSaving}
/>
@@ -890,19 +1232,8 @@ export const SettingsPage: React.FC = () => {
isLoading={isLoading}
isSaving={isSaving}
/>
<KeywordSetsSettingsSection
isLoading={isLoading}
isSaving={isSaving}
/>
<JobSourcesSettingsSection
values={jobSources}
isLoading={isLoading}
isSaving={isSaving}
/>
<WebhooksSection
pipelineWebhook={pipelineWebhook}
jobCompleteWebhook={jobCompleteWebhook}
webhookSecretHint={envSettings.private.webhookSecretHint}
<ScoringSettingsSection
values={scoring}
isLoading={isLoading}
isSaving={isSaving}
/>
@@ -911,16 +1242,23 @@ export const SettingsPage: React.FC = () => {
isLoading={isLoading}
isSaving={isSaving}
/>
<BackupSettingsSection
values={backup}
backups={backups}
nextScheduled={nextScheduled}
isLoading={isLoading || isLoadingBackups}
isSaving={isSaving}
onCreateBackup={handleCreateBackup}
onDeleteBackup={handleDeleteBackup}
isCreatingBackup={isCreatingBackup}
isDeletingBackup={isDeletingBackup}
/>
<DangerZoneSection
statusesToClear={statusesToClear}
toggleStatusToClear={toggleStatusToClear}
handleClearByStatuses={handleClearByStatuses}
handleClearDatabase={handleClearDatabase}
handleClearByScore={handleClearByScore}
handleClearDiscoveredMatchingArchived={
handleClearDiscoveredMatchingArchived
}
handleExpireDeadListings={handleExpireDeadListings}
isLoading={isLoading}
isSaving={isSaving}
/>
@@ -114,7 +114,6 @@ beforeEach(() => {
status: "applied",
suitabilityScore: null,
sponsorMatchScore: null,
sponsorshipSignals: null,
jobType: null,
jobFunction: null,
salaryMinAmount: null,
@@ -138,7 +137,6 @@ beforeEach(() => {
status: "applied",
suitabilityScore: null,
sponsorMatchScore: null,
sponsorshipSignals: null,
jobType: null,
jobFunction: null,
salaryMinAmount: null,
@@ -25,10 +25,6 @@ vi.mock("@/lib/user-location", () => ({
getDetectedCountryKey: getDetectedCountryKeyMock,
}));
vi.mock("./KeywordSetPicker", () => ({
KeywordSetPicker: () => null,
}));
describe("AutomaticRunTab", () => {
beforeEach(() => {
getDetectedCountryKeyMock.mockReset();
@@ -5,7 +5,7 @@ import {
normalizeCountryKey,
SUPPORTED_COUNTRY_KEYS,
} from "@shared/location-support.js";
import type { AppSettings, JobSource, KeywordSet } from "@shared/types";
import type { AppSettings, JobSource } from "@shared/types";
import { Loader2, Sparkles } from "lucide-react";
import { useCallback, useEffect, useMemo, useState } from "react";
import { useForm } from "react-hook-form";
@@ -45,7 +45,6 @@ import {
WORKPLACE_TYPE_OPTIONS,
type WorkplaceType,
} from "./automatic-run";
import { KeywordSetPicker } from "./KeywordSetPicker";
import { TokenizedInput } from "./TokenizedInput";
interface AutomaticRunTabProps {
@@ -63,8 +62,7 @@ const DEFAULT_VALUES: AutomaticRunValues = {
topN: 10,
minSuitabilityScore: 50,
searchTerms: ["web developer"],
activeKeywordSetId: null,
runBudget: 500,
runBudget: 200,
country: "united kingdom",
cityLocations: [],
workplaceTypes: ["remote", "hybrid", "onsite"],
@@ -165,9 +163,6 @@ export const AutomaticRunTab: React.FC<AutomaticRunTabProps> = ({
}) => {
const [isSaving, setIsSaving] = useState(false);
const [advancedOpen, setAdvancedOpen] = useState(false);
const [activeKeywordSetId, setActiveKeywordSetId] = useState<string | null>(
null,
);
const { watch, reset, setValue } = useForm<AutomaticRunFormValues>({
defaultValues: {
topN: String(DEFAULT_VALUES.topN),
@@ -192,25 +187,6 @@ export const AutomaticRunTab: React.FC<AutomaticRunTabProps> = ({
const searchTerms = watch("searchTerms");
const searchTermDraft = watch("searchTermDraft");
const handleActiveKeywordSetChange = useCallback(
(set: KeywordSet) => {
setActiveKeywordSetId(set.id);
const terms = set.terms.map((t) => t.trim()).filter(Boolean);
if (terms.length > 0) {
setValue("searchTerms", terms, { shouldDirty: false });
return;
}
const profileRoles =
settings?.jobSearchProfile?.value?.targetRoles
?.map((t) => t.trim())
.filter((t): t is string => Boolean(t)) ?? [];
if (profileRoles.length > 0) {
setValue("searchTerms", profileRoles, { shouldDirty: false });
}
},
[setValue, settings?.jobSearchProfile?.value?.targetRoles],
);
useEffect(() => {
if (!open) return;
const memory = loadAutomaticRunMemory();
@@ -298,7 +274,6 @@ export const AutomaticRunTab: React.FC<AutomaticRunTabProps> = ({
cityLocations,
workplaceTypes: normalizeWorkplaceTypes(workplaceTypes),
searchTerms,
activeKeywordSetId,
};
}, [
topNInput,
@@ -308,7 +283,6 @@ export const AutomaticRunTab: React.FC<AutomaticRunTabProps> = ({
cityLocations,
workplaceTypes,
searchTerms,
activeKeywordSetId,
]);
const workplaceTypeSelectionInvalid = workplaceTypes.length === 0;
@@ -346,20 +320,13 @@ export const AutomaticRunTab: React.FC<AutomaticRunTabProps> = ({
const filtered = pipelineSources.filter((source) =>
isSourceAvailableForRun(source),
);
const missing = compatibleEnabledSources.filter(
(source) => !filtered.includes(source),
);
if (missing.length > 0) {
onSetPipelineSources([...filtered, ...missing]);
return;
}
if (filtered.length === pipelineSources.length) return;
if (filtered.length > 0) {
onSetPipelineSources(filtered);
return;
}
if (compatibleEnabledSources.length > 0) {
onSetPipelineSources([...compatibleEnabledSources]);
onSetPipelineSources([compatibleEnabledSources[0]]);
}
}, [
compatibleEnabledSources,
@@ -368,19 +335,14 @@ export const AutomaticRunTab: React.FC<AutomaticRunTabProps> = ({
pipelineSources,
]);
const mergedDiscoveryTerms = useMemo(() => {
if (values.activeKeywordSetId) {
return values.searchTerms.map((t) => t.trim()).filter(Boolean);
}
return mergeDiscoverySearchTerms(
values.searchTerms,
settings?.jobSearchProfile?.value?.targetRoles,
);
}, [
values.searchTerms,
values.activeKeywordSetId,
settings?.jobSearchProfile?.value?.targetRoles,
]);
const mergedDiscoveryTerms = useMemo(
() =>
mergeDiscoverySearchTerms(
values.searchTerms,
settings?.jobSearchProfile?.value?.targetRoles,
),
[values.searchTerms, settings?.jobSearchProfile?.value?.targetRoles],
);
const estimate = useMemo(
() =>
@@ -638,12 +600,7 @@ export const AutomaticRunTab: React.FC<AutomaticRunTabProps> = ({
<CardHeader className="pb-3">
<CardTitle>Search terms</CardTitle>
</CardHeader>
<CardContent className="space-y-4">
<KeywordSetPicker
key={settings?.activeProfileId ?? "default"}
disabled={isSaving || isPipelineRunning}
onActiveSetChange={handleActiveKeywordSetChange}
/>
<CardContent>
<TokenizedInput
id="search-terms-input"
values={searchTerms}
@@ -662,37 +619,10 @@ export const AutomaticRunTab: React.FC<AutomaticRunTabProps> = ({
<Card>
<CardHeader className="pb-3">
<div className="flex flex-wrap items-center justify-between gap-2">
<CardTitle>
Sources ({compatiblePipelineSources.length}/
{compatibleEnabledSources.length})
</CardTitle>
<div className="flex gap-2">
<Button
type="button"
size="sm"
variant="outline"
disabled={isSaving || isPipelineRunning}
onClick={() =>
onSetPipelineSources([...compatibleEnabledSources])
}
>
Select all
</Button>
<Button
type="button"
size="sm"
variant="outline"
disabled={isSaving || isPipelineRunning}
onClick={() => {
if (compatibleEnabledSources.length === 0) return;
onSetPipelineSources([compatibleEnabledSources[0]]);
}}
>
Clear
</Button>
</div>
</div>
<CardTitle>
Sources ({compatiblePipelineSources.length}/
{compatibleEnabledSources.length})
</CardTitle>
</CardHeader>
<CardContent className="flex flex-wrap gap-2">
<TooltipProvider>
@@ -37,7 +37,7 @@ describe("JobCommandBar", () => {
expect(
screen.getByPlaceholderText(
"Search jobs by title, company, location, or sponsorship signals...",
"Search jobs by job title or company name...",
),
).toBeInTheDocument();
});
@@ -52,7 +52,7 @@ describe("JobCommandBar", () => {
openWithKeyboard();
const input = screen.getByPlaceholderText(
"Search jobs by title, company, location, or sponsorship signals...",
"Search jobs by job title or company name...",
);
fireEvent.change(input, { target: { value: "@disc" } });
fireEvent.keyDown(input, { key: "Tab" });
@@ -70,7 +70,7 @@ describe("JobCommandBar", () => {
openWithKeyboard();
const input = screen.getByPlaceholderText(
"Search jobs by title, company, location, or sponsorship signals...",
"Search jobs by job title or company name...",
);
fireEvent.change(input, { target: { value: "@ready" } });
fireEvent.keyDown(input, { key: "Enter" });
@@ -91,7 +91,7 @@ describe("JobCommandBar", () => {
expect(dialog.className).not.toContain("border-sky-500/50");
const input = screen.getByPlaceholderText(
"Search jobs by title, company, location, or sponsorship signals...",
"Search jobs by job title or company name...",
);
fireEvent.change(input, { target: { value: "@disc" } });
fireEvent.keyDown(input, { key: "Tab" });
@@ -118,7 +118,7 @@ describe("JobCommandBar", () => {
openWithKeyboard();
const input = screen.getByPlaceholderText(
"Search jobs by title, company, location, or sponsorship signals...",
"Search jobs by job title or company name...",
);
fireEvent.change(input, { target: { value: "@ready" } });
@@ -141,7 +141,7 @@ describe("JobCommandBar", () => {
openWithKeyboard();
const input = screen.getByPlaceholderText(
"Search jobs by title, company, location, or sponsorship signals...",
"Search jobs by job title or company name...",
);
fireEvent.change(input, { target: { value: "@" } });
@@ -163,7 +163,7 @@ describe("JobCommandBar", () => {
openWithKeyboard();
const input = screen.getByPlaceholderText(
"Search jobs by title, company, location, or sponsorship signals...",
"Search jobs by job title or company name...",
);
fireEvent.change(input, { target: { value: "@prog" } });
fireEvent.keyDown(input, { key: "Tab" });
@@ -192,7 +192,7 @@ describe("JobCommandBar", () => {
openWithKeyboard();
fireEvent.change(
screen.getByPlaceholderText(
"Search jobs by title, company, location, or sponsorship signals...",
"Search jobs by job title or company name...",
),
{
target: { value: "Globex" },
@@ -220,7 +220,7 @@ describe("JobCommandBar", () => {
openWithKeyboard();
const input = screen.getByPlaceholderText(
"Search jobs by title, company, location, or sponsorship signals...",
"Search jobs by job title or company name...",
);
fireEvent.change(input, { target: { value: "@disc" } });
fireEvent.keyDown(input, { key: "Tab" });
@@ -257,7 +257,7 @@ describe("JobCommandBar", () => {
openWithKeyboard();
const input = screen.getByPlaceholderText(
"Search jobs by title, company, location, or sponsorship signals...",
"Search jobs by job title or company name...",
);
fireEvent.change(input, { target: { value: "@disc" } });
fireEvent.keyDown(input, { key: "Tab" });
@@ -280,7 +280,7 @@ describe("JobCommandBar", () => {
openWithKeyboard();
const input = screen.getByPlaceholderText(
"Search jobs by title, company, location, or sponsorship signals...",
"Search jobs by job title or company name...",
);
fireEvent.change(input, { target: { value: "@ready" } });
fireEvent.keyDown(input, { key: "Tab" });
@@ -302,7 +302,7 @@ describe("JobCommandBar", () => {
openWithKeyboard();
const input = screen.getByPlaceholderText(
"Search jobs by title, company, location, or sponsorship signals...",
"Search jobs by job title or company name...",
);
fireEvent.change(input, { target: { value: "@ready" } });
fireEvent.keyDown(input, { key: "Tab" });
@@ -323,7 +323,7 @@ describe("JobCommandBar", () => {
openWithKeyboard();
const input = screen.getByPlaceholderText(
"Search jobs by title, company, location, or sponsorship signals...",
"Search jobs by job title or company name...",
);
fireEvent.change(input, { target: { value: "@ready" } });
fireEvent.keyDown(input, { key: "Tab" });
@@ -342,7 +342,7 @@ describe("JobCommandBar", () => {
openWithKeyboard();
const input = screen.getByPlaceholderText(
"Search jobs by title, company, location, or sponsorship signals...",
"Search jobs by job title or company name...",
);
fireEvent.change(input, { target: { value: "@ready" } });
fireEvent.keyDown(input, { key: "Tab" });
@@ -353,7 +353,7 @@ describe("JobCommandBar", () => {
expect(screen.queryByText("@ready")).not.toBeInTheDocument();
expect(
screen.getByPlaceholderText(
"Search jobs by title, company, location, or sponsorship signals...",
"Search jobs by job title or company name...",
),
).toBeInTheDocument();
});
@@ -368,7 +368,7 @@ describe("JobCommandBar", () => {
openWithKeyboard();
const input = screen.getByPlaceholderText(
"Search jobs by title, company, location, or sponsorship signals...",
"Search jobs by job title or company name...",
);
fireEvent.change(input, { target: { value: "@ready" } });
fireEvent.keyDown(input, { key: "Tab" });
@@ -390,7 +390,7 @@ describe("JobCommandBar", () => {
openWithKeyboard();
const input = screen.getByPlaceholderText(
"Search jobs by title, company, location, or sponsorship signals...",
"Search jobs by job title or company name...",
);
fireEvent.change(input, { target: { value: "@all" } });
fireEvent.keyDown(input, { key: "Tab" });
@@ -423,7 +423,7 @@ describe("JobCommandBar", () => {
openWithKeyboard();
fireEvent.change(
screen.getByPlaceholderText(
"Search jobs by title, company, location, or sponsorship signals...",
"Search jobs by job title or company name...",
),
{
target: { value: "Globex" },
@@ -472,7 +472,7 @@ describe("JobCommandBar", () => {
openWithKeyboard();
const input = screen.getByPlaceholderText(
"Search jobs by title, company, location, or sponsorship signals...",
"Search jobs by job title or company name...",
);
const lockTokens = ["@ready", "@disc", "@applied", "@skip", "@exp"];
@@ -169,11 +169,10 @@ export const JobCommandBar: React.FC<JobCommandBarProps> = ({
>
<DialogTitle className="sr-only">Job Search</DialogTitle>
<DialogDescription className="sr-only">
Search jobs across all states by title, company, location, or
sponsorship signals such as TN or visa sponsorship.
Search jobs across all states by job title or company name.
</DialogDescription>
<CommandInput
placeholder="Search jobs by title, company, location, or sponsorship signals..."
placeholder="Search jobs by job title or company name..."
value={query}
onValueChange={setQuery}
onKeyDown={handleInputKeyDown}
@@ -1,4 +1,3 @@
import { sponsorshipSignalSearchHaystack } from "@shared/sponsorship-signals";
import type { JobListItem, JobStatus } from "@shared/types.js";
import type { FilterTab } from "./constants";
@@ -145,23 +144,12 @@ export const computeJobMatchScore = (
job.location ?? "",
normalizedQuery,
);
const sponsorshipScore = computeFieldMatchScore(
sponsorshipSignalSearchHaystack(job.sponsorshipSignals),
normalizedQuery,
);
// Prefer title/company matches over location when scores tie.
// Only apply bias when a field actually matched.
const titleRankedScore = titleScore > 0 ? titleScore + 8 : 0;
const employerRankedScore = employerScore > 0 ? employerScore + 12 : 0;
const sponsorshipRankedScore =
sponsorshipScore > 0 ? sponsorshipScore + 4 : 0;
return Math.max(
titleRankedScore,
employerRankedScore,
locationScore,
sponsorshipRankedScore,
);
return Math.max(titleRankedScore, employerRankedScore, locationScore);
};
export const groupJobsForCommandBar = (
@@ -1,4 +1,3 @@
import { SponsorshipSignalsPills } from "@client/components/SponsorshipSignalsPills";
import type { JobListItem } from "@shared/types.js";
import { cn } from "@/lib/utils";
import { defaultStatusToken, statusTokens } from "./constants";
@@ -78,11 +77,6 @@ export const JobRowContent = ({
Found {formatDiscoveredAt(job.discoveredAt)}
</div>
)}
<SponsorshipSignalsPills
sponsorshipSignals={job.sponsorshipSignals}
size="xs"
className="mt-1"
/>
</div>
{hasScore && (
@@ -1,253 +0,0 @@
import * as api from "@client/api";
import type { KeywordSet } from "@shared/types";
import { Loader2, Plus, Trash2 } from "lucide-react";
import type React from "react";
import { useCallback, useEffect, useRef, useState } from "react";
import { toast } from "sonner";
import { Button } from "@/components/ui/button";
import { Input } from "@/components/ui/input";
import {
Select,
SelectContent,
SelectItem,
SelectTrigger,
SelectValue,
} from "@/components/ui/select";
import { cn } from "@/lib/utils";
type KeywordSetPickerProps = {
disabled?: boolean;
onActiveSetChange: (set: KeywordSet) => void;
};
export const KeywordSetPicker: React.FC<KeywordSetPickerProps> = ({
disabled = false,
onActiveSetChange,
}) => {
const [sets, setSets] = useState<KeywordSet[]>([]);
const [loading, setLoading] = useState(false);
const [creating, setCreating] = useState(false);
const [newSetName, setNewSetName] = useState("");
const [showCreate, setShowCreate] = useState(false);
const onActiveSetChangeRef = useRef(onActiveSetChange);
const lastNotifiedIdRef = useRef<string | null>(null);
onActiveSetChangeRef.current = onActiveSetChange;
const notifyActiveSet = useCallback((set: KeywordSet) => {
if (lastNotifiedIdRef.current === set.id) return;
lastNotifiedIdRef.current = set.id;
onActiveSetChangeRef.current(set);
}, []);
const loadSets = useCallback(async () => {
setLoading(true);
try {
const list = await api.listKeywordSets();
setSets(list);
const active = list.find((s) => s.isActive) ?? list[0];
if (active) notifyActiveSet(active);
} catch {
toast.error("Failed to load keyword sets");
} finally {
setLoading(false);
}
}, [notifyActiveSet]);
useEffect(() => {
void loadSets();
}, [loadSets]);
const handleSelect = useCallback(
async (setId: string) => {
if (disabled || loading) return;
const current = sets.find((s) => s.id === setId);
if (current?.isActive) return;
setLoading(true);
try {
const activated = await api.activateKeywordSet(setId);
setSets((prev) =>
prev.map((s) => ({ ...s, isActive: s.id === activated.id })),
);
notifyActiveSet(activated);
} catch {
toast.error("Failed to switch keyword set");
} finally {
setLoading(false);
}
},
[disabled, loading, notifyActiveSet, sets],
);
const handleCreate = useCallback(async () => {
const name = newSetName.trim();
if (!name) {
toast.error("Enter a name for the keyword set");
return;
}
setCreating(true);
try {
const created = await api.createKeywordSet({ name, terms: [] });
const activated = await api.activateKeywordSet(created.id);
setSets((prev) => [
...prev.map((s) => ({ ...s, isActive: false })),
{ ...activated, isActive: true },
]);
notifyActiveSet(activated);
setNewSetName("");
setShowCreate(false);
toast.success(`Keyword set "${activated.name}" created`);
} catch (error) {
const message =
error instanceof Error ? error.message : "Failed to create keyword set";
toast.error(message);
} finally {
setCreating(false);
}
}, [newSetName, notifyActiveSet]);
const handleDelete = useCallback(
async (setId: string) => {
if (disabled || loading) return;
setLoading(true);
try {
await api.deleteKeywordSet(setId);
lastNotifiedIdRef.current = null;
const list = await api.listKeywordSets();
setSets(list);
const active = list.find((s) => s.isActive) ?? list[0];
if (active) notifyActiveSet(active);
toast.success("Keyword set deleted");
} catch (error) {
const message =
error instanceof Error
? error.message
: "Failed to delete keyword set";
toast.error(message);
} finally {
setLoading(false);
}
},
[disabled, loading, notifyActiveSet],
);
const activeSet = sets.find((s) => s.isActive) ?? sets[0];
return (
<div className="space-y-3">
<div className="flex flex-wrap items-center gap-2">
<span className="text-sm font-medium shrink-0">Active set</span>
{activeSet ? (
<Select
value={activeSet.id}
onValueChange={handleSelect}
disabled={disabled || loading}
>
<SelectTrigger
className="w-[min(100%,240px)]"
aria-label="Keyword set"
>
<SelectValue placeholder="Keyword set" />
</SelectTrigger>
<SelectContent>
{sets.map((set) => (
<SelectItem key={set.id} value={set.id}>
{set.name}
{set.isActive ? " (active)" : ""}
</SelectItem>
))}
</SelectContent>
</Select>
) : (
<span className="text-sm text-muted-foreground">
{loading ? "Loading sets…" : "No keyword sets"}
</span>
)}
{loading ? (
<Loader2 className="h-4 w-4 animate-spin text-muted-foreground" />
) : null}
<Button
type="button"
variant="outline"
size="sm"
disabled={disabled || creating}
onClick={() => setShowCreate((v) => !v)}
>
<Plus className="h-4 w-4 mr-1" />
New set
</Button>
{activeSet && sets.length > 1 ? (
<Button
type="button"
variant="ghost"
size="sm"
disabled={disabled || loading}
onClick={() => void handleDelete(activeSet.id)}
title={`Delete "${activeSet.name}"`}
>
<Trash2 className="h-4 w-4" />
</Button>
) : null}
</div>
{showCreate ? (
<div className="flex flex-wrap items-center gap-2">
<Input
value={newSetName}
onChange={(e) => setNewSetName(e.target.value)}
placeholder="e.g. SDET, Caseware, Lead"
className="max-w-xs"
disabled={disabled || creating}
onKeyDown={(e) => {
if (e.key === "Enter") {
e.preventDefault();
void handleCreate();
}
}}
/>
<Button
type="button"
size="sm"
disabled={disabled || creating}
onClick={() => void handleCreate()}
>
{creating ? <Loader2 className="h-4 w-4 animate-spin" /> : "Create"}
</Button>
</div>
) : null}
{sets.length > 0 ? (
<div className="flex flex-wrap gap-1.5 pt-1">
{sets.map((set) => {
const isActive = set.id === activeSet?.id;
return (
<button
key={set.id}
type="button"
disabled={disabled || loading || isActive}
onClick={() => void handleSelect(set.id)}
className={cn(
"rounded-md border px-2.5 py-1 text-xs font-medium transition-colors",
isActive
? "border-primary bg-primary/10 text-foreground"
: "border-border bg-muted/40 text-muted-foreground hover:bg-muted",
)}
>
{set.name}
<span className="ml-1 text-muted-foreground">
({set.terms.length})
</span>
</button>
);
})}
</div>
) : !loading ? (
<p className="text-xs text-amber-600 dark:text-amber-400">
No keyword sets loaded. Use &quot;New set&quot; or check that the API
is running.
</p>
) : null}
<p className="text-xs text-muted-foreground">
Switch between named lists for different searches. Terms below belong to
the active set.
</p>
</div>
);
};
@@ -53,15 +53,9 @@ const renderFilters = (overrides?: Partial<FiltersProps>) => {
onEmployerIncludeChange: vi.fn(),
onEmployerExcludeChange: vi.fn(),
applySettingsCompanySkipList: true,
hidePriorSkips: true,
onHidePriorSkipsChange: vi.fn(),
onApplySettingsCompanySkipListChange: vi.fn(),
sponsorFilter: "all" as SponsorFilter,
onSponsorFilterChange: vi.fn(),
sponsorshipSignalsFilter: [],
onSponsorshipSignalsFilterChange: vi.fn(),
hideSponsorBlockers: false,
onHideSponsorBlockersChange: vi.fn(),
workplaceFilter: "all" as WorkplaceFilter,
onWorkplaceFilterChange: vi.fn(),
salaryFilter: {
@@ -1,13 +1,6 @@
import { KbdHint } from "@client/components/KbdHint";
import { getDisplayKey, SHORTCUTS } from "@client/lib/shortcut-map";
import { formatCountryLabel } from "@shared/location-support";
import {
SPONSORSHIP_GREEN_FLAG_IDS,
SPONSORSHIP_RED_FLAG_IDS,
SPONSORSHIP_SIGNAL_META,
SPONSORSHIP_YELLOW_FLAG_IDS,
type SponsorshipSignalId,
} from "@shared/sponsorship-signals";
import type { JobSource } from "@shared/types.js";
import { Filter, Search } from "lucide-react";
import type React from "react";
@@ -69,14 +62,8 @@ interface OrchestratorFiltersProps {
onEmployerExcludeChange: (values: string[]) => void;
applySettingsCompanySkipList: boolean;
onApplySettingsCompanySkipListChange: (value: boolean) => void;
hidePriorSkips: boolean;
onHidePriorSkipsChange: (value: boolean) => void;
sponsorFilter: SponsorFilter;
onSponsorFilterChange: (value: SponsorFilter) => void;
sponsorshipSignalsFilter: SponsorshipSignalId[];
onSponsorshipSignalsFilterChange: (values: SponsorshipSignalId[]) => void;
hideSponsorBlockers: boolean;
onHideSponsorBlockersChange: (value: boolean) => void;
workplaceFilter: WorkplaceFilter;
onWorkplaceFilterChange: (value: WorkplaceFilter) => void;
salaryFilter: SalaryFilter;
@@ -185,14 +172,8 @@ export const OrchestratorFilters: React.FC<OrchestratorFiltersProps> = ({
onEmployerExcludeChange,
applySettingsCompanySkipList,
onApplySettingsCompanySkipListChange,
hidePriorSkips,
onHidePriorSkipsChange,
sponsorFilter,
onSponsorFilterChange,
sponsorshipSignalsFilter,
onSponsorshipSignalsFilterChange,
hideSponsorBlockers,
onHideSponsorBlockersChange,
workplaceFilter,
onWorkplaceFilterChange,
salaryFilter,
@@ -257,10 +238,7 @@ export const OrchestratorFilters: React.FC<OrchestratorFiltersProps> = ({
employerIncludeFilter.length > 0 || employerExcludeFilter.length > 0,
) +
Number(!applySettingsCompanySkipList) +
Number(!hidePriorSkips) +
Number(sponsorFilter !== "all") +
Number(sponsorshipSignalsFilter.length > 0) +
Number(hideSponsorBlockers) +
Number(workplaceFilter !== "all") +
Number(
(typeof salaryFilter.min === "number" && salaryFilter.min > 0) ||
@@ -276,10 +254,7 @@ export const OrchestratorFilters: React.FC<OrchestratorFiltersProps> = ({
employerIncludeFilter.length,
employerExcludeFilter.length,
applySettingsCompanySkipList,
hidePriorSkips,
sponsorFilter,
sponsorshipSignalsFilter.length,
hideSponsorBlockers,
workplaceFilter,
salaryFilter.min,
salaryFilter.max,
@@ -548,29 +523,6 @@ export const OrchestratorFilters: React.FC<OrchestratorFiltersProps> = ({
</div>
</div>
<div className="flex items-start gap-3 rounded-md border border-border/60 bg-muted/20 px-3 py-3">
<Checkbox
id="hide-prior-skips"
checked={hidePriorSkips}
onCheckedChange={(checked) => {
onHidePriorSkipsChange(checked === true);
}}
/>
<div className="space-y-1">
<label
htmlFor="hide-prior-skips"
className="text-sm font-medium leading-none cursor-pointer"
>
Hide roles you already skipped
</label>
<p className="text-xs text-muted-foreground">
Hides Discovered and Ready rows that match a skipped
or applied job with the same company and title from
the last 90 days.
</p>
</div>
</div>
<TokenizedInput
id="employer-include-filter"
values={employerIncludeFilter}
@@ -601,160 +553,22 @@ export const OrchestratorFilters: React.FC<OrchestratorFiltersProps> = ({
<CardHeader className="pb-3">
<CardTitle>Sponsor status</CardTitle>
</CardHeader>
<CardContent className="space-y-4">
<div className="flex flex-wrap gap-2">
{sponsorOptions.map((option) => (
<Button
key={option.value}
type="button"
size="sm"
variant={
sponsorFilter === option.value
? "default"
: "outline"
}
onClick={() => onSponsorFilterChange(option.value)}
>
{option.label}
</Button>
))}
</div>
<div className="space-y-3">
<div className="flex items-start gap-3 rounded-md border border-border/60 bg-muted/20 px-3 py-3">
<Checkbox
id="hide-sponsor-blockers"
checked={hideSponsorBlockers}
onCheckedChange={(checked) => {
onHideSponsorBlockersChange(checked === true);
}}
/>
<div className="space-y-1">
<label
htmlFor="hide-sponsor-blockers"
className="text-sm font-medium leading-none cursor-pointer"
>
Hide red-flag roles
</label>
<p className="text-xs text-muted-foreground">
Hides citizenship/GC/EAD-only, clearance, and
unrestricted-work-auth posts. Yellow no-sponsor
listings stay visible so you can ask about TN.
</p>
</div>
</div>
<div className="space-y-2">
<p className="text-xs font-medium text-emerald-200/90">
Green flags (TN-friendly)
</p>
<p className="text-xs text-muted-foreground">
Show jobs matching any selected signal. Search{" "}
<span className="font-mono">TN</span>,{" "}
<span className="font-mono">sponsor</span>, or{" "}
<span className="font-mono">
authorized to work
</span>{" "}
in the job search bar too.
</p>
<div className="flex flex-wrap gap-2">
{SPONSORSHIP_GREEN_FLAG_IDS.map((signalId) => {
const selected =
sponsorshipSignalsFilter.includes(signalId);
const meta = SPONSORSHIP_SIGNAL_META[signalId];
return (
<Button
key={signalId}
type="button"
size="sm"
variant={selected ? "default" : "outline"}
onClick={() => {
const next = selected
? sponsorshipSignalsFilter.filter(
(value) => value !== signalId,
)
: [...sponsorshipSignalsFilter, signalId];
onSponsorshipSignalsFilterChange(next);
}}
>
{meta.label}
</Button>
);
})}
</div>
</div>
<div className="space-y-2">
<p className="text-xs font-medium text-amber-200/90">
Yellow flags (verify TN)
</p>
<p className="text-xs text-muted-foreground">
Says no sponsorship ask if TN support letter is
available.
</p>
<div className="flex flex-wrap gap-2">
{SPONSORSHIP_YELLOW_FLAG_IDS.map((signalId) => {
const selected =
sponsorshipSignalsFilter.includes(signalId);
const meta = SPONSORSHIP_SIGNAL_META[signalId];
return (
<Button
key={signalId}
type="button"
size="sm"
variant={selected ? "secondary" : "outline"}
className={
selected
? "border-amber-500/40 bg-amber-500/20 text-amber-100"
: undefined
}
onClick={() => {
const next = selected
? sponsorshipSignalsFilter.filter(
(value) => value !== signalId,
)
: [...sponsorshipSignalsFilter, signalId];
onSponsorshipSignalsFilterChange(next);
}}
>
{meta.label}
</Button>
);
})}
</div>
</div>
<div className="space-y-2">
<p className="text-xs font-medium text-rose-200/90">
Red flags (hard skip)
</p>
<p className="text-xs text-muted-foreground">
Citizenship/GC/EAD only, clearance, or unrestricted
work authorization usually not TN-viable.
</p>
<div className="flex flex-wrap gap-2">
{SPONSORSHIP_RED_FLAG_IDS.map((signalId) => {
const selected =
sponsorshipSignalsFilter.includes(signalId);
const meta = SPONSORSHIP_SIGNAL_META[signalId];
return (
<Button
key={signalId}
type="button"
size="sm"
variant={selected ? "destructive" : "outline"}
onClick={() => {
const next = selected
? sponsorshipSignalsFilter.filter(
(value) => value !== signalId,
)
: [...sponsorshipSignalsFilter, signalId];
onSponsorshipSignalsFilterChange(next);
}}
>
{meta.label}
</Button>
);
})}
</div>
</div>
</div>
<CardContent className="flex flex-wrap gap-2">
{sponsorOptions.map((option) => (
<Button
key={option.value}
type="button"
size="sm"
variant={
sponsorFilter === option.value
? "default"
: "outline"
}
onClick={() => onSponsorFilterChange(option.value)}
>
{option.label}
</Button>
))}
</CardContent>
</Card>
@@ -5,7 +5,7 @@ import type { SearchProfile } from "@shared/types";
import { useMutation, useQuery, useQueryClient } from "@tanstack/react-query";
import { Loader2 } from "lucide-react";
import type React from "react";
import { useEffect, useMemo, useState } from "react";
import { useMemo } from "react";
import { toast } from "sonner";
import { Button } from "@/components/ui/button";
import {
@@ -27,38 +27,17 @@ export const ProfileQuickSwitch: React.FC = () => {
const { data: profiles = [], isLoading } = useQuery<SearchProfile[]>({
queryKey: queryKeys.searchProfiles.list(),
queryFn: api.listProfiles,
select: (rows) => {
const seen = new Set<string>();
return rows.filter((profile) => {
if (seen.has(profile.id)) return false;
seen.add(profile.id);
return true;
});
},
});
const [selectedId, setSelectedId] = useState<string | null>(null);
const activeId = useMemo(() => {
if (selectedId && profiles.some((profile) => profile.id === selectedId)) {
return selectedId;
}
const sid = settings?.activeProfileId?.trim() ?? "";
if (sid && profiles.some((profile) => profile.id === sid)) return sid;
if (sid && profiles.some((p) => p.id === sid)) return sid;
return profiles[0]?.id ?? "";
}, [selectedId, settings?.activeProfileId, profiles]);
useEffect(() => {
const sid = settings?.activeProfileId?.trim() ?? "";
if (sid && profiles.some((profile) => profile.id === sid)) {
setSelectedId(sid);
}
}, [settings?.activeProfileId, profiles]);
const activateMutation = useMutation({
mutationFn: (id: string) => api.activateProfile(id),
onSuccess: async (_data, profileId) => {
setSelectedId(profileId);
onSuccess: async () => {
await Promise.all([
queryClient.invalidateQueries({ queryKey: queryKeys.jobs.all }),
queryClient.invalidateQueries({ queryKey: queryKeys.profile.all }),
@@ -67,10 +46,7 @@ export const ProfileQuickSwitch: React.FC = () => {
}),
]);
await refreshSettings();
const profileName =
profiles.find((profile) => profile.id === profileId)?.name ??
"Search profile";
toast.success(`${profileName} active`);
toast.success("Search profile updated");
},
onError: (err: unknown) => {
const message =
@@ -1,7 +1,6 @@
import { describe, expect, it } from "vitest";
import {
AUTOMATIC_PRESETS,
buildMaxCoverageDiscoveryLimits,
calculateAutomaticEstimate,
deriveExtractorLimits,
mergeDiscoverySearchTerms,
@@ -27,7 +26,6 @@ describe("automatic-run utilities", () => {
topN: 10,
minSuitabilityScore: 50,
searchTerms: ["backend", "platform"],
activeKeywordSetId: null,
runBudget: 100,
country: "united kingdom",
cityLocations: [],
@@ -73,7 +71,6 @@ describe("automatic-run utilities", () => {
topN: 10,
minSuitabilityScore: 50,
searchTerms: [],
activeKeywordSetId: null,
runBudget: 750,
country: "united kingdom",
cityLocations: [],
@@ -115,7 +112,6 @@ describe("automatic-run utilities", () => {
topN: 10,
minSuitabilityScore: 50,
searchTerms: ["backend", "platform"],
activeKeywordSetId: null,
runBudget: 120,
country: "united kingdom",
cityLocations: [],
@@ -134,7 +130,6 @@ describe("automatic-run utilities", () => {
topN: 10,
minSuitabilityScore: 50,
searchTerms: ["backend", "platform"],
activeKeywordSetId: null,
runBudget: 120,
country: "united kingdom",
cityLocations: [],
@@ -153,7 +148,6 @@ describe("automatic-run utilities", () => {
topN: 10,
minSuitabilityScore: 50,
searchTerms: ["backend", "platform"],
activeKeywordSetId: null,
runBudget: 120,
country: "united kingdom",
cityLocations: [],
@@ -165,16 +159,4 @@ describe("automatic-run utilities", () => {
expect(estimate.discovered.cap).toBeGreaterThan(0);
expect(estimate.discovered.cap).toBeLessThanOrEqual(120);
});
it("raises discovery caps for max coverage runs", () => {
const limits = buildMaxCoverageDiscoveryLimits({
budget: 500,
searchTerms: ["qa engineer"],
sources: ["workingnomads", "testdevjobs", "indeed"],
});
expect(limits.workingnomadsMaxJobsPerTerm).toBeGreaterThanOrEqual(100);
expect(limits.testdevjobsMaxPages).toBe(10);
expect(limits.jobspyResultsWanted).toBeGreaterThanOrEqual(100);
});
});
@@ -2,7 +2,6 @@ import {
parseSearchCitiesSetting,
serializeSearchCitiesSetting,
} from "@shared/search-cities.js";
import type { UpdateSettingsInput } from "@shared/settings-schema.js";
import type { JobSource } from "@shared/types";
/**
@@ -40,7 +39,6 @@ export interface AutomaticRunValues {
topN: number;
minSuitabilityScore: number;
searchTerms: string[];
activeKeywordSetId: string | null;
runBudget: number;
country: string;
cityLocations: string[];
@@ -82,7 +80,7 @@ export const AUTOMATIC_PRESETS: Record<
detailed: {
topN: 20,
minSuitabilityScore: 35,
runBudget: 1000,
runBudget: 750,
},
};
@@ -165,51 +163,6 @@ export function deriveExtractorLimits(args: {
};
}
/** Raise per-source caps before a pipeline run for maximum discovery coverage. */
export function buildMaxCoverageDiscoveryLimits(args: {
budget: number;
searchTerms: string[];
sources: JobSource[];
}): Partial<UpdateSettingsInput> {
const limits = deriveExtractorLimits(args);
const perTermCap = Math.max(100, limits.jobspyResultsWanted);
const boardCap = Math.max(150, perTermCap);
return {
jobspyResultsWanted: perTermCap,
gradcrackerMaxJobsPerTerm: boardCap,
ukvisajobsMaxJobs: boardCap,
adzunaMaxJobsPerTerm: boardCap,
startupjobsMaxJobsPerTerm: boardCap,
usajobsMaxJobsPerTerm: boardCap,
jobicyMaxJobsPerTerm: boardCap,
themuseMaxJobsPerTerm: boardCap,
joobleMaxJobsPerTerm: boardCap,
careerjetMaxJobsPerTerm: boardCap,
reedMaxJobsPerTerm: boardCap,
remoteokMaxJobsPerTerm: boardCap,
remotiveMaxJobsPerTerm: boardCap,
arbeitnowMaxJobsPerTerm: boardCap,
himalayasMaxJobsPerTerm: boardCap,
weworkremotelyMaxJobsPerTerm: boardCap,
workingnomadsMaxJobsPerTerm: boardCap,
fourdayweekMaxJobsPerTerm: boardCap,
testdevjobsMaxJobsPerTerm: boardCap,
testdevjobsMaxPages: 10,
builtinMaxJobsPerTerm: boardCap,
builtinMaxPagesPerTerm: 10,
wellfoundMaxJobsPerTerm: Math.min(100, boardCap),
googleJobsMaxJobsPerTerm: Math.min(30, boardCap),
qajobsboardMaxJobsPerTerm: boardCap,
arcMaxJobsPerPath: boardCap,
smartrecruitersMaxJobsPerCompany: boardCap,
elutaMaxJobsPerTerm: boardCap,
bctenetMaxJobsPerTerm: 400,
icimsMaxJobsPerTenant: 250,
icimsMaxPagesPerSearch: 10,
};
}
export function parseSearchTermsInput(input: string): string[] {
return input
.split(/[\n,]/g)
@@ -3,26 +3,13 @@ import {
EXTRACTOR_SOURCE_METADATA,
PIPELINE_EXTRACTOR_SOURCE_IDS,
} from "@shared/extractors";
import {
SPONSORSHIP_SIGNAL_IDS,
type SponsorshipSignalId,
} from "@shared/sponsorship-signals";
import type { JobSource, JobStatus } from "@shared/types";
export type { SponsorshipSignalId };
export const sponsorshipSignalFilterOptions = [
...SPONSORSHIP_SIGNAL_IDS,
] as const;
/** Defaults aligned with the live cron set on jobs.levkin.ca. */
export const DEFAULT_PIPELINE_SOURCES: JobSource[] = [
"linkedin",
"gradcracker",
"indeed",
"glassdoor",
"qajobsboard",
"arcdev",
"eluta",
"bctenet",
"linkedin",
"ukvisajobs",
];
export const PIPELINE_SOURCES_STORAGE_KEY = "jobops.pipeline.sources";
@@ -1,4 +1,3 @@
import { serializeSponsorshipSignals } from "@shared/sponsorship-signals";
import { createJob } from "@shared/testing/factories";
import type { Job } from "@shared/types";
import { renderHook } from "@testing-library/react";
@@ -37,8 +36,6 @@ describe("useFilteredJobs", () => {
[],
[],
"all",
[],
false,
"all",
{ mode: "at_least", min: null, max: null },
{
@@ -70,8 +67,6 @@ describe("useFilteredJobs", () => {
[],
[],
"all",
[],
false,
"all",
{ mode: "at_least", min: null, max: null },
{
@@ -104,8 +99,6 @@ describe("useFilteredJobs", () => {
[],
[],
"confirmed",
[],
false,
"all",
{ mode: "at_least", min: null, max: null },
{
@@ -118,90 +111,6 @@ describe("useFilteredJobs", () => {
expect(result.current.map((job) => job.id)).toEqual(["confirmed"]);
});
it("filters by sponsorship text signals", () => {
const jobs: Job[] = [
{
...baseJob,
id: "tn",
sponsorshipSignals: serializeSponsorshipSignals(["tn_eligible"]),
},
{
...baseJob,
id: "sponsor",
sponsorshipSignals: serializeSponsorshipSignals(["will_sponsor"]),
},
{ ...baseJob, id: "none", sponsorshipSignals: null },
];
const { result } = renderHook(() =>
useFilteredJobs(
jobs,
"all",
[],
[],
[],
[],
"all",
["tn_eligible", "will_sponsor"],
false,
"all",
{ mode: "at_least", min: null, max: null },
{ key: "score", direction: "desc" },
),
);
expect(result.current.map((job) => job.id).sort()).toEqual([
"sponsor",
"tn",
]);
});
it("hides red-flag jobs but keeps yellow no-sponsor listings when enabled", () => {
const jobs: Job[] = [
{
...baseJob,
id: "red",
sponsorshipSignals: serializeSponsorshipSignals([
"citizenship_required",
]),
},
{
...baseJob,
id: "yellow",
sponsorshipSignals: serializeSponsorshipSignals([
"no_sponsorship_stated",
]),
},
{
...baseJob,
id: "tn",
sponsorshipSignals: serializeSponsorshipSignals(["tn_eligible"]),
},
];
const { result } = renderHook(() =>
useFilteredJobs(
jobs,
"all",
[],
[],
[],
[],
"all",
[],
true,
"all",
{ mode: "at_least", min: null, max: null },
{ key: "score", direction: "desc" },
),
);
expect(result.current.map((job) => job.id).sort()).toEqual([
"tn",
"yellow",
]);
});
it("filters by salary range using structured and text salary fields", () => {
const jobs: Job[] = [
{ ...baseJob, id: "structured", salaryMinAmount: 70000 },
@@ -219,8 +128,6 @@ describe("useFilteredJobs", () => {
[],
[],
"all",
[],
false,
"all",
{ mode: "between", min: 60000, max: 80000 },
{
@@ -253,8 +160,6 @@ describe("useFilteredJobs", () => {
[],
[],
"all",
[],
false,
"all",
{ mode: "at_least", min: null, max: null },
{
@@ -288,8 +193,6 @@ describe("useFilteredJobs", () => {
[],
[],
"all",
[],
false,
"remote",
{ mode: "at_least", min: null, max: null },
{ key: "score", direction: "desc" },
@@ -306,8 +209,6 @@ describe("useFilteredJobs", () => {
[],
[],
"all",
[],
false,
"not_remote",
{ mode: "at_least", min: null, max: null },
{ key: "score", direction: "desc" },
@@ -324,8 +225,6 @@ describe("useFilteredJobs", () => {
[],
[],
"all",
[],
false,
"unknown",
{ mode: "at_least", min: null, max: null },
{ key: "score", direction: "desc" },
@@ -350,8 +249,6 @@ describe("useFilteredJobs", () => {
[],
[],
"all",
[],
false,
"all",
{ mode: "at_least", min: null, max: null },
{ key: "score", direction: "desc" },
@@ -391,8 +288,6 @@ describe("useFilteredJobs", () => {
["united kingdom"],
[],
"all",
[],
false,
"all",
{ mode: "at_least", min: null, max: null },
{ key: "score", direction: "desc" },
@@ -417,8 +312,6 @@ describe("useFilteredJobs", () => {
[],
[],
"all",
[],
false,
"all",
{ mode: "at_least", min: null, max: null },
{ key: "score", direction: "desc" },
@@ -466,8 +359,6 @@ describe("useFilteredJobs", () => {
[],
["united kingdom"],
"all",
[],
false,
"all",
{ mode: "at_least", min: null, max: null },
{ key: "score", direction: "desc" },
@@ -490,8 +381,6 @@ describe("useFilteredJobs", () => {
[],
[],
"all",
[],
false,
"all",
{ mode: "at_least", min: null, max: null },
{ key: "score", direction: "desc" },
@@ -523,8 +412,6 @@ describe("useFilteredJobs", () => {
[],
[],
"all",
[],
false,
"all",
{ mode: "at_least", min: null, max: null },
{ key: "score", direction: "desc" },
@@ -556,8 +443,6 @@ describe("useFilteredJobs", () => {
[],
[],
"all",
[],
false,
"all",
{ mode: "at_least", min: null, max: null },
{ key: "score", direction: "desc" },
@@ -581,8 +466,6 @@ describe("useFilteredJobs", () => {
[],
[],
"all",
[],
false,
"all",
{ mode: "at_least", min: null, max: null },
{ key: "score", direction: "desc" },
@@ -614,8 +497,6 @@ describe("useFilteredJobs", () => {
[],
[],
"all",
[],
false,
"all",
{ mode: "at_least", min: null, max: null },
{ key: "score", direction: "desc" },
@@ -639,8 +520,6 @@ describe("useFilteredJobs", () => {
[],
[],
"all",
[],
false,
"all",
{ mode: "at_least", min: null, max: null },
{ key: "score", direction: "desc" },
@@ -1,11 +1,6 @@
import type { DuplicateDismissReason } from "@client/lib/job-dedup";
import { jobMatchesAllowedCountry } from "@shared/blocked-countries";
import { textMatchesKeyword } from "@shared/keyword-match";
import type { SponsorshipSignalId } from "@shared/sponsorship-signals";
import {
jobHasAnySponsorshipSignal,
jobHasRedSponsorshipFlag,
} from "@shared/sponsorship-signals";
import type { JobListItem, JobSource } from "@shared/types";
import { useMemo } from "react";
import type {
@@ -62,8 +57,6 @@ export const useFilteredJobs = (
countriesFilter: string[],
countriesExcludeFilter: string[],
sponsorFilter: SponsorFilter,
sponsorshipSignalsFilter: SponsorshipSignalId[],
hideSponsorBlockers: boolean,
workplaceFilter: WorkplaceFilter,
salaryFilter: SalaryFilter,
sort: JobSort,
@@ -161,21 +154,6 @@ export const useFilteredJobs = (
);
}
if (sponsorshipSignalsFilter.length > 0) {
filtered = filtered.filter((job) =>
jobHasAnySponsorshipSignal(
job.sponsorshipSignals,
sponsorshipSignalsFilter,
),
);
}
if (hideSponsorBlockers) {
filtered = filtered.filter(
(job) => !jobHasRedSponsorshipFlag(job.sponsorshipSignals),
);
}
if (workplaceFilter !== "all") {
filtered = filtered.filter((job) => {
if (workplaceFilter === "remote") return job.isRemote === true;
@@ -274,8 +252,6 @@ export const useFilteredJobs = (
countriesFilter,
countriesExcludeFilter,
sponsorFilter,
sponsorshipSignalsFilter,
hideSponsorBlockers,
workplaceFilter,
salaryFilter,
sort,
@@ -635,7 +635,6 @@ describe("useOrchestratorData", () => {
status: "discovered",
suitabilityScore: null,
sponsorMatchScore: null,
sponsorshipSignals: null,
jobType: null,
jobFunction: null,
salaryMinAmount: null,
@@ -3,7 +3,6 @@ import {
normalizeCountryKey,
SUPPORTED_COUNTRY_KEYS,
} from "@shared/location-support";
import type { SponsorshipSignalId } from "@shared/sponsorship-signals";
import type { JobSource } from "@shared/types";
import { useCallback, useEffect, useMemo } from "react";
import { useSearchParams } from "react-router-dom";
@@ -14,7 +13,7 @@ import type {
SponsorFilter,
WorkplaceFilter,
} from "./constants";
import { DEFAULT_SORT, sponsorshipSignalFilterOptions } from "./constants";
import { DEFAULT_SORT } from "./constants";
const allowedSponsorFilters: SponsorFilter[] = [
"all",
@@ -44,10 +43,6 @@ const allowedWorkplaceFilters: WorkplaceFilter[] = [
"unknown",
];
const allowedSponsorshipSignals = new Set<string>(
sponsorshipSignalFilterOptions,
);
const allowedJobSources = new Set<string>(EXTRACTOR_SOURCE_IDS);
const allowedCountryKeys = new Set(SUPPORTED_COUNTRY_KEYS);
@@ -219,58 +214,6 @@ export const useOrchestratorFilters = () => {
: "all";
}, [searchParams]);
const sponsorshipSignalsFilter = useMemo((): SponsorshipSignalId[] => {
const raw = searchParams.getAll("sponsorSignal");
const out: SponsorshipSignalId[] = [];
const seen = new Set<string>();
for (const value of raw) {
const trimmed = value.trim();
if (!trimmed || !allowedSponsorshipSignals.has(trimmed)) continue;
if (seen.has(trimmed)) continue;
seen.add(trimmed);
out.push(trimmed as SponsorshipSignalId);
}
return out;
}, [searchParams]);
const hideSponsorBlockers = useMemo(
() => searchParams.get("hideSponsorBlockers") === "1",
[searchParams],
);
const setHideSponsorBlockers = useCallback(
(value: boolean) => {
setSearchParams(
(prev) => {
if (value) prev.set("hideSponsorBlockers", "1");
else prev.delete("hideSponsorBlockers");
return prev;
},
{ replace: true },
);
},
[setSearchParams],
);
const setSponsorshipSignalsFilter = useCallback(
(values: SponsorshipSignalId[]) => {
setSearchParams(
(prev) => {
prev.delete("sponsorSignal");
const unique = [
...new Set(
values.filter((value) => allowedSponsorshipSignals.has(value)),
),
];
for (const value of unique) prev.append("sponsorSignal", value);
return prev;
},
{ replace: true },
);
},
[setSearchParams],
);
const setSponsorFilter = useCallback(
(value: SponsorFilter) => {
setSearchParams(
@@ -435,11 +378,6 @@ export const useOrchestratorFilters = () => {
[searchParams],
);
const hidePriorSkips = useMemo(
() => searchParams.get("hidePriorSkips") !== "0",
[searchParams],
);
const setApplySettingsCompanySkipList = useCallback(
(value: boolean) => {
setSearchParams(
@@ -454,20 +392,6 @@ export const useOrchestratorFilters = () => {
[setSearchParams],
);
const setHidePriorSkips = useCallback(
(value: boolean) => {
setSearchParams(
(prev) => {
if (value) prev.delete("hidePriorSkips");
else prev.set("hidePriorSkips", "0");
return prev;
},
{ replace: true },
);
},
[setSearchParams],
);
const sort = useMemo((): JobSort => {
const sortValue = searchParams.get("sort");
if (!sortValue) return DEFAULT_SORT;
@@ -514,8 +438,6 @@ export const useOrchestratorFilters = () => {
prev.delete("countries");
prev.delete("countriesExclude");
prev.delete("sponsor");
prev.delete("sponsorSignal");
prev.delete("hideSponsorBlockers");
prev.delete("workplace");
prev.delete("salaryMode");
prev.delete("salaryMin");
@@ -527,7 +449,6 @@ export const useOrchestratorFilters = () => {
prev.delete("employer");
prev.delete("employerExclude");
prev.delete("skipList");
prev.delete("hidePriorSkips");
return prev;
},
{ replace: true },
@@ -544,10 +465,6 @@ export const useOrchestratorFilters = () => {
setCountrySelection,
sponsorFilter,
setSponsorFilter,
sponsorshipSignalsFilter,
setSponsorshipSignalsFilter,
hideSponsorBlockers,
setHideSponsorBlockers,
workplaceFilter,
setWorkplaceFilter,
salaryFilter,
@@ -560,8 +477,6 @@ export const useOrchestratorFilters = () => {
setEmployerFilterTokens,
applySettingsCompanySkipList,
setApplySettingsCompanySkipList,
hidePriorSkips,
setHidePriorSkips,
sort,
setSort,
resetFilters,
@@ -10,7 +10,7 @@ import { toast } from "sonner";
import { trackProductEvent } from "@/lib/analytics";
import type { AutomaticRunValues } from "./automatic-run";
import {
buildMaxCoverageDiscoveryLimits,
deriveExtractorLimits,
mergeDiscoverySearchTerms,
serializeCityLocationsSetting,
} from "./automatic-run";
@@ -169,12 +169,10 @@ export function usePipelineControls(
return;
}
const mergedSearchTerms = values.activeKeywordSetId
? values.searchTerms.map((t) => t.trim()).filter(Boolean)
: mergeDiscoverySearchTerms(
values.searchTerms,
settings?.jobSearchProfile?.value?.targetRoles,
);
const mergedSearchTerms = mergeDiscoverySearchTerms(
values.searchTerms,
settings?.jobSearchProfile?.value?.targetRoles,
);
if (mergedSearchTerms.length === 0) {
toast.error(
"Add at least one search term, or set target roles on your job search profile.",
@@ -182,6 +180,11 @@ export function usePipelineControls(
return;
}
const limits = deriveExtractorLimits({
budget: values.runBudget,
searchTerms: mergedSearchTerms,
sources: compatibleSources,
});
const hasJobSpySite = compatibleSources.some(
(source) =>
source === "indeed" ||
@@ -199,19 +202,14 @@ export function usePipelineControls(
serializedCities
? serializedCities
: formatCountryLabel(values.country);
if (values.activeKeywordSetId) {
await api.updateKeywordSet(values.activeKeywordSetId, {
terms: values.searchTerms,
});
}
await api.updateSettings({
searchTerms: values.searchTerms,
workplaceTypes: values.workplaceTypes,
...buildMaxCoverageDiscoveryLimits({
budget: values.runBudget,
searchTerms: mergedSearchTerms,
sources: compatibleSources,
}),
jobspyResultsWanted: limits.jobspyResultsWanted,
gradcrackerMaxJobsPerTerm: limits.gradcrackerMaxJobsPerTerm,
ukvisajobsMaxJobs: limits.ukvisajobsMaxJobs,
adzunaMaxJobsPerTerm: limits.adzunaMaxJobsPerTerm,
startupjobsMaxJobsPerTerm: limits.startupjobsMaxJobsPerTerm,
jobspyCountryIndeed: values.country,
searchCities,
});
@@ -67,36 +67,7 @@ describe("usePipelineSources", () => {
expect(result.current.pipelineSources).toEqual(["gradcracker"]);
});
it("merges newly enabled sources into stored selection", () => {
ensureStorage().setItem(
PIPELINE_SOURCES_STORAGE_KEY,
JSON.stringify(["gradcracker"]),
);
const enabledSources = [
"gradcracker",
"linkedin",
"workingnomads",
] as const;
const { result } = renderHook(() => usePipelineSources(enabledSources));
expect(result.current.pipelineSources).toEqual([
"gradcracker",
"linkedin",
"workingnomads",
]);
});
it("selects all enabled sources when storage is empty", () => {
const enabledSources = ["gradcracker", "linkedin"] as const;
const { result } = renderHook(() => usePipelineSources(enabledSources));
expect(result.current.pipelineSources).toEqual(["gradcracker", "linkedin"]);
});
it("falls back to all enabled sources when stored sources are unavailable", () => {
it("falls back to the first enabled source", () => {
ensureStorage().setItem(
PIPELINE_SOURCES_STORAGE_KEY,
JSON.stringify(["ukvisajobs"]),
@@ -106,7 +77,7 @@ describe("usePipelineSources", () => {
const { result } = renderHook(() => usePipelineSources(enabledSources));
expect(result.current.pipelineSources).toEqual(["gradcracker", "linkedin"]);
expect(result.current.pipelineSources).toEqual(["gradcracker"]);
});
it("ignores toggles for disabled sources", () => {
@@ -11,14 +11,12 @@ const resolveAllowedSources = (enabledSources?: readonly JobSource[]) =>
? (enabledSources as JobSource[])
: DEFAULT_PIPELINE_SOURCES;
const mergeAllowedSources = (
const normalizeSources = (
sources: JobSource[],
allowedSources: JobSource[],
): JobSource[] => {
const kept = sources.filter((value) => allowedSources.includes(value));
const missing = allowedSources.filter((value) => !kept.includes(value));
const merged = [...kept, ...missing];
return merged.length > 0 ? merged : [...allowedSources];
) => {
const filtered = sources.filter((value) => allowedSources.includes(value));
return filtered.length > 0 ? filtered : allowedSources.slice(0, 1);
};
const sourcesMatch = (left: JobSource[], right: JobSource[]) =>
@@ -33,22 +31,22 @@ export const usePipelineSources = (enabledSources?: readonly JobSource[]) => {
const [pipelineSources, setPipelineSources] = useState<JobSource[]>(() => {
try {
const raw = localStorage.getItem(PIPELINE_SOURCES_STORAGE_KEY);
if (!raw) return mergeAllowedSources([], allowedSources);
if (!raw) return normalizeSources(allowedSources, allowedSources);
const parsed = JSON.parse(raw) as unknown;
if (!Array.isArray(parsed))
return mergeAllowedSources([], allowedSources);
return normalizeSources(allowedSources, allowedSources);
const next = parsed.filter((value): value is JobSource =>
orderedSources.includes(value as JobSource),
);
return mergeAllowedSources(next, allowedSources);
return normalizeSources(next, allowedSources);
} catch {
return mergeAllowedSources([], allowedSources);
return normalizeSources(allowedSources, allowedSources);
}
});
useEffect(() => {
setPipelineSources((current) => {
const normalized = mergeAllowedSources(current, allowedSources);
const normalized = normalizeSources(current, allowedSources);
return sourcesMatch(current, normalized) ? current : normalized;
});
}, [allowedSources]);
@@ -27,38 +27,6 @@ describe("orchestrator utils", () => {
expect(enabled).toContain("themuse");
});
it("enables new public job boards without credentials", () => {
const enabled = getEnabledSources(createAppSettings());
for (const source of [
"workingnomads",
"testdevjobs",
"wellfound",
"builtin",
"google-jobs",
] as const) {
expect(enabled).toContain(source);
}
});
it("enables ATS sources when company lists are configured", () => {
const enabled = getEnabledSources(
createAppSettings({
teamtailorCompanies: {
value: ["testgorilla"],
default: [],
override: null,
},
careersPageUrls: {
value: ["https://sentry.io/careers/"],
default: [],
override: null,
},
}),
);
expect(enabled).toContain("teamtailor");
expect(enabled).toContain("careerspages");
});
it("counts processing jobs in ready and discovered tabs", () => {
const jobs = [
createJob({ id: "ready", status: "ready", closedAt: null }),
@@ -212,12 +212,6 @@ export const getEnabledSources = (
const hasWorkdayTenants = (settings.workdayTenants?.value ?? []).length > 0;
const hasSmartrecruitersCompanies =
(settings.smartrecruitersCompanies?.value ?? []).length > 0;
const hasTeamtailorCompanies =
(settings.teamtailorCompanies?.value ?? []).length > 0;
const hasHuntflowTenants = (settings.huntflowTenants?.value ?? []).length > 0;
const hasFactorialTenants =
(settings.factorialTenants?.value ?? []).length > 0;
const hasCareersPageUrls = (settings.careersPageUrls?.value ?? []).length > 0;
const hasElutaRssLocations =
(settings.elutaRssLocations?.value ?? []).length > 0;
const hasIcimsTenants = (settings.icimsTenants?.value ?? []).length > 0;
@@ -287,22 +281,6 @@ export const getEnabledSources = (
if (hasSmartrecruitersCompanies) enabled.push(source);
continue;
}
if (source === "teamtailor") {
if (hasTeamtailorCompanies) enabled.push(source);
continue;
}
if (source === "huntflow") {
if (hasHuntflowTenants) enabled.push(source);
continue;
}
if (source === "factorial") {
if (hasFactorialTenants) enabled.push(source);
continue;
}
if (source === "careerspages") {
if (hasCareersPageUrls) enabled.push(source);
continue;
}
if (source === "icims") {
if (hasIcimsTenants) enabled.push(source);
continue;
@@ -329,12 +307,7 @@ export const getEnabledSources = (
source === "arbeitnow" ||
source === "himalayas" ||
source === "weworkremotely" ||
source === "workingnomads" ||
source === "fourdayweek" ||
source === "testdevjobs" ||
source === "wellfound" ||
source === "builtin" ||
source === "google-jobs" ||
source === "qajobsboard" ||
source === "arcdev"
) {
@@ -0,0 +1,137 @@
import * as api from "@client/api";
import type { RxResumeMode } from "@shared/types.js";
import { RefreshCw } from "lucide-react";
import type React from "react";
import { useCallback, useEffect, useState } from "react";
import { Button } from "@/components/ui/button";
import {
Select,
SelectContent,
SelectItem,
SelectTrigger,
SelectValue,
} from "@/components/ui/select";
type BaseResumeSelectionProps = {
value: string | null;
onValueChange: (value: string | null) => void;
hasRxResumeAccess: boolean;
rxresumeMode?: RxResumeMode;
disabled?: boolean;
isLoading?: boolean;
};
export const BaseResumeSelection: React.FC<BaseResumeSelectionProps> = ({
value,
onValueChange,
hasRxResumeAccess,
rxresumeMode,
disabled = false,
isLoading = false,
}) => {
const [resumes, setResumes] = useState<{ id: string; name: string }[]>([]);
const [isFetchingResumes, setIsFetchingResumes] = useState(false);
const [fetchError, setFetchError] = useState<string | null>(null);
const fetchResumes = useCallback(async () => {
if (!hasRxResumeAccess) {
setResumes([]);
setFetchError(null);
return;
}
setIsFetchingResumes(true);
setFetchError(null);
try {
const data = await api.getRxResumes(rxresumeMode);
setResumes(data);
// Preselect if only one option is available and no value is currently set
if (data.length === 1 && !value) {
onValueChange(data[0].id);
}
} catch (error) {
setResumes([]);
setFetchError(
error instanceof Error ? error.message : "Failed to fetch resumes",
);
} finally {
setIsFetchingResumes(false);
}
}, [hasRxResumeAccess, onValueChange, rxresumeMode, value]);
useEffect(() => {
if (hasRxResumeAccess) {
fetchResumes();
}
}, [hasRxResumeAccess, fetchResumes]);
useEffect(() => {
if (!hasRxResumeAccess) {
setResumes([]);
setFetchError(null);
}
}, [hasRxResumeAccess]);
return (
<div className="space-y-2">
<div className="flex items-center justify-between">
<div className="text-sm font-medium">Template Resume</div>
<Button
variant="ghost"
size="sm"
onClick={fetchResumes}
disabled={isFetchingResumes || isLoading || disabled}
className="h-8 px-2"
>
<RefreshCw
className={`h-3 w-3 mr-1 ${isFetchingResumes ? "animate-spin" : ""}`}
/>
Refresh
</Button>
</div>
<Select
value={value || ""}
onValueChange={(val: string) => onValueChange(val || null)}
disabled={disabled || isLoading || isFetchingResumes}
>
<SelectTrigger>
<SelectValue
placeholder={
resumes.length > 0
? "Select a template resume..."
: "No resumes found"
}
/>
</SelectTrigger>
<SelectContent>
{resumes.map((resume) => (
<SelectItem key={resume.id} value={resume.id}>
{resume.name}
</SelectItem>
))}
</SelectContent>
</Select>
{resumes.length === 0 && !isFetchingResumes && !fetchError && (
<div className="text-xs text-amber-600 dark:text-amber-400 mt-2">
No resumes found in your account. Please create a resume on the{" "}
<a
href="https://rxresu.me"
target="_blank"
rel="noreferrer"
className="font-semibold underline underline-offset-2"
>
Reactive Resume website
</a>{" "}
first.
</div>
)}
{fetchError && (
<div className="text-xs text-destructive mt-1">{fetchError}</div>
)}
</div>
);
};
@@ -34,8 +34,6 @@ type DangerZoneSectionProps = {
handleClearByStatuses: () => void;
handleClearDatabase: () => void;
handleClearByScore?: (threshold: number) => void;
handleClearDiscoveredMatchingArchived?: () => void;
handleExpireDeadListings?: () => void;
isLoading: boolean;
isSaving: boolean;
};
@@ -46,8 +44,6 @@ export const DangerZoneSection: React.FC<DangerZoneSectionProps> = ({
handleClearByStatuses,
handleClearDatabase,
handleClearByScore,
handleClearDiscoveredMatchingArchived,
handleExpireDeadListings,
isLoading,
isSaving,
}) => {
@@ -158,106 +154,6 @@ export const DangerZoneSection: React.FC<DangerZoneSectionProps> = ({
<Separator />
{handleClearDiscoveredMatchingArchived && (
<div className="flex flex-col gap-4 sm:flex-row sm:items-center sm:justify-between p-3 rounded-md">
<div className="space-y-0.5">
<div className="text-sm font-semibold text-destructive">
Clear Rediscovered Archive Matches
</div>
<div className="text-xs text-muted-foreground">
Delete Discovered jobs that match roles you already skipped or
applied (same employer + title, including cross-source
reposts).
</div>
</div>
<AlertDialog>
<AlertDialogTrigger asChild>
<Button
variant="destructive"
size="sm"
disabled={isLoading || isSaving}
>
<Trash2 className="mr-2 h-4 w-4" />
Clear Matches
</Button>
</AlertDialogTrigger>
<AlertDialogContent>
<AlertDialogHeader>
<AlertDialogTitle>
Clear rediscovered archive matches?
</AlertDialogTitle>
<AlertDialogDescription>
This permanently deletes Discovered jobs that match
skipped or applied roles. Ready, applied, and skipped jobs
are not deleted. This action cannot be undone.
</AlertDialogDescription>
</AlertDialogHeader>
<AlertDialogFooter>
<AlertDialogCancel>Cancel</AlertDialogCancel>
<AlertDialogAction
onClick={handleClearDiscoveredMatchingArchived}
className="bg-destructive text-destructive-foreground hover:bg-destructive/90"
>
Clear matches
</AlertDialogAction>
</AlertDialogFooter>
</AlertDialogContent>
</AlertDialog>
</div>
)}
{handleExpireDeadListings && (
<>
<Separator />
<div className="flex flex-col gap-4 sm:flex-row sm:items-center sm:justify-between p-3 rounded-md">
<div className="space-y-0.5">
<div className="text-sm font-semibold text-destructive">
Mark Expired Listings
</div>
<div className="text-xs text-muted-foreground">
Open each Discovered job URL. If the page is gone (404/410
or job expired), mark it expired so it leaves the queue.
</div>
</div>
<AlertDialog>
<AlertDialogTrigger asChild>
<Button
variant="destructive"
size="sm"
disabled={isLoading || isSaving}
>
<Trash2 className="mr-2 h-4 w-4" />
Mark Expired
</Button>
</AlertDialogTrigger>
<AlertDialogContent>
<AlertDialogHeader>
<AlertDialogTitle>
Mark expired listings?
</AlertDialogTitle>
<AlertDialogDescription>
This probes Discovered job links and marks dead ones as
expired. Network failures are ignored (jobs are kept).
This can take a few minutes for large queues.
</AlertDialogDescription>
</AlertDialogHeader>
<AlertDialogFooter>
<AlertDialogCancel>Cancel</AlertDialogCancel>
<AlertDialogAction
onClick={handleExpireDeadListings}
className="bg-destructive text-destructive-foreground hover:bg-destructive/90"
>
Mark expired
</AlertDialogAction>
</AlertDialogFooter>
</AlertDialogContent>
</AlertDialog>
</div>
</>
)}
<Separator />
{/* Clear Jobs Below Score */}
{handleClearByScore && (
<div className="p-3 rounded-md space-y-4">

Some files were not shown because too many files have changed in this diff Show More