Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
6a5c9e8cc4 | ||
|
|
abfc5d8d0a | ||
|
|
2c930ba40f | ||
|
|
529f0a7b02 | ||
|
|
40c7cdece3 | ||
|
|
84e6835b11 | ||
|
|
ecf1829ed8 |
@@ -254,3 +254,47 @@ ADZUNA_APP_KEY=
|
||||
# Comma-separated paths under https://arc.dev used when seeding defaults (e.g. Playwright + Cypress feeds).
|
||||
# ARC_REMOTE_JOBS_PATHS=/remote-jobs/playwright,/remote-jobs/cypress
|
||||
# Prefer Settings for overrides: arcRemoteJobsPaths (JSON array), arcMaxJobsPerPath (default 120).
|
||||
|
||||
# =============================================================================
|
||||
# New public job boards — optional (auto-enabled, no credentials)
|
||||
# =============================================================================
|
||||
# Caps via Settings: workingnomadsMaxJobsPerTerm, testdevjobsMaxJobsPerTerm,
|
||||
# testdevjobsMaxPages, builtinMaxJobsPerTerm, builtinMaxPagesPerTerm,
|
||||
# wellfoundMaxJobsPerTerm, googleJobsMaxJobsPerTerm (Wellfound and Google
|
||||
# Jobs both use Camoufox when installed).
|
||||
|
||||
# =============================================================================
|
||||
# Google Jobs — optional (auto-enabled, no credentials, unofficial scrape)
|
||||
# =============================================================================
|
||||
# GOOGLE_JOBS_PROXY=http://user:pass@proxy-host:port
|
||||
# ^ Most reliable way to avoid Google's CAPTCHA/rate-limit wall (residential or
|
||||
# mobile proxy recommended; datacenter IPs, including most VPS/cloud hosts,
|
||||
# are aggressively blocked). Applies to both manual runs and the pipeline.
|
||||
#
|
||||
# GOOGLE_JOBS_HEADLESS=false
|
||||
# ^ Manual/local use ONLY (do not set this in the server .env for the
|
||||
# unattended pipeline/cron — a headless server has no display for a human
|
||||
# to see the browser). Opens a real Firefox window so you can solve a
|
||||
# CAPTCHA by hand; waits up to GOOGLE_JOBS_UNBLOCK_TIMEOUT_MS (default
|
||||
# 300000 = 5 min) before giving up. Run directly, e.g.:
|
||||
# GOOGLE_JOBS_HEADLESS=false npm --workspace google-jobs-extractor run start
|
||||
# Session cookies are saved to extractors/google-jobs/storage/state.json
|
||||
# and reused by later runs (including headless/automated ones), which can
|
||||
# reduce how often future runs hit a CAPTCHA.
|
||||
#
|
||||
# Caps via Settings: googleJobsMaxJobsPerTerm (default 30, max 150).
|
||||
#
|
||||
# Note: there is no country/location override for this extractor — Google
|
||||
# resolves location from the browser's real (or GOOGLE_JOBS_PROXY's) IP.
|
||||
# Forcing a mismatched country has been observed to make Google's Jobs
|
||||
# vertical return "no matches" even though the tab loads correctly. To
|
||||
# target a different country's jobs, use a proxy with an exit node there.
|
||||
|
||||
# =============================================================================
|
||||
# Teamtailor / Huntflow / Factorial / Career pages — optional
|
||||
# =============================================================================
|
||||
# TEAMTAILOR_COMPANIES=testgorilla,polestar
|
||||
# HUNTFLOW_TENANTS=apicworld
|
||||
# FACTORIAL_TENANTS=yourbourse
|
||||
# CAREERS_PAGE_URLS=https://sentry.io/careers/,https://example.com/careers
|
||||
# Prefer Settings → Job sources & ATS for the same fields.
|
||||
|
||||
@@ -1,4 +1,5 @@
|
||||
---
|
||||
# ci-sync: 2026-05-30T02:31:18Z
|
||||
# Homelab CI — Docker/heavy lane (git-ci-02)
|
||||
name: CI
|
||||
|
||||
|
||||
+6
-1
@@ -18,6 +18,11 @@ data/
|
||||
# Extractor storage outputs and cached auth (per-extractor runtime data)
|
||||
extractors/*/storage/
|
||||
|
||||
# OS files
|
||||
# OS / editor
|
||||
.DS_Store
|
||||
Thumbs.db
|
||||
.vscode/
|
||||
|
||||
# Local scratch / extractor debug dumps (not part of the app)
|
||||
storage/
|
||||
solution-workload-test.*
|
||||
|
||||
@@ -20,9 +20,13 @@ paths = [
|
||||
'''(?i).*\.example\.(yml|yaml|env|json|toml)$''',
|
||||
'''(?i).*vault\.example\.(yml|yaml)$''',
|
||||
'''(?i).*\.env\.example$''',
|
||||
'''(?i)docs-site/docs/extractors/.*''',
|
||||
'''(?i)extractors/golangjobs/.*''',
|
||||
'''(?i)orchestrator/src/server/infra/product-analytics\.ts$''',
|
||||
]
|
||||
regexes = [
|
||||
'''(?i)(invalid|fake|dummy|placeholder|example|changeme|change_me|not-a-real)''',
|
||||
'''(?i)sk-or-invalid''',
|
||||
'''(?i)msk-or-invalid''',
|
||||
'''(?i)YOUR_API_KEY''',
|
||||
]
|
||||
|
||||
@@ -55,6 +55,14 @@ git add -A && git commit -m "Your message" && git push gitea main
|
||||
|
||||
5. Open the UI: `http://<VM-IP>:3005` (port mapped in `docker-compose.yml`).
|
||||
|
||||
**Local machine (loopback only, avoids binding `0.0.0.0:3005`):**
|
||||
|
||||
```bash
|
||||
docker compose -f docker-compose.yml -f docker-compose.localhost.yml up -d --build
|
||||
```
|
||||
|
||||
UI: `http://127.0.0.1:13005`
|
||||
|
||||
6. Persist data: compose mounts `./data` — back up that directory.
|
||||
|
||||
---
|
||||
|
||||
+2
-1
@@ -12,8 +12,9 @@
|
||||
"!!**/.venv",
|
||||
"!!docs-site/.docusaurus",
|
||||
"!!docs-site/build",
|
||||
"!!extractors/jobspy/storage",
|
||||
"!!extractors/*/storage",
|
||||
"!!orchestrator/storage",
|
||||
"!!storage",
|
||||
"!!data"
|
||||
]
|
||||
},
|
||||
|
||||
@@ -0,0 +1,6 @@
|
||||
# Local-only overlay: single loopback port (does not publish 0.0.0.0:3005).
|
||||
# Usage: docker compose -f docker-compose.yml -f docker-compose.localhost.yml up -d --build
|
||||
services:
|
||||
job-ops:
|
||||
ports: !override
|
||||
- "127.0.0.1:13005:3001"
|
||||
@@ -32,10 +32,11 @@ Eluta surfaces Canadian roles indexed directly from employer career sites, often
|
||||
### Empty feeds
|
||||
|
||||
- The `location` string may be too broad or spelled differently than Eluta expects. Try a major city plus province (e.g. `Calgary, AB`).
|
||||
- Eluta's RSS is a small recent slice of *all* jobs for that location (often retail/hospitality-heavy). Pipeline search terms like `software engineer` can legitimately match zero items even when the feed itself is healthy. Broaden terms or add more metro locations.
|
||||
|
||||
### RSS HTTP errors
|
||||
### RSS HTTP errors / "Too Many Requests"
|
||||
|
||||
- Eluta may block unusual clients; the extractor sends a conventional User-Agent. Retry later or reduce the number of location feeds per run.
|
||||
- Eluta rate-limits aggressive or browser-like clients. The extractor uses a dedicated `JobOps/... Eluta RSS consumer` User-Agent; retry later or reduce the number of location feeds per run.
|
||||
|
||||
## Related pages
|
||||
|
||||
|
||||
@@ -0,0 +1,107 @@
|
||||
---
|
||||
id: google-jobs
|
||||
title: Google Jobs Extractor
|
||||
description: Browser-backed scraping of the Google for Jobs search widget.
|
||||
sidebar_position: 8
|
||||
---
|
||||
|
||||
## What it is
|
||||
|
||||
Original feature: [Google for Jobs](https://www.google.com/search?q=jobs&udm=8)
|
||||
|
||||
Google Jobs is a browser-backed extractor that opens Google's Jobs vertical (`udm=8`) in Firefox (Camoufox when available) for each pipeline search term, scrolls the results panel to load additional cards, then clicks into each card to resolve a real "Apply on <site>" link and description before mapping rows into the orchestrator `CreateJobInput` shape.
|
||||
|
||||
Implementation split:
|
||||
|
||||
1. `extractors/google-jobs/src/main.ts` builds the search URL, waits for the jobs list, scrolls to lazy-load more cards, extracts card metadata (title/employer/location/"via" source), clicks each card to read the detail pane, and writes dataset JSON.
|
||||
2. `extractors/google-jobs/src/run.ts` spawns the browser subprocess, applies a hard timeout, reads the dataset, and maps rows to `CreateJobInput`.
|
||||
3. `extractors/google-jobs/manifest.ts` wires pipeline `searchTerms` into the extractor.
|
||||
|
||||
### A note on the search URL and location
|
||||
|
||||
Google's Jobs vertical is reached with `udm=8` (its current internal vertical id). An older, commonly-documented deep link — `ibp=htl;jobs` — still works but gets silently redirected to `udm=8` anyway, and combining it with extra parameters (a `num=` result-count hint, or a `gl=` country override that doesn't match the browser's real/proxied IP) has been observed to make Google return "no matches" or "can't determine location" even though the tab itself loads correctly.
|
||||
|
||||
Because of that, this extractor does **not** send a `gl` (country) parameter — location is resolved by Google from the browser's real IP (or `GOOGLE_JOBS_PROXY`'s exit IP), which Camoufox's `geoip` option keeps internally consistent. To target a different country's jobs, use a residential proxy with an exit node in that country rather than trying to override the country via a parameter.
|
||||
|
||||
### Dealing with Google's CAPTCHA / bot detection
|
||||
|
||||
Google actively fingerprints and challenges automated browser traffic, especially from datacenter/VPS IPs (which most servers running JobOps' cron pipeline will have). The extractor has layered mitigations, from always-on to opt-in:
|
||||
|
||||
1. **Always on:** Camoufox launches with `humanize: true` and `geoip: true` (human-like cursor movement, IP-consistent locale/geolocation) to reduce — not eliminate — fingerprint-based detection.
|
||||
2. **Always on:** browser cookies/session are persisted to `extractors/google-jobs/storage/state.json` and reused on the next run. A session that has cleared a CAPTCHA once (including via option 4 below) tends to see fewer challenges for a while afterward.
|
||||
3. **Opt-in, most reliable:** set `GOOGLE_JOBS_PROXY` (e.g. a residential/mobile proxy URL) to avoid datacenter-IP blocks in the first place. This is the standard real-world fix; unproxied runs from cloud servers should expect to be blocked often.
|
||||
4. **Opt-in, manual/local only:** set `GOOGLE_JOBS_HEADLESS=false` and run the extractor directly (not via the pipeline) to open a real, visible Firefox window. If Google shows a CAPTCHA, solve it yourself in that window — the script polls and automatically resumes once it clears (up to `GOOGLE_JOBS_UNBLOCK_TIMEOUT_MS`, default 5 minutes). The resulting session cookies are saved and reused by later automated/headless runs. **Do not** set `GOOGLE_JOBS_HEADLESS=false` in the server's always-on `.env` — a headless server has no display for a human to interact with, and the run will just wait until the timeout.
|
||||
|
||||
## Why it exists
|
||||
|
||||
Google for Jobs aggregates postings across the open web — including boards and employer career pages that JobOps does not otherwise scrape. Adding it as a proper source closes some of the gap between "jobs found by manually Googling a role" and "jobs discovered by the pipeline."
|
||||
|
||||
It is **not** a general web crawler and does not replace dedicated extractors: it only surfaces whatever Google's Jobs widget chooses to show for a query, subject to Google's own ranking, freshness, and regional availability.
|
||||
|
||||
## How to use it
|
||||
|
||||
1. Open **Run jobs** and choose **Automatic**.
|
||||
2. **Google Jobs** is enabled by default in **Sources** (toggle it off if you do not want it for this run).
|
||||
3. Set your existing automatic run knobs:
|
||||
- `searchTerms` become the `"<term> jobs"` query sent to Google per term.
|
||||
- `googleJobsMaxJobsPerTerm` (Settings) caps jobs scraped per run (default `30`, max `150`).
|
||||
- Location is derived from the browser's real/proxied IP, not from a country setting — see [the note above](#a-note-on-the-search-url-and-location).
|
||||
4. Start the run and watch progress in the pipeline progress card.
|
||||
|
||||
Local run example (headless, no proxy):
|
||||
|
||||
```bash
|
||||
GOOGLE_JOBS_SEARCH_TERMS='["automation engineer"]' \
|
||||
GOOGLE_JOBS_MAX_JOBS='10' \
|
||||
npm --workspace google-jobs-extractor run start
|
||||
```
|
||||
|
||||
Local run example (headed, solve the CAPTCHA yourself if one appears):
|
||||
|
||||
```bash
|
||||
GOOGLE_JOBS_HEADLESS=false \
|
||||
GOOGLE_JOBS_SEARCH_TERMS='["automation engineer"]' \
|
||||
npm --workspace google-jobs-extractor run start
|
||||
```
|
||||
|
||||
Defaults and constraints:
|
||||
|
||||
- No credentials required; this is a direct HTML/DOM scrape of Google's public search results, not an official API (Google does not offer a free Google Jobs API).
|
||||
- Google's markup and CSS class names change frequently and are obfuscated; the scraper favors known current card classes (e.g. `.EimVGf`) with multiple fallback selectors and a leaf-node text heuristic, but selectors can still rot and return zero results until updated.
|
||||
- Every card carries Google's own `data-share-url` permalink for that specific listing, which the extractor uses as `jobUrl` — so a job stays unique and reviewable even if the "click into the card for a detail pane" step below fails for it.
|
||||
- Because resolving a real application link requires clicking each card in a real browser, this extractor is slower per job than API-backed sources — keep `googleJobsMaxJobsPerTerm` modest for frequent runs.
|
||||
- If no external "Apply on <site>" link is found for a card, the extractor falls back to Google's own permalink for that listing so it's still reviewable, but not directly one-click-applyable.
|
||||
- After scraping, JobOps probes each apply URL (preferring the external link over Google's share permalink). Listings that return **404/410** or pages that say the job expired / is no longer available are **dropped before import**. Network failures fail-open (job is kept).
|
||||
- Automated, frequent, or high-volume scraping of Google search results may be against Google's Terms of Service; treat this extractor as best-effort and keep run frequency/volume conservative.
|
||||
- See [Dealing with Google's CAPTCHA / bot detection](#dealing-with-googles-captcha--bot-detection) above for proxy and interactive-solve options.
|
||||
|
||||
## Common problems
|
||||
|
||||
### Google Jobs returns 0 jobs for a term
|
||||
|
||||
The extractor logs one of two distinct messages to help tell these apart:
|
||||
|
||||
- `Google's Jobs tab reported no matches for "<term>"` — Google correctly landed on the Jobs tab but genuinely has no results for that query/locale. Try a broader, more common phrasing (e.g. `"automation engineer"` instead of a very narrow title).
|
||||
- `Jobs list did not appear for "<term>"` — an unrecognized page state (consent wall, layout change, or rotted selectors). The extractor saves a screenshot and full HTML dump to `extractors/google-jobs/storage/debug-last.png` / `debug-last.html` for inspection; compare it against a fresh manual search to see what changed.
|
||||
|
||||
### Results look sparse or missing company/location
|
||||
|
||||
- The list-view heuristics rely on leaf DOM nodes near the job title; if Google restructures the card markup, company/location extraction can degrade even when titles still work. This is expected best-effort behavior for an unofficial scrape.
|
||||
|
||||
### Run is slow
|
||||
|
||||
- Each job requires a real click + detail-pane render, unlike single-request API extractors. Lower `googleJobsMaxJobsPerTerm` or reduce the number of search terms for faster runs.
|
||||
|
||||
### Blocked, CAPTCHA'd, or consistently empty results
|
||||
|
||||
- Google may rate-limit or challenge automated browser traffic, especially from datacenter/VPS IPs. In order of effort:
|
||||
1. Set `GOOGLE_JOBS_PROXY` to a residential/mobile proxy — the most reliable fix for unattended/server runs.
|
||||
2. Run locally with `GOOGLE_JOBS_HEADLESS=false` and solve the CAPTCHA yourself once; the saved session (`storage/state.json`) may reduce blocks on subsequent headless runs for a while.
|
||||
3. Reduce run frequency, lower the per-term cap, or disable the source for that run and rely on other extractors / Manual Import instead.
|
||||
|
||||
## Related pages
|
||||
|
||||
- [Extractors Overview](/docs/next/extractors/overview)
|
||||
- [Manual Import](/docs/next/extractors/manual)
|
||||
- [Pipeline Run](/docs/next/features/pipeline-run)
|
||||
- [Settings](/docs/next/features/settings)
|
||||
@@ -46,3 +46,11 @@ Set `JOBOPS_EMIT_PROGRESS=1` for structured progress lines consumable by orchest
|
||||
|
||||
- Uses Playwright + Crawlee via Camoufox.
|
||||
- Low concurrency and longer timeouts for stability.
|
||||
- Employer logo (`figure img` alt text) and employer link are optional. Cards that omit them used to throw a Playwright `getAttribute` timeout and fail the whole region page (0 jobs saved even after cards were counted). Missing employer fields are now skipped per card.
|
||||
|
||||
## Common problems
|
||||
|
||||
### Region pages time out / run returns 0 jobs
|
||||
|
||||
- Gradcracker Livewire markup sometimes omits the employer logo on a card. Older builds aborted the whole page on that timeout. Current builds continue and leave employer blank.
|
||||
- Anti-bot / Camoufox issues: ensure Camoufox is installed (`npx camoufox-js fetch`) and prefer headed runs when debugging.
|
||||
|
||||
@@ -0,0 +1,45 @@
|
||||
---
|
||||
id: himalayas
|
||||
title: Himalayas Extractor
|
||||
description: Public remote-jobs API pagination and client-side term filtering.
|
||||
sidebar_position: 16
|
||||
---
|
||||
|
||||
## What it is
|
||||
|
||||
Original site: [himalayas.app](https://himalayas.app)
|
||||
|
||||
The extractor lives in `extractors/himalayas/manifest.ts`. It paginates the public JSON API (`https://himalayas.app/jobs/api?limit=&offset=`), filters rows client-side by pipeline search terms (title + categories), and maps matches into `CreateJobInput`.
|
||||
|
||||
## Why it exists
|
||||
|
||||
Himalayas is a large remote-jobs index with a stable, unauthenticated API — useful for remote-first discovery without browser automation or credentials.
|
||||
|
||||
## How to use it
|
||||
|
||||
1. Enable **Himalayas** in pipeline sources.
|
||||
2. Optionally set **Himalayas max jobs per term** (`himalayasMaxJobsPerTerm`, default `100`).
|
||||
3. Pipeline `searchTerms` filter titles/categories client-side (substring match, case-insensitive).
|
||||
4. Run the pipeline.
|
||||
|
||||
### Defaults and constraints
|
||||
|
||||
- The upstream API **silently caps `limit` at 20** per request. The extractor uses `PAGE_SIZE = 20` and paginates with `offset` (up to 10 pages / 200 rows scanned per run before the per-term cap).
|
||||
- There is no server-side search — if a term is rare, early pages may not contain matches even though later pages do. Raise the page budget only by changing the extractor constants if you need deeper scans.
|
||||
- No auth required.
|
||||
|
||||
## Common problems
|
||||
|
||||
### Zero jobs for a term I know exists
|
||||
|
||||
- The term may not appear in the first ~200 API rows (newest-first feed). Try a shorter token (`software`, `engineer`, `SDET`) or accept that Himalayas skews toward whatever is currently at the top of the feed.
|
||||
- Confirm you are not hitting an old build that requested `limit=50` and stopped after one page (fixed: the API returns at most 20, which used to look like end-of-feed).
|
||||
|
||||
### Rate limits / HTTP errors
|
||||
|
||||
- Retry later; the public API is generally stable but can throttle burst traffic.
|
||||
|
||||
## Related pages
|
||||
|
||||
- [Extractors Overview](/docs/next/extractors/overview)
|
||||
- [Add an Extractor](/docs/next/workflows/add-an-extractor)
|
||||
@@ -11,11 +11,11 @@ Original website: [hiring.cafe](https://hiring.cafe)
|
||||
|
||||
Special thanks: Initial implementation inspiration came from [umur957/hiring-cafe-job-scraper](https://github.com/umur957/hiring-cafe-job-scraper).
|
||||
|
||||
Hiring Cafe is a browser-backed extractor that queries Hiring Cafe search APIs and maps results into the orchestrator `CreateJobInput` shape.
|
||||
Hiring Cafe is a browser-backed extractor that loads the site in Firefox (Camoufox when available), then fetches paginated search results through Hiring Cafe's Next.js SSR data endpoint and maps rows into the orchestrator `CreateJobInput` shape.
|
||||
|
||||
Implementation split:
|
||||
|
||||
1. `extractors/hiringcafe/src/main.ts` builds search state, calls Hiring Cafe APIs, and writes dataset JSON.
|
||||
1. `extractors/hiringcafe/src/main.ts` builds search state, reads the page `buildId`, calls `/_next/data/{buildId}/index.json?searchState=...&page=...`, and writes dataset JSON.
|
||||
2. `orchestrator/src/server/services/hiring-cafe.ts` runs the extractor, streams progress events, and maps rows for pipeline import.
|
||||
|
||||
## Why it exists
|
||||
@@ -57,6 +57,12 @@ npm --workspace hiringcafe-extractor run start
|
||||
|
||||
## Common problems
|
||||
|
||||
### Hiring Cafe returns 401 Unauthorized or 404 on `/api/search-jobs`
|
||||
|
||||
- Hiring Cafe removed or locked down the legacy GET `/api/search-jobs` and `/api/search-jobs/get-total-count` endpoints.
|
||||
- Current JobOps builds use the Next.js SSR transport (`ssrHits`, `ssrTotalCount`, `ssrIsLastPage`) instead.
|
||||
- Upgrade to a build that includes this change if logs still show `401` / `404` on the old API paths.
|
||||
|
||||
### Hiring Cafe returns 429 / Vercel security checkpoint
|
||||
|
||||
- The extractor first attempts Camoufox-backed Firefox and falls back to vanilla Firefox startup if Camoufox is unstable locally.
|
||||
|
||||
@@ -61,4 +61,8 @@ The service in `orchestrator/src/server/services/jobspy.ts`:
|
||||
- A run returned fewer LinkedIn descriptions than expected.
|
||||
`JOBSPY_LINKEDIN_FETCH_DESCRIPTION=0` disables description fetching to speed up runs.
|
||||
- Different cities need different workplace-type filters.
|
||||
This is not supported in the current automatic-run flow. JobSpy receives one global workplace-type selection per run/query invocation.
|
||||
This is not supported in the current automatic-run flow. JobSpy receives one global workplace-type selection per run/query invocation.
|
||||
- Glassdoor always returns 0 jobs (fixed).
|
||||
Upstream `python-jobspy` (through at least `1.1.82`) discards an entire page of Glassdoor results whenever Glassdoor's GraphQL response includes *any* `errors` entry — even for an unrelated field. In practice, Glassdoor's backend consistently returns a partial error for `jobsPageSeoData` (`SUBREQUEST_HTTP_ERROR` / "dns error" on Glassdoor's own infra) while the actual job listings still come back fine, so every Glassdoor run silently returned 0 jobs. `extractors/jobspy/scrape_jobs.py` monkeypatches `Glassdoor._fetch_jobs_page` at import time to only treat the response as failed when `data.jobListings` is actually missing, and logs a warning (not an error) when it ignores an unrelated partial error. `requirements.txt` pins `python-jobspy==1.1.82` so an upstream version bump doesn't silently drop this patch — re-verify the patched method still matches before removing the pin.
|
||||
- A Glassdoor run logs `429` / "Blocked by Glassdoor for too many requests".
|
||||
This is a real rate limit from running too many Glassdoor requests back-to-back (e.g. repeated manual test runs). Space out runs; the automatic pipeline's normal cadence does not trigger this in practice.
|
||||
@@ -16,7 +16,7 @@ Extractor integrations are now registered through manifests and loaded automatic
|
||||
| [Gradcracker](/docs/next/extractors/gradcracker) | UK graduate roles from Gradcracker | Crawling stability depends on page structure and anti-bot behavior; tuned for low concurrency | `GRADCRACKER_SEARCH_TERMS`, `GRADCRACKER_MAX_JOBS_PER_TERM`, `JOBOPS_SKIP_APPLY_FOR_EXISTING` | Scrapes listing metadata, then detail pages and apply URL resolution |
|
||||
| [JobSpy](/docs/next/extractors/jobspy) | Multi-source discovery (Indeed, LinkedIn, Glassdoor) | Requires Python wrapper execution per term; source availability and quality vary by site/location | `JOBSPY_SITES`, `JOBSPY_SEARCH_TERMS`, `JOBSPY_RESULTS_WANTED`, `JOBSPY_HOURS_OLD`, `JOBSPY_LINKEDIN_FETCH_DESCRIPTION` | Produces JSON per term, then orchestrator normalizes and de-duplicates by `jobUrl` |
|
||||
| [Adzuna](/docs/next/extractors/adzuna) | API-based multi-country discovery with low scraping overhead | Requires valid App ID/App Key; country must be in Adzuna-supported list | `ADZUNA_APP_ID`, `ADZUNA_APP_KEY`, `ADZUNA_MAX_JOBS_PER_TERM` | API pagination to dataset output; orchestrator maps progress and de-duplicates by `sourceJobId`/`jobUrl` |
|
||||
| [Hiring Cafe](/docs/next/extractors/hiring-cafe) | Browser-backed discovery using Hiring Cafe search APIs | Subject to upstream anti-bot checks; uses browser context and encoded search-state payloads | `HIRING_CAFE_SEARCH_TERMS`, `HIRING_CAFE_COUNTRY`, `HIRING_CAFE_MAX_JOBS_PER_TERM`, `HIRING_CAFE_DATE_FETCHED_PAST_N_DAYS` | Uses existing pipeline term/country/budget knobs and maps directly to normalized jobs |
|
||||
| [Hiring Cafe](/docs/next/extractors/hiring-cafe) | Browser-backed discovery via Hiring Cafe Next.js SSR search payload | Subject to upstream anti-bot checks; uses browser context and JSON `searchState` query params | `HIRING_CAFE_SEARCH_TERMS`, `HIRING_CAFE_COUNTRY`, `HIRING_CAFE_MAX_JOBS_PER_TERM`, `HIRING_CAFE_DATE_FETCHED_PAST_N_DAYS` | Uses existing pipeline term/country/budget knobs and maps directly to normalized jobs |
|
||||
| [startup.jobs](/docs/next/extractors/startup-jobs) | Startup-focused discovery through the published `startup-jobs-scraper` package | No credentials required; detail enrichment depends on Playwright browser binaries being installed | existing pipeline `searchTerms`, selected country/cities, `jobspyResultsWanted`; `npx playwright install` for fresh environments | Algolia-backed search plus detail-page enrichment via package import; orchestrator maps normalized records and de-duplicates by `jobUrl` |
|
||||
| [UKVisaJobs](/docs/next/extractors/ukvisajobs) | UK visa sponsorship-focused roles | Requires authenticated session and periodic token/cookie refresh | `UKVISAJOBS_EMAIL`, `UKVISAJOBS_PASSWORD`, `UKVISAJOBS_MAX_JOBS`, `UKVISAJOBS_SEARCH_KEYWORD` | API pagination + dataset output; orchestrator de-dupes and may fetch missing descriptions |
|
||||
| [SmartRecruiters](/docs/next/extractors/smartrecruiters) | Enterprise employers on SmartRecruiters public boards | No auth; needs configured company identifiers; one HTTP round-trip per posting for apply URLs + descriptions | `SMARTRECRUITERS_COMPANIES`, `SMARTRECRUITERS_MAX_JOBS_PER_COMPANY` | Paginates the public Posting API, filters by pipeline terms, normalizes to `CreateJobInput` |
|
||||
@@ -25,6 +25,7 @@ Extractor integrations are now registered through manifests and loaded automatic
|
||||
| [Eluta](/docs/next/extractors/eluta) | Canadian listings aggregated from employer career sites (RSS) | Canada-only source (skipped when search geography is not Canada); RSS `location` strings must be set | `ELUTA_RSS_LOCATIONS`, `ELUTA_MAX_JOBS_PER_TERM` | Fetches one or more `eluta.ca` RSS feeds, filters by terms, de-duplicates by guid/URL |
|
||||
| [QAJobsBoard](/docs/next/extractors/qajobsboard) | QA / SDET / automation-heavy board (global JSON feed) | No auth; geography skew is manual/filter downstream | `qajobsboardMaxJobsPerTerm` | Fetches JobBoardly JSON, filters by pipeline terms |
|
||||
| [Arc.dev](/docs/next/extractors/arcdev) | Remote roles from Arc.dev listing pages (tool-tagged paths) | Parses SSR `__NEXT_DATA__`; relies on stable Next payload | `ARC_REMOTE_JOBS_PATHS` (seeds defaults), `arcRemoteJobsPaths`, `arcMaxJobsPerPath` | Merges Arc-managed + external rows; dedupes by URL |
|
||||
| [Google Jobs](/docs/next/extractors/google-jobs) | Broader open-web coverage via the Google for Jobs widget | Unofficial scrape (no API); CAPTCHA-prone from datacenter IPs; selectors can rot; slower (click-through per job) | `googleJobsMaxJobsPerTerm`, `GOOGLE_JOBS_PROXY`, `GOOGLE_JOBS_HEADLESS` | Browser scrape of `udm=8` Jobs vertical; scrolls list, double-clicks cards, expands description |
|
||||
| [Manual Import](/docs/next/extractors/manual) | One-off jobs not covered by scrapers | Inference quality depends on model/provider and input quality; some URLs cannot be fetched reliably | App/API endpoints (`/api/manual-jobs/infer`, `/api/manual-jobs/import`) | Accepts text/HTML/URL, runs inference, then saves and scores job after review |
|
||||
|
||||
## Which extractor should I use?
|
||||
@@ -40,6 +41,7 @@ Extractor integrations are now registered through manifests and loaded automatic
|
||||
- Use **BC T-Net** for British Columbia tech RSS listings (runs only when search geography is Canada).
|
||||
- Use **Eluta** for Canadian employer-direct listings via RSS (set metro/province `location` strings).
|
||||
- Use **QAJobsBoard** or **Arc.dev** when you want QA- or remote-stack-focused feeds without extra credentials.
|
||||
- Use **Google Jobs** when you want broader open-web coverage beyond dedicated job boards (unofficial scrape; best-effort).
|
||||
- Use **Manual Import** when you already have a specific posting and need direct import.
|
||||
|
||||
Many runs combine sources: broad discovery first, then manual import for high-priority jobs that scraping misses.
|
||||
@@ -82,5 +84,7 @@ JobOps ships **BC T-Net** and **iCIMS tenant HTML** extractors for two cases tha
|
||||
- [Arc.dev](/docs/next/extractors/arcdev)
|
||||
- [Canadian / NA QA contracting firms](/docs/next/extractors/qa-contract-staffing-canada)
|
||||
- [Canadian companies — QA-strong ATS](/docs/next/extractors/canadian-companies-qa-ats)
|
||||
- [Google Jobs](/docs/next/extractors/google-jobs)
|
||||
- [Himalayas](/docs/next/extractors/himalayas)
|
||||
- [Manual Import](/docs/next/extractors/manual)
|
||||
- [Add an Extractor](/docs/next/workflows/add-an-extractor)
|
||||
|
||||
@@ -45,11 +45,31 @@ During a pipeline run, if a new posting matches an existing row by URL, source i
|
||||
|
||||
Open jobs that match a prior skip or apply are **hidden** from Discovered, Ready, and All tabs so the queue stays fresh. Skipped and applied rows themselves remain visible in their statuses.
|
||||
|
||||
Use **Filters → Employer keywords → Hide roles you already skipped** to toggle this behavior (off by default). When enabled, JobOps compares **company + job title** only:
|
||||
|
||||
- Same company after normalization (legal suffixes stripped; short names like `CGI` match longer forms like `CGI IT UK Limited`).
|
||||
- Same title after normalization (including simple plural variants such as `Engineer` vs `Engineers`).
|
||||
- Prior skip or apply must be within the **last 90 days** (`updatedAt` on the skipped/applied row).
|
||||
|
||||
Uncheck the filter to temporarily review rediscovered rows that match older skips.
|
||||
|
||||
### Clean up rediscovered rows already in Discovered
|
||||
|
||||
If older pipeline runs left Discovered copies of roles you already skipped or applied:
|
||||
|
||||
1. Open **Settings → Danger Zone**.
|
||||
2. Use **Clear Rediscovered Archive Matches**.
|
||||
3. Confirm. JobOps permanently deletes matching **Discovered** rows only (Ready / Applied / Skipped stay).
|
||||
|
||||
Matching uses the same employer+title (and employer+description) keys as skip/import dedup.
|
||||
|
||||
## Defaults and constraints
|
||||
|
||||
- Description matching requires at least **80 characters** of normalized text; short or empty descriptions fall back to employer+title only.
|
||||
- Prior-skip hiding uses employer + title only (not description text).
|
||||
- The 90-day window applies to both the Jobs list filter and pipeline import suppression for skipped/applied rows.
|
||||
- Description matching during import still requires at least **80 characters** of normalized text; short or empty descriptions fall back to employer+title only.
|
||||
- Matching is **per profile** (`ownerProfileId`); different login profiles do not share dedup state.
|
||||
- Dedup does **not** delete existing rows retroactively when you change skip list or country filters — run discovery again or skip manually for old data.
|
||||
- Dedup does **not** delete existing rows retroactively when you change skip list or country filters — use **Clear Rediscovered Archive Matches** in Danger Zone, run discovery again, or skip manually for old data.
|
||||
- Very different titles at the same company (for example `SDET` vs `Product Designer`) are **not** collapsed.
|
||||
|
||||
## Common problems
|
||||
|
||||
@@ -153,6 +153,14 @@ Readiness requires:
|
||||
- Adzuna app ID/app key
|
||||
- Optional basic authentication for write operations
|
||||
|
||||
### Job search profiles and local resume
|
||||
|
||||
- **Search profiles** store target roles, deal-breakers, locations, and scoring context. Use the profile picker in Settings to create, switch, or activate a profile.
|
||||
- When **Basic Auth** is enabled, each login user maps to profiles whose `basicAuthUser` field lists that username. Comma-separated aliases are supported (for example `ilia,dobkin` accepts either login name).
|
||||
- Jobs are scoped per profile (`owner_profile_id`). Pipeline runs and the orchestrator board only show jobs for the profile tied to your current login.
|
||||
- **Local resume** JSON paths can be set per profile via `resumeLocalPath` in profile data (for example `../data/resumes/ilia-dobkin.json`). JobOps resolves that path relative to the orchestrator working directory when `JOBOPS_LOCAL_RESUME_PATH` is unset.
|
||||
- Activating a profile on Settings calls `POST /api/profiles/:id/activate` and updates `activeProfileId`, `jobSearchProfile`, and `localResumeProfilePath` when Basic Auth is off.
|
||||
|
||||
### Backup
|
||||
|
||||

|
||||
@@ -178,6 +186,8 @@ Readiness requires:
|
||||

|
||||
|
||||
- Clear jobs by selected statuses
|
||||
- Clear rediscovered archive matches (Discovered jobs that match skipped/applied roles)
|
||||
- Mark expired listings (probe Discovered URLs; mark 404/410 / “job expired” as expired)
|
||||
- Clear jobs below a score threshold
|
||||
- Clear the full database
|
||||
|
||||
@@ -260,6 +270,12 @@ curl -X POST "http://localhost:3001/api/backups"
|
||||
- Ensure `JOBOPS_PUBLIC_BASE_URL` is set for background/pipeline usage.
|
||||
- Ensure the configured host is publicly reachable and `/health` responds.
|
||||
|
||||
### Profile activate returns 403 or jobs list is empty after login
|
||||
|
||||
- Confirm your Basic Auth username appears in the profile's `basicAuthUser` list (comma-separated aliases are allowed).
|
||||
- If you previously ran pipelines before per-profile ownership existed, older jobs may still be under `__default__`. Re-run discovery while logged in, or migrate rows in `jobs.owner_profile_id` to your profile id.
|
||||
- Ensure the profile has `resumeLocalPath` pointing at an on-disk resume JSON if you rely on per-profile resumes without `JOBOPS_LOCAL_RESUME_PATH`.
|
||||
|
||||
## Related pages
|
||||
|
||||
- [Company skip list](./company-skip-list)
|
||||
|
||||
@@ -0,0 +1,189 @@
|
||||
/**
|
||||
* Built In — tech job board (JSON-LD ItemList embedded in SSR HTML).
|
||||
*
|
||||
* https://builtin.com/jobs
|
||||
*/
|
||||
|
||||
import type {
|
||||
ExtractorManifest,
|
||||
ExtractorRunResult,
|
||||
} from "@shared/types/extractors";
|
||||
import type { CreateJobInput } from "@shared/types/jobs";
|
||||
|
||||
const ORIGIN = "https://builtin.com";
|
||||
|
||||
interface BuiltinListItem {
|
||||
title: string;
|
||||
url: string;
|
||||
description?: string;
|
||||
}
|
||||
|
||||
function asString(value: unknown): string | undefined {
|
||||
if (typeof value !== "string") return undefined;
|
||||
const trimmed = value.trim();
|
||||
return trimmed ? trimmed : undefined;
|
||||
}
|
||||
|
||||
function decodeJsonString(value: string): string {
|
||||
return value
|
||||
.replace(/\\"/g, '"')
|
||||
.replace(/\\n/g, "\n")
|
||||
.replace(/\\t/g, "\t")
|
||||
.replace(/\\\\/g, "\\");
|
||||
}
|
||||
|
||||
function readMaxPages(raw: string | undefined): number {
|
||||
const parsed = raw ? Number.parseInt(raw, 10) : Number.NaN;
|
||||
if (!Number.isFinite(parsed)) return 3;
|
||||
return Math.min(Math.max(parsed, 1), 15);
|
||||
}
|
||||
|
||||
function parseListItems(html: string): BuiltinListItem[] {
|
||||
const items: BuiltinListItem[] = [];
|
||||
const pattern =
|
||||
/\{"@type":"ListItem","position":\d+,"name":"((?:\\.|[^"\\])*)","url":"(https:\/\/builtin\.com\/job\/[^"]+)"(?:,"description":"((?:\\.|[^"\\])*)")?\}/g;
|
||||
|
||||
for (const match of html.matchAll(pattern)) {
|
||||
const title = decodeJsonString(match[1] ?? "");
|
||||
const url = match[2];
|
||||
if (!title || !url) continue;
|
||||
items.push({
|
||||
title,
|
||||
url,
|
||||
description: match[3] ? decodeJsonString(match[3]) : undefined,
|
||||
});
|
||||
}
|
||||
|
||||
return items;
|
||||
}
|
||||
|
||||
function slugToWords(slug: string): string {
|
||||
return slug
|
||||
.split("-")
|
||||
.filter(Boolean)
|
||||
.map((part) => part.charAt(0).toUpperCase() + part.slice(1))
|
||||
.join(" ");
|
||||
}
|
||||
|
||||
function employerFromUrl(jobUrl: string): string {
|
||||
const match = jobUrl.match(/\/job\/[^/]+\/(\d+)/);
|
||||
if (!match) return "Unknown Employer";
|
||||
const slugMatch = jobUrl.match(/\/job\/([^/]+)\/\d+/);
|
||||
const slug = slugMatch?.[1] ?? "";
|
||||
const parts = slug.split("-");
|
||||
const maybeId = parts[parts.length - 1];
|
||||
if (/^\d+$/.test(maybeId ?? "")) parts.pop();
|
||||
const trimmed = parts.slice(-2);
|
||||
if (trimmed.length === 0) return slugToWords(slug);
|
||||
return slugToWords(trimmed.join("-"));
|
||||
}
|
||||
|
||||
function searchPath(term: string | null): string {
|
||||
if (!term) return "/jobs/remote";
|
||||
const query = encodeURIComponent(term.trim());
|
||||
return `/jobs/remote?search=${query}`;
|
||||
}
|
||||
|
||||
function matchesTerm(item: BuiltinListItem, term: string): boolean {
|
||||
const lower = term.toLowerCase();
|
||||
if (item.title.toLowerCase().includes(lower)) return true;
|
||||
if (item.description?.toLowerCase().includes(lower)) return true;
|
||||
return false;
|
||||
}
|
||||
|
||||
async function fetchPage(path: string, page: number): Promise<string> {
|
||||
const separator = path.includes("?") ? "&" : "?";
|
||||
const url =
|
||||
page <= 1 ? `${ORIGIN}${path}` : `${ORIGIN}${path}${separator}page=${page}`;
|
||||
const response = await fetch(url, {
|
||||
headers: { Accept: "text/html", "User-Agent": "JobOps/1.0" },
|
||||
});
|
||||
if (!response.ok) {
|
||||
throw new Error(`Built In request failed (${response.status}) for ${url}`);
|
||||
}
|
||||
return response.text();
|
||||
}
|
||||
|
||||
export const manifest: ExtractorManifest = {
|
||||
id: "builtin",
|
||||
displayName: "Built In",
|
||||
providesSources: ["builtin"],
|
||||
async run(context): Promise<ExtractorRunResult> {
|
||||
if (context.shouldCancel?.()) return { success: true, jobs: [] };
|
||||
|
||||
const maxJobs = context.settings.builtinMaxJobsPerTerm
|
||||
? Number.parseInt(context.settings.builtinMaxJobsPerTerm, 10)
|
||||
: 100;
|
||||
const cap = Number.isFinite(maxJobs)
|
||||
? Math.min(Math.max(maxJobs, 1), 500)
|
||||
: 100;
|
||||
const maxPages = readMaxPages(context.settings.builtinMaxPagesPerTerm);
|
||||
|
||||
const terms = context.searchTerms.length > 0 ? context.searchTerms : [null];
|
||||
|
||||
const seen = new Set<string>();
|
||||
const out: CreateJobInput[] = [];
|
||||
|
||||
try {
|
||||
for (let termIndex = 0; termIndex < terms.length; termIndex += 1) {
|
||||
if (context.shouldCancel?.()) break;
|
||||
const term = terms[termIndex];
|
||||
const path = searchPath(term);
|
||||
|
||||
for (let page = 1; page <= maxPages; page += 1) {
|
||||
if (context.shouldCancel?.()) break;
|
||||
if (out.length >= cap) break;
|
||||
|
||||
context.onProgress?.({
|
||||
phase: "list",
|
||||
termsProcessed: termIndex,
|
||||
termsTotal: terms.length,
|
||||
currentUrl: `${ORIGIN}${path}`,
|
||||
detail: `Built In: term ${termIndex + 1}/${terms.length}, page ${page}`,
|
||||
});
|
||||
|
||||
const html = await fetchPage(path, page);
|
||||
const items = parseListItems(html);
|
||||
if (items.length === 0) break;
|
||||
|
||||
for (const item of items) {
|
||||
if (out.length >= cap) break;
|
||||
if (term && !matchesTerm(item, term)) continue;
|
||||
|
||||
const key = item.url;
|
||||
if (seen.has(key)) continue;
|
||||
seen.add(key);
|
||||
|
||||
out.push({
|
||||
source: "builtin",
|
||||
sourceJobId: item.url.split("/").pop(),
|
||||
title: item.title,
|
||||
employer: employerFromUrl(item.url),
|
||||
jobUrl: item.url,
|
||||
applicationLink: item.url,
|
||||
location: "Remote",
|
||||
isRemote: true,
|
||||
jobDescription: item.description,
|
||||
});
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
context.onProgress?.({
|
||||
phase: "list",
|
||||
termsProcessed: terms.length,
|
||||
termsTotal: terms.length,
|
||||
currentUrl: `${ORIGIN}/jobs`,
|
||||
jobPagesProcessed: out.length,
|
||||
detail: `Built In: ${out.length} jobs`,
|
||||
});
|
||||
|
||||
return { success: true, jobs: out };
|
||||
} catch (error) {
|
||||
const message = error instanceof Error ? error.message : "Unknown error";
|
||||
return { success: false, jobs: out, error: message };
|
||||
}
|
||||
},
|
||||
};
|
||||
|
||||
export default manifest;
|
||||
@@ -0,0 +1,17 @@
|
||||
{
|
||||
"name": "builtin-extractor",
|
||||
"version": "0.0.1",
|
||||
"type": "module",
|
||||
"description": "Built In tech job board extractor",
|
||||
"main": "manifest.ts",
|
||||
"dependencies": {
|
||||
"job-ops-shared": "^1.0.0"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@types/node": "^24.0.0",
|
||||
"typescript": "~5.9.0"
|
||||
},
|
||||
"scripts": {
|
||||
"check:types": "tsc --noEmit"
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,17 @@
|
||||
{
|
||||
"compilerOptions": {
|
||||
"module": "ESNext",
|
||||
"moduleResolution": "bundler",
|
||||
"target": "ES2022",
|
||||
"outDir": "dist",
|
||||
"strict": true,
|
||||
"noUnusedLocals": false,
|
||||
"lib": ["ES2022", "DOM"],
|
||||
"types": ["node"],
|
||||
"baseUrl": ".",
|
||||
"paths": {
|
||||
"@shared/*": ["../../shared/src/*"]
|
||||
}
|
||||
},
|
||||
"include": ["./manifest.ts", "./src/**/*"]
|
||||
}
|
||||
@@ -0,0 +1,89 @@
|
||||
/**
|
||||
* Direct company career pages — auto-detect ATS backends and pull public job feeds.
|
||||
*/
|
||||
|
||||
import type {
|
||||
ExtractorManifest,
|
||||
ExtractorRunResult,
|
||||
} from "@shared/types/extractors";
|
||||
import type { CreateJobInput } from "@shared/types/jobs";
|
||||
import {
|
||||
detectAtsTargets,
|
||||
fetchCareersPage,
|
||||
fetchJobsForTarget,
|
||||
readUrlList,
|
||||
} from "./src/resolve-ats.js";
|
||||
|
||||
function matchesTerm(job: CreateJobInput, term: string): boolean {
|
||||
const lower = term.toLowerCase();
|
||||
if (job.title.toLowerCase().includes(lower)) return true;
|
||||
if (job.employer.toLowerCase().includes(lower)) return true;
|
||||
if (job.jobDescription?.toLowerCase().includes(lower)) return true;
|
||||
return false;
|
||||
}
|
||||
|
||||
export const manifest: ExtractorManifest = {
|
||||
id: "careerspages",
|
||||
displayName: "Career Pages",
|
||||
providesSources: ["careerspages"],
|
||||
async run(context): Promise<ExtractorRunResult> {
|
||||
if (context.shouldCancel?.()) return { success: true, jobs: [] };
|
||||
|
||||
const pageUrls = readUrlList(context.settings.careersPageUrls);
|
||||
if (pageUrls.length === 0) {
|
||||
return {
|
||||
success: true,
|
||||
jobs: [],
|
||||
error:
|
||||
"No career page URLs configured. Set CAREERS_PAGE_URLS or careersPageUrls (JSON array or comma/newline-separated URLs).",
|
||||
};
|
||||
}
|
||||
|
||||
const terms = context.searchTerms.length > 0 ? context.searchTerms : [];
|
||||
const seen = new Set<string>();
|
||||
const out: CreateJobInput[] = [];
|
||||
|
||||
try {
|
||||
for (let i = 0; i < pageUrls.length; i += 1) {
|
||||
if (context.shouldCancel?.()) break;
|
||||
const pageUrl = pageUrls[i];
|
||||
|
||||
context.onProgress?.({
|
||||
phase: "list",
|
||||
termsProcessed: i,
|
||||
termsTotal: pageUrls.length,
|
||||
currentUrl: pageUrl,
|
||||
detail: `Career pages: resolving ATS for ${pageUrl}`,
|
||||
});
|
||||
|
||||
const html = await fetchCareersPage(pageUrl);
|
||||
const targets = detectAtsTargets(html, pageUrl);
|
||||
if (targets.length === 0) continue;
|
||||
|
||||
for (const target of targets) {
|
||||
if (context.shouldCancel?.()) break;
|
||||
const jobs = await fetchJobsForTarget(target);
|
||||
for (const job of jobs) {
|
||||
if (
|
||||
terms.length > 0 &&
|
||||
!terms.some((term) => matchesTerm(job, term))
|
||||
) {
|
||||
continue;
|
||||
}
|
||||
const key = job.jobUrl;
|
||||
if (seen.has(key)) continue;
|
||||
seen.add(key);
|
||||
out.push(job);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return { success: true, jobs: out };
|
||||
} catch (error) {
|
||||
const message = error instanceof Error ? error.message : "Unknown error";
|
||||
return { success: false, jobs: out, error: message };
|
||||
}
|
||||
},
|
||||
};
|
||||
|
||||
export default manifest;
|
||||
@@ -0,0 +1,17 @@
|
||||
{
|
||||
"name": "careerspages-extractor",
|
||||
"version": "0.0.1",
|
||||
"type": "module",
|
||||
"description": "Direct company career page ATS resolver extractor",
|
||||
"main": "manifest.ts",
|
||||
"dependencies": {
|
||||
"job-ops-shared": "^1.0.0"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@types/node": "^24.0.0",
|
||||
"typescript": "~5.9.0"
|
||||
},
|
||||
"scripts": {
|
||||
"check:types": "tsc --noEmit"
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,345 @@
|
||||
import type { CreateJobInput } from "@shared/types/jobs";
|
||||
|
||||
export type AtsKind =
|
||||
| "ashby"
|
||||
| "greenhouse"
|
||||
| "lever"
|
||||
| "teamtailor"
|
||||
| "huntflow"
|
||||
| "factorial";
|
||||
|
||||
export interface DetectedAts {
|
||||
kind: AtsKind;
|
||||
slug: string;
|
||||
}
|
||||
|
||||
export function readUrlList(raw: string | undefined): string[] {
|
||||
if (!raw) return [];
|
||||
try {
|
||||
const parsed = JSON.parse(raw);
|
||||
if (Array.isArray(parsed)) {
|
||||
return parsed
|
||||
.map((entry) => (typeof entry === "string" ? entry.trim() : ""))
|
||||
.filter(Boolean);
|
||||
}
|
||||
} catch {
|
||||
// fall through
|
||||
}
|
||||
return raw
|
||||
.split(/[\n,;|]+/)
|
||||
.map((entry) => entry.trim())
|
||||
.filter(Boolean);
|
||||
}
|
||||
|
||||
export function detectAtsTargets(html: string, pageUrl: string): DetectedAts[] {
|
||||
const found: DetectedAts[] = [];
|
||||
const seen = new Set<string>();
|
||||
|
||||
const add = (kind: AtsKind, slug: string) => {
|
||||
const key = `${kind}:${slug.toLowerCase()}`;
|
||||
if (!slug || seen.has(key)) return;
|
||||
seen.add(key);
|
||||
found.push({ kind, slug });
|
||||
};
|
||||
|
||||
for (const match of html.matchAll(/jobs\.ashbyhq\.com\/([a-zA-Z0-9_-]+)/g)) {
|
||||
add("ashby", match[1]);
|
||||
}
|
||||
for (const match of html.matchAll(
|
||||
/boards-api\.greenhouse\.io\/v1\/boards\/([a-zA-Z0-9_-]+)/g,
|
||||
)) {
|
||||
add("greenhouse", match[1]);
|
||||
}
|
||||
for (const match of html.matchAll(
|
||||
/boards\.greenhouse\.io\/([a-zA-Z0-9_-]+)/g,
|
||||
)) {
|
||||
add("greenhouse", match[1]);
|
||||
}
|
||||
for (const match of html.matchAll(/jobs\.lever\.co\/([a-zA-Z0-9_-]+)/g)) {
|
||||
add("lever", match[1]);
|
||||
}
|
||||
for (const match of html.matchAll(
|
||||
/https?:\/\/([a-z0-9-]+)\.teamtailor\.com/gi,
|
||||
)) {
|
||||
add("teamtailor", match[1]);
|
||||
}
|
||||
for (const match of html.matchAll(
|
||||
/https?:\/\/([a-z0-9-]+)\.huntflow\.io/gi,
|
||||
)) {
|
||||
add("huntflow", match[1]);
|
||||
}
|
||||
for (const match of html.matchAll(
|
||||
/https?:\/\/([a-z0-9-]+)\.factorialhr\.com/gi,
|
||||
)) {
|
||||
add("factorial", match[1]);
|
||||
}
|
||||
|
||||
try {
|
||||
const host = new URL(pageUrl).hostname.toLowerCase();
|
||||
if (host.endsWith(".huntflow.io")) {
|
||||
add("huntflow", host.replace(/\.huntflow\.io$/, ""));
|
||||
}
|
||||
if (host.endsWith(".factorialhr.com")) {
|
||||
add("factorial", host.replace(/\.factorialhr\.com$/, ""));
|
||||
}
|
||||
if (host.endsWith(".teamtailor.com")) {
|
||||
add("teamtailor", host.replace(/\.teamtailor\.com$/, ""));
|
||||
}
|
||||
} catch {
|
||||
// ignore invalid pageUrl
|
||||
}
|
||||
|
||||
return found;
|
||||
}
|
||||
|
||||
function asString(value: unknown): string | undefined {
|
||||
if (typeof value !== "string") return undefined;
|
||||
const trimmed = value.trim();
|
||||
return trimmed ? trimmed : undefined;
|
||||
}
|
||||
|
||||
function decodeHtmlEntities(value: string): string {
|
||||
return value
|
||||
.replace(/&/g, "&")
|
||||
.replace(/</g, "<")
|
||||
.replace(/>/g, ">")
|
||||
.replace(/"/g, '"')
|
||||
.replace(/'/g, "'");
|
||||
}
|
||||
|
||||
function stripHtml(html: string): string {
|
||||
return decodeHtmlEntities(html)
|
||||
.replace(/<[^>]+>/g, " ")
|
||||
.replace(/\s+/g, " ")
|
||||
.trim();
|
||||
}
|
||||
|
||||
async function fetchAshby(slug: string): Promise<CreateJobInput[]> {
|
||||
const url = `https://api.ashbyhq.com/posting-api/job-board/${encodeURIComponent(slug)}`;
|
||||
const response = await fetch(url, {
|
||||
headers: { Accept: "application/json" },
|
||||
});
|
||||
if (!response.ok) return [];
|
||||
const body = (await response.json()) as {
|
||||
jobs?: Array<Record<string, unknown>>;
|
||||
};
|
||||
return (body.jobs ?? []).flatMap((job) => {
|
||||
const jobUrl = asString(job.jobUrl);
|
||||
if (!jobUrl) return [];
|
||||
return [
|
||||
{
|
||||
source: "careerspages" as const,
|
||||
sourceJobId: asString(job.id),
|
||||
title: asString(job.title) ?? "Unknown Title",
|
||||
employer: slug,
|
||||
jobUrl,
|
||||
applicationLink: asString(job.applyUrl) ?? jobUrl,
|
||||
location:
|
||||
asString(job.location) ?? asString(job.locationName) ?? "Unknown",
|
||||
isRemote: job.isRemote === true,
|
||||
datePosted: asString(job.publishedAt),
|
||||
jobDescription:
|
||||
asString(job.descriptionPlain) ??
|
||||
(job.descriptionHtml
|
||||
? stripHtml(String(job.descriptionHtml))
|
||||
: undefined),
|
||||
},
|
||||
];
|
||||
});
|
||||
}
|
||||
|
||||
async function fetchGreenhouse(slug: string): Promise<CreateJobInput[]> {
|
||||
const url = `https://boards-api.greenhouse.io/v1/boards/${encodeURIComponent(slug)}/jobs?content=true`;
|
||||
const response = await fetch(url, {
|
||||
headers: { Accept: "application/json" },
|
||||
});
|
||||
if (!response.ok) return [];
|
||||
const body = (await response.json()) as {
|
||||
jobs?: Array<Record<string, unknown>>;
|
||||
};
|
||||
return (body.jobs ?? []).flatMap((job) => {
|
||||
const jobUrl = asString(job.absolute_url);
|
||||
if (!jobUrl) return [];
|
||||
const location = job.location as { name?: string } | undefined;
|
||||
return [
|
||||
{
|
||||
source: "careerspages" as const,
|
||||
sourceJobId: String(job.id ?? jobUrl),
|
||||
title: asString(job.title) ?? "Unknown Title",
|
||||
employer: slug,
|
||||
jobUrl,
|
||||
applicationLink: jobUrl,
|
||||
location: asString(location?.name) ?? "Unknown",
|
||||
datePosted: asString(job.updated_at),
|
||||
jobDescription: job.content
|
||||
? stripHtml(String(job.content))
|
||||
: undefined,
|
||||
},
|
||||
];
|
||||
});
|
||||
}
|
||||
|
||||
async function fetchLever(slug: string): Promise<CreateJobInput[]> {
|
||||
const url = `https://api.lever.co/v0/postings/${encodeURIComponent(slug)}?mode=json`;
|
||||
const response = await fetch(url, {
|
||||
headers: { Accept: "application/json" },
|
||||
});
|
||||
if (!response.ok) return [];
|
||||
const body = (await response.json()) as Array<Record<string, unknown>>;
|
||||
if (!Array.isArray(body)) return [];
|
||||
return body.flatMap((job) => {
|
||||
const jobUrl = asString(job.hostedUrl) ?? asString(job.applyUrl);
|
||||
if (!jobUrl) return [];
|
||||
const categories = job.categories as Record<string, unknown> | undefined;
|
||||
return [
|
||||
{
|
||||
source: "careerspages" as const,
|
||||
sourceJobId: asString(job.id),
|
||||
title: asString(job.text) ?? "Unknown Title",
|
||||
employer: slug,
|
||||
jobUrl,
|
||||
applicationLink: asString(job.applyUrl) ?? jobUrl,
|
||||
location: asString(categories?.location) ?? "Unknown",
|
||||
datePosted: asString(job.createdAt),
|
||||
jobDescription: job.descriptionPlain
|
||||
? String(job.descriptionPlain)
|
||||
: job.description
|
||||
? stripHtml(String(job.description))
|
||||
: undefined,
|
||||
},
|
||||
];
|
||||
});
|
||||
}
|
||||
|
||||
function xmlText(xml: string, tag: string): string | undefined {
|
||||
const pattern = new RegExp(`<${tag}[^>]*>([\\s\\S]*?)</${tag}>`, "i");
|
||||
const match = xml.match(pattern);
|
||||
if (!match?.[1]) return undefined;
|
||||
return (
|
||||
match[1].replace(/<!\[CDATA\[([\s\S]*?)\]\]>/g, "$1").trim() || undefined
|
||||
);
|
||||
}
|
||||
|
||||
async function fetchTeamtailor(slug: string): Promise<CreateJobInput[]> {
|
||||
const url = `https://${encodeURIComponent(slug)}.teamtailor.com/jobs.rss?per_page=200`;
|
||||
const response = await fetch(url, {
|
||||
headers: { Accept: "application/rss+xml, application/xml, text/xml" },
|
||||
});
|
||||
if (!response.ok) return [];
|
||||
const xml = await response.text();
|
||||
const out: CreateJobInput[] = [];
|
||||
for (const raw of xml.match(/<item>([\s\S]*?)<\/item>/gi) ?? []) {
|
||||
const block = raw.replace(/^<item>/i, "").replace(/<\/item>$/i, "");
|
||||
const jobUrl = xmlText(block, "link");
|
||||
if (!jobUrl) continue;
|
||||
out.push({
|
||||
source: "careerspages",
|
||||
sourceJobId: xmlText(block, "guid") ?? jobUrl,
|
||||
title: xmlText(block, "title") ?? "Unknown Title",
|
||||
employer: xmlText(block, "company_name") ?? slug,
|
||||
jobUrl,
|
||||
applicationLink: jobUrl,
|
||||
location: xmlText(block, "location") ?? "Unknown",
|
||||
datePosted: xmlText(block, "pubDate"),
|
||||
jobDescription: xmlText(block, "description")
|
||||
? stripHtml(xmlText(block, "description") ?? "")
|
||||
: undefined,
|
||||
});
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
async function fetchHuntflow(slug: string): Promise<CreateJobInput[]> {
|
||||
const origin = `https://${slug}.huntflow.io`;
|
||||
const response = await fetch(`${origin}/`, {
|
||||
headers: { Accept: "text/html", "User-Agent": "JobOps/1.0" },
|
||||
});
|
||||
if (!response.ok) return [];
|
||||
const html = await response.text();
|
||||
const out: CreateJobInput[] = [];
|
||||
const pattern =
|
||||
/<article class="_item_[^"]*">[\s\S]*?<a href="(\/vacancy\/[^"]+)"[^>]*>([^<]+)<\/a>[\s\S]*?<div class="_info_[^"]*">([^<]*)<\/div>/g;
|
||||
for (const match of html.matchAll(pattern)) {
|
||||
const jobUrl = `${origin}${match[1]}`;
|
||||
out.push({
|
||||
source: "careerspages",
|
||||
sourceJobId: match[1]?.replace(/^\/vacancy\//, ""),
|
||||
title: decodeHtmlEntities(match[2]?.trim() ?? "Unknown Title"),
|
||||
employer: slug,
|
||||
jobUrl,
|
||||
applicationLink: jobUrl,
|
||||
location: match[3]?.trim() || "Unknown",
|
||||
isRemote: match[3]?.toLowerCase().includes("remote"),
|
||||
});
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
async function fetchFactorial(slug: string): Promise<CreateJobInput[]> {
|
||||
const origin = `https://${slug}.factorialhr.com`;
|
||||
const response = await fetch(origin, {
|
||||
headers: { Accept: "text/html", "User-Agent": "JobOps/1.0" },
|
||||
});
|
||||
if (!response.ok) return [];
|
||||
const html = await response.text();
|
||||
const out: CreateJobInput[] = [];
|
||||
for (const match of html.matchAll(
|
||||
/(?:href|data-job-postings-url)=['"]([^'"]*\/job_posting\/[^'"]+)['"]/g,
|
||||
)) {
|
||||
const raw = match[1];
|
||||
const path = raw.startsWith("http")
|
||||
? raw.replace(origin, "")
|
||||
: raw.startsWith("/")
|
||||
? raw
|
||||
: `/${raw}`;
|
||||
const jobUrl = `${origin}${path}`;
|
||||
const slugPart = path.split("/").pop() ?? "";
|
||||
const title = slugPart
|
||||
.replace(/-\d+$/, "")
|
||||
.split("-")
|
||||
.map((part) => part.charAt(0).toUpperCase() + part.slice(1))
|
||||
.join(" ");
|
||||
out.push({
|
||||
source: "careerspages",
|
||||
sourceJobId: slugPart,
|
||||
title,
|
||||
employer: slug,
|
||||
jobUrl,
|
||||
applicationLink: jobUrl,
|
||||
location: "Unknown",
|
||||
});
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
export async function fetchJobsForTarget(
|
||||
target: DetectedAts,
|
||||
): Promise<CreateJobInput[]> {
|
||||
switch (target.kind) {
|
||||
case "ashby":
|
||||
return fetchAshby(target.slug);
|
||||
case "greenhouse":
|
||||
return fetchGreenhouse(target.slug);
|
||||
case "lever":
|
||||
return fetchLever(target.slug);
|
||||
case "teamtailor":
|
||||
return fetchTeamtailor(target.slug);
|
||||
case "huntflow":
|
||||
return fetchHuntflow(target.slug);
|
||||
case "factorial":
|
||||
return fetchFactorial(target.slug);
|
||||
}
|
||||
}
|
||||
|
||||
export async function fetchCareersPage(pageUrl: string): Promise<string> {
|
||||
const response = await fetch(pageUrl, {
|
||||
headers: { Accept: "text/html", "User-Agent": "JobOps/1.0" },
|
||||
redirect: "follow",
|
||||
});
|
||||
if (!response.ok) {
|
||||
throw new Error(
|
||||
`Careers page request failed (${response.status}) for ${pageUrl}`,
|
||||
);
|
||||
}
|
||||
return response.text();
|
||||
}
|
||||
@@ -0,0 +1,17 @@
|
||||
{
|
||||
"compilerOptions": {
|
||||
"module": "ESNext",
|
||||
"moduleResolution": "bundler",
|
||||
"target": "ES2022",
|
||||
"outDir": "dist",
|
||||
"strict": true,
|
||||
"noUnusedLocals": false,
|
||||
"lib": ["ES2022", "DOM"],
|
||||
"types": ["node"],
|
||||
"baseUrl": ".",
|
||||
"paths": {
|
||||
"@shared/*": ["../../shared/src/*"]
|
||||
}
|
||||
},
|
||||
"include": ["./manifest.ts", "./src/**/*"]
|
||||
}
|
||||
@@ -0,0 +1,216 @@
|
||||
/**
|
||||
* Factorial HR public career sites — tenant HTML job_posting pages.
|
||||
*
|
||||
* https://{tenant}.factorialhr.com/
|
||||
*/
|
||||
|
||||
import type {
|
||||
ExtractorManifest,
|
||||
ExtractorRunResult,
|
||||
} from "@shared/types/extractors";
|
||||
import type { CreateJobInput } from "@shared/types/jobs";
|
||||
|
||||
function asString(value: unknown): string | undefined {
|
||||
if (typeof value !== "string") return undefined;
|
||||
const trimmed = value.trim();
|
||||
return trimmed ? trimmed : undefined;
|
||||
}
|
||||
|
||||
function readTenants(raw: string | undefined): string[] {
|
||||
if (!raw) return [];
|
||||
try {
|
||||
const parsed = JSON.parse(raw);
|
||||
if (Array.isArray(parsed)) {
|
||||
return parsed
|
||||
.map((entry) => (typeof entry === "string" ? entry.trim() : ""))
|
||||
.filter(Boolean);
|
||||
}
|
||||
} catch {
|
||||
// fall through
|
||||
}
|
||||
return raw
|
||||
.split(/[\n,;|]+/)
|
||||
.map((entry) => entry.trim())
|
||||
.filter(Boolean);
|
||||
}
|
||||
|
||||
function decodeHtmlEntities(value: string): string {
|
||||
return value
|
||||
.replace(/&/g, "&")
|
||||
.replace(/</g, "<")
|
||||
.replace(/>/g, ">")
|
||||
.replace(/"/g, '"')
|
||||
.replace(/'/g, "'");
|
||||
}
|
||||
|
||||
function stripHtml(html: string): string {
|
||||
return decodeHtmlEntities(html)
|
||||
.replace(/<[^>]+>/g, " ")
|
||||
.replace(/\s+/g, " ")
|
||||
.trim();
|
||||
}
|
||||
|
||||
function parseJobLinks(html: string, origin: string): string[] {
|
||||
const links = new Set<string>();
|
||||
for (const match of html.matchAll(
|
||||
/(?:href|data-job-postings-url)=['"]([^'"]*\/job_posting\/[^'"]+)['"]/g,
|
||||
)) {
|
||||
const raw = match[1];
|
||||
const path = raw.startsWith("http")
|
||||
? raw.replace(origin, "")
|
||||
: raw.startsWith("/")
|
||||
? raw
|
||||
: `/${raw}`;
|
||||
links.add(path);
|
||||
}
|
||||
return [...links];
|
||||
}
|
||||
|
||||
function parseJobDetail(html: string): {
|
||||
title?: string;
|
||||
description?: string;
|
||||
} {
|
||||
const titleMatch =
|
||||
html.match(/property='og:title'[^>]*content='([^']+)'/) ??
|
||||
html.match(/<h1[^>]*>([^<]+)<\/h1>/);
|
||||
const bodyMatch = html.match(
|
||||
/<div class='mb-12'>[\s\S]*?<div class='mb-2 sm:mb-4'>[\s\S]*?<\/h1>([\s\S]*?)<\/div>\s*<\/div>/,
|
||||
);
|
||||
|
||||
return {
|
||||
title: titleMatch?.[1]?.trim(),
|
||||
description: bodyMatch?.[1] ? stripHtml(bodyMatch[1]) : undefined,
|
||||
};
|
||||
}
|
||||
|
||||
function matchesTerm(values: string[], term: string): boolean {
|
||||
const lower = term.toLowerCase();
|
||||
return values.some((value) => value.toLowerCase().includes(lower));
|
||||
}
|
||||
|
||||
async function fetchText(url: string): Promise<string> {
|
||||
const response = await fetch(url, {
|
||||
headers: { Accept: "text/html", "User-Agent": "JobOps/1.0" },
|
||||
});
|
||||
if (!response.ok) {
|
||||
throw new Error(`Factorial request failed (${response.status}) for ${url}`);
|
||||
}
|
||||
return response.text();
|
||||
}
|
||||
|
||||
function tenantOrigin(tenant: string): string {
|
||||
const host = tenant.includes(".") ? tenant : `${tenant}.factorialhr.com`;
|
||||
return host.startsWith("http") ? host : `https://${host}`;
|
||||
}
|
||||
|
||||
function employerLabel(tenant: string, pageHtml: string): string {
|
||||
const ogSiteMatch = pageHtml.match(
|
||||
/property=['"]og:site_name['"][^>]*content=['"]([^'"]+)['"]/i,
|
||||
);
|
||||
if (ogSiteMatch?.[1]) {
|
||||
const cleaned = ogSiteMatch[1].split(/\s[-|–]\s/)[0]?.trim();
|
||||
if (cleaned) return cleaned;
|
||||
}
|
||||
|
||||
return tenant
|
||||
.split(".")[0]
|
||||
.split(/[-_]/)
|
||||
.map((part) => part.charAt(0).toUpperCase() + part.slice(1))
|
||||
.join(" ");
|
||||
}
|
||||
|
||||
export const manifest: ExtractorManifest = {
|
||||
id: "factorial",
|
||||
displayName: "Factorial (ATS)",
|
||||
providesSources: ["factorial"],
|
||||
async run(context): Promise<ExtractorRunResult> {
|
||||
if (context.shouldCancel?.()) return { success: true, jobs: [] };
|
||||
|
||||
const tenants = readTenants(context.settings.factorialTenants);
|
||||
if (tenants.length === 0) {
|
||||
return {
|
||||
success: true,
|
||||
jobs: [],
|
||||
error:
|
||||
"No Factorial tenants configured. Set FACTORIAL_TENANTS or factorialTenants (comma- or newline-separated subdomains, e.g. yourbourse).",
|
||||
};
|
||||
}
|
||||
|
||||
const terms = context.searchTerms.length > 0 ? context.searchTerms : [];
|
||||
const seen = new Set<string>();
|
||||
const out: CreateJobInput[] = [];
|
||||
|
||||
try {
|
||||
for (let i = 0; i < tenants.length; i += 1) {
|
||||
if (context.shouldCancel?.()) break;
|
||||
const tenant = tenants[i];
|
||||
const origin = tenantOrigin(tenant);
|
||||
|
||||
context.onProgress?.({
|
||||
phase: "list",
|
||||
termsProcessed: i,
|
||||
termsTotal: tenants.length,
|
||||
currentUrl: origin,
|
||||
detail: `Factorial: ${tenant} (${i + 1}/${tenants.length})`,
|
||||
});
|
||||
|
||||
const indexHtml = await fetchText(origin);
|
||||
const employer = employerLabel(tenant, indexHtml);
|
||||
const links = parseJobLinks(indexHtml, origin);
|
||||
|
||||
for (const link of links) {
|
||||
if (context.shouldCancel?.()) break;
|
||||
|
||||
const jobUrl = `${origin}${link}`;
|
||||
let title =
|
||||
link.split("/").pop()?.replace(/-\d+$/, "").replace(/-/g, " ") ??
|
||||
"Unknown Title";
|
||||
title = title
|
||||
.split(" ")
|
||||
.map((part) => part.charAt(0).toUpperCase() + part.slice(1))
|
||||
.join(" ");
|
||||
let jobDescription: string | undefined;
|
||||
|
||||
try {
|
||||
const detailHtml = await fetchText(jobUrl);
|
||||
const detail = parseJobDetail(detailHtml);
|
||||
if (detail.title) title = detail.title.replace(/\s+$/, "");
|
||||
if (detail.description) jobDescription = detail.description;
|
||||
} catch {
|
||||
// keep index-derived row when detail fetch fails
|
||||
}
|
||||
|
||||
const haystack = [title, jobDescription ?? ""];
|
||||
if (
|
||||
terms.length > 0 &&
|
||||
!terms.some((term) => matchesTerm(haystack, term))
|
||||
) {
|
||||
continue;
|
||||
}
|
||||
|
||||
const key = jobUrl;
|
||||
if (seen.has(key)) continue;
|
||||
seen.add(key);
|
||||
|
||||
out.push({
|
||||
source: "factorial",
|
||||
sourceJobId: link.split("/").pop(),
|
||||
title,
|
||||
employer,
|
||||
jobUrl,
|
||||
applicationLink: jobUrl,
|
||||
location: "Unknown",
|
||||
jobDescription,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
return { success: true, jobs: out };
|
||||
} catch (error) {
|
||||
const message = error instanceof Error ? error.message : "Unknown error";
|
||||
return { success: false, jobs: out, error: message };
|
||||
}
|
||||
},
|
||||
};
|
||||
|
||||
export default manifest;
|
||||
@@ -0,0 +1,17 @@
|
||||
{
|
||||
"name": "factorial-extractor",
|
||||
"version": "0.0.1",
|
||||
"type": "module",
|
||||
"description": "Factorial HR public career-site extractor",
|
||||
"main": "manifest.ts",
|
||||
"dependencies": {
|
||||
"job-ops-shared": "^1.0.0"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@types/node": "^24.0.0",
|
||||
"typescript": "~5.9.0"
|
||||
},
|
||||
"scripts": {
|
||||
"check:types": "tsc --noEmit"
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,17 @@
|
||||
{
|
||||
"compilerOptions": {
|
||||
"module": "ESNext",
|
||||
"moduleResolution": "bundler",
|
||||
"target": "ES2022",
|
||||
"outDir": "dist",
|
||||
"strict": true,
|
||||
"noUnusedLocals": false,
|
||||
"lib": ["ES2022", "DOM"],
|
||||
"types": ["node"],
|
||||
"baseUrl": ".",
|
||||
"paths": {
|
||||
"@shared/*": ["../../shared/src/*"]
|
||||
}
|
||||
},
|
||||
"include": ["./manifest.ts", "./src/**/*"]
|
||||
}
|
||||
@@ -0,0 +1,49 @@
|
||||
import type {
|
||||
ExtractorManifest,
|
||||
ExtractorRunResult,
|
||||
} from "@shared/types/extractors";
|
||||
import { runGoogleJobs } from "./src/run.js";
|
||||
|
||||
export const manifest: ExtractorManifest = {
|
||||
id: "google-jobs",
|
||||
displayName: "Google Jobs",
|
||||
providesSources: ["google-jobs"],
|
||||
async run(context): Promise<ExtractorRunResult> {
|
||||
if (context.shouldCancel?.()) return { success: true, jobs: [] };
|
||||
|
||||
const parsedMax = context.settings.googleJobsMaxJobsPerTerm
|
||||
? Number.parseInt(context.settings.googleJobsMaxJobsPerTerm, 10)
|
||||
: Number.NaN;
|
||||
const maxJobs = Number.isFinite(parsedMax) ? Math.max(1, parsedMax) : 30;
|
||||
|
||||
context.onProgress?.({
|
||||
phase: "list",
|
||||
termsProcessed: 0,
|
||||
termsTotal: context.searchTerms.length || 1,
|
||||
currentUrl: "https://www.google.com/search?udm=8",
|
||||
detail: "Google Jobs: launching browser scrape",
|
||||
});
|
||||
|
||||
const result = await runGoogleJobs({
|
||||
searchTerms: context.searchTerms,
|
||||
maxJobs,
|
||||
});
|
||||
|
||||
if (!result.success) {
|
||||
return { success: false, jobs: [], error: result.error };
|
||||
}
|
||||
|
||||
context.onProgress?.({
|
||||
phase: "list",
|
||||
termsProcessed: context.searchTerms.length || 1,
|
||||
termsTotal: context.searchTerms.length || 1,
|
||||
currentUrl: "https://www.google.com/search?udm=8",
|
||||
jobPagesProcessed: result.jobs.length,
|
||||
detail: `Google Jobs: ${result.jobs.length} jobs`,
|
||||
});
|
||||
|
||||
return { success: true, jobs: result.jobs };
|
||||
},
|
||||
};
|
||||
|
||||
export default manifest;
|
||||
@@ -0,0 +1,22 @@
|
||||
{
|
||||
"name": "google-jobs-extractor",
|
||||
"version": "0.0.1",
|
||||
"type": "module",
|
||||
"description": "Google Jobs (Google for Jobs) search widget extractor (browser-backed)",
|
||||
"main": "manifest.ts",
|
||||
"dependencies": {
|
||||
"camoufox-js": "^0.8.0",
|
||||
"job-ops-shared": "^1.0.0",
|
||||
"playwright": "^1.57.0",
|
||||
"tsx": "^4.4.0"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@types/node": "^24.0.0",
|
||||
"typescript": "~5.9.0"
|
||||
},
|
||||
"scripts": {
|
||||
"start": "tsx src/main.ts",
|
||||
"check:types": "tsc --noEmit",
|
||||
"get-binaries": "camoufox-js fetch"
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,616 @@
|
||||
import { access, mkdir, writeFile } from "node:fs/promises";
|
||||
import { dirname, join } from "node:path";
|
||||
import { fileURLToPath } from "node:url";
|
||||
import { launchOptions } from "camoufox-js";
|
||||
import {
|
||||
type Browser,
|
||||
type BrowserContext,
|
||||
firefox,
|
||||
type Page,
|
||||
} from "playwright";
|
||||
|
||||
const __dirname = dirname(fileURLToPath(import.meta.url));
|
||||
const OUTPUT_PATH = join(__dirname, "../storage/jobs.json");
|
||||
const STATE_PATH = join(__dirname, "../storage/state.json");
|
||||
const DEBUG_HTML_PATH = join(__dirname, "../storage/debug-last.html");
|
||||
const DEBUG_SCREENSHOT_PATH = join(__dirname, "../storage/debug-last.png");
|
||||
|
||||
interface ScrapedJob {
|
||||
title: string;
|
||||
employer: string;
|
||||
jobUrl: string;
|
||||
applicationLink?: string;
|
||||
location?: string;
|
||||
jobDescription?: string;
|
||||
postedVia?: string;
|
||||
isRemote?: boolean;
|
||||
}
|
||||
|
||||
interface ScrapedCard {
|
||||
docId?: string;
|
||||
title: string;
|
||||
employer: string;
|
||||
location?: string;
|
||||
via?: string;
|
||||
/** Google's own permalink into this specific listing (from `data-share-url`). Always present when the card matched, so it works as a stable jobUrl even if the detail click fails. */
|
||||
shareUrl?: string;
|
||||
}
|
||||
|
||||
interface ScrapedDetail {
|
||||
applyLinks: Array<{ href: string; text: string }>;
|
||||
description?: string;
|
||||
}
|
||||
|
||||
/**
|
||||
* Confirmed against Google's live markup (2026). Google renders each job as
|
||||
* a `.EimVGf` card with a `data-share-url` permalink baked in — no `<ul>`/
|
||||
* `<li>`/`aria-label="Jobs list"` wrapper as older scraping guides assumed.
|
||||
* Kept as a short list (not a single hardcoded class) since Google rotates
|
||||
* these obfuscated names periodically.
|
||||
*/
|
||||
const CARD_SELECTORS = [".EimVGf", 'li[role="listitem"]', ".iFjolb"] as const;
|
||||
|
||||
function parseTerms(raw: string | undefined): string[] {
|
||||
if (!raw) return ["software engineer"];
|
||||
try {
|
||||
const parsed = JSON.parse(raw) as unknown;
|
||||
if (Array.isArray(parsed)) {
|
||||
return parsed.map((entry) => String(entry).trim()).filter(Boolean);
|
||||
}
|
||||
} catch {
|
||||
// fall through to delimited parsing
|
||||
}
|
||||
return raw
|
||||
.split(/[\n|,]+/)
|
||||
.map((entry) => entry.trim())
|
||||
.filter(Boolean);
|
||||
}
|
||||
|
||||
/**
|
||||
* `udm=8` is Google's current internal id for the Jobs vertical (the legacy
|
||||
* `ibp=htl;jobs` deep link now gets silently redirected to it anyway, but
|
||||
* only when requested directly — combining the two, or adding `num=`/`gl=`
|
||||
* overrides that don't match the browser's real/proxied IP, has been
|
||||
* observed to make the vertical come back "no matches" or "can't determine
|
||||
* location" even though the widget itself loads correctly). `hl=en` keeps
|
||||
* copy in English so downstream text parsing (e.g. "via LinkedIn") is
|
||||
* consistent regardless of where the request is geolocated. Location is
|
||||
* intentionally NOT set via a `gl` param — it's derived from the real (or
|
||||
* `GOOGLE_JOBS_PROXY`) IP through Camoufox's `geoip` option instead.
|
||||
*/
|
||||
function buildSearchUrl(term: string): string {
|
||||
const params = new URLSearchParams({
|
||||
q: `${term} jobs`,
|
||||
udm: "8",
|
||||
hl: "en",
|
||||
});
|
||||
return `https://www.google.com/search?${params.toString()}`;
|
||||
}
|
||||
|
||||
/**
|
||||
* Launches Camoufox-hardened Firefox. `humanize`/`geoip` reduce (but do not
|
||||
* eliminate) Google's automated-traffic detection. `GOOGLE_JOBS_PROXY` (e.g.
|
||||
* a residential proxy URL) is the most reliable way to avoid datacenter-IP
|
||||
* blocks entirely, if configured.
|
||||
*/
|
||||
async function launchBrowser(headless: boolean): Promise<Browser> {
|
||||
const proxy = process.env.GOOGLE_JOBS_PROXY?.trim();
|
||||
return firefox.launch(
|
||||
await launchOptions({
|
||||
headless,
|
||||
humanize: true,
|
||||
geoip: true,
|
||||
...(proxy ? { proxy } : {}),
|
||||
}),
|
||||
);
|
||||
}
|
||||
|
||||
/** Google shows a cookie/consent wall in some locales before rendering results. */
|
||||
async function dismissConsent(page: Page): Promise<void> {
|
||||
const labels = ["Reject all", "I agree", "Accept all"];
|
||||
for (const label of labels) {
|
||||
try {
|
||||
const button = page.getByRole("button", { name: label }).first();
|
||||
if (await button.isVisible({ timeout: 1500 })) {
|
||||
await button.click({ timeout: 3000 });
|
||||
await page.waitForTimeout(500);
|
||||
return;
|
||||
}
|
||||
} catch {
|
||||
// selector not present in this render; try the next label
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/** Thrown when Google serves a CAPTCHA / rate-limit challenge instead of results. */
|
||||
class GoogleBlockedError extends Error {}
|
||||
|
||||
/** Google serves an interstitial CAPTCHA page for automated / datacenter traffic. */
|
||||
async function isBlockedPage(page: Page): Promise<boolean> {
|
||||
if (/\/sorry\//.test(page.url())) return true;
|
||||
try {
|
||||
return await page.evaluate(() => {
|
||||
const bodyText = document.body?.textContent ?? "";
|
||||
return (
|
||||
document.querySelector("#captcha-form") !== null ||
|
||||
document.querySelector(".g-recaptcha") !== null ||
|
||||
/unusual traffic/i.test(bodyText)
|
||||
);
|
||||
});
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* In headed/interactive mode, give a human at the keyboard a chance to solve
|
||||
* the CAPTCHA in the visible browser window instead of failing immediately.
|
||||
* Polls until the challenge clears (Google's form redirects on success) or
|
||||
* the timeout elapses.
|
||||
*/
|
||||
async function waitForHumanToUnblock(
|
||||
page: Page,
|
||||
timeoutMs: number,
|
||||
): Promise<boolean> {
|
||||
const deadline = Date.now() + timeoutMs;
|
||||
console.error(
|
||||
`[google-jobs] Google is showing a CAPTCHA in the open browser window — please solve it there. Waiting up to ${Math.round(timeoutMs / 1000)}s...`,
|
||||
);
|
||||
while (Date.now() < deadline) {
|
||||
await page.waitForTimeout(3_000);
|
||||
if (!(await isBlockedPage(page))) {
|
||||
console.error("[google-jobs] CAPTCHA cleared, resuming.");
|
||||
return true;
|
||||
}
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
async function waitForJobsList(page: Page): Promise<boolean> {
|
||||
for (const selector of CARD_SELECTORS) {
|
||||
try {
|
||||
await page.waitForSelector(selector, { timeout: 8_000 });
|
||||
return true;
|
||||
} catch {
|
||||
// try the next fallback selector
|
||||
}
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
/**
|
||||
* Google sometimes lands correctly on the Jobs vertical/tab but reports no
|
||||
* matches for the query (distinct from a CAPTCHA block or a selector-rot
|
||||
* failure) — e.g. "It looks like there aren't any 'Jobs' matches on this topic."
|
||||
*/
|
||||
async function isEmptyJobsVertical(page: Page): Promise<boolean> {
|
||||
try {
|
||||
return await page.evaluate(() => {
|
||||
const bodyText = document.body?.textContent ?? "";
|
||||
return /aren.t any .Jobs. matches/i.test(bodyText);
|
||||
});
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
function countCardsScript(selectors: readonly string[]): number {
|
||||
for (const selector of selectors) {
|
||||
const count = document.querySelectorAll(selector).length;
|
||||
if (count > 0) return count;
|
||||
}
|
||||
return 0;
|
||||
}
|
||||
|
||||
/** Google lazy-loads additional cards (via a custom `<infinity-scrolling>` element) as the page is scrolled. */
|
||||
function scrollPageScript(): number {
|
||||
const before = window.scrollY;
|
||||
window.scrollTo(0, document.body.scrollHeight);
|
||||
return window.scrollY - before;
|
||||
}
|
||||
|
||||
async function growJobsList(
|
||||
page: Page,
|
||||
selectors: readonly string[],
|
||||
targetCount: number,
|
||||
): Promise<void> {
|
||||
for (let attempt = 0; attempt < 8; attempt += 1) {
|
||||
const count = await page.evaluate(countCardsScript, selectors);
|
||||
if (count >= targetCount) return;
|
||||
const scrolled = await page.evaluate(scrollPageScript);
|
||||
if (scrolled <= 0) return;
|
||||
await page.waitForTimeout(900);
|
||||
}
|
||||
}
|
||||
|
||||
function extractCardsScript(selectors: readonly string[]): ScrapedCard[] {
|
||||
// NOTE: this whole function is serialized via Function.prototype.toString()
|
||||
// and re-run inside the browser page by Playwright — it cannot reference
|
||||
// anything from the outer module scope (constants, other functions, etc.).
|
||||
// All helpers must be declared inside this function body.
|
||||
function clean(value: string | null | undefined): string {
|
||||
return (value ?? "").replace(/\s+/g, " ").trim();
|
||||
}
|
||||
|
||||
/** Extracts Google's internal `htidocid` from a job's `data-share-url`, matching `data-encoded-docid` in the detail panel's apply-link list. */
|
||||
function extractDocId(shareUrl: string | null): string | undefined {
|
||||
if (!shareUrl) return undefined;
|
||||
try {
|
||||
const url = new URL(shareUrl, "https://www.google.com");
|
||||
return url.searchParams.get("htidocid") ?? undefined;
|
||||
} catch {
|
||||
return undefined;
|
||||
}
|
||||
}
|
||||
|
||||
function findCards(): Element[] {
|
||||
for (const selector of selectors) {
|
||||
const items = Array.from(document.querySelectorAll(selector));
|
||||
if (items.length > 0) return items;
|
||||
}
|
||||
return [];
|
||||
}
|
||||
|
||||
const cards: ScrapedCard[] = [];
|
||||
|
||||
for (const item of findCards()) {
|
||||
// Known current classes first (title/employer/location+via), falling
|
||||
// back to a leaf-node text heuristic if Google has rotated them.
|
||||
const titleEl = item.querySelector(".tNxQIb, h2, [role='heading']");
|
||||
const title = clean(titleEl?.textContent);
|
||||
if (!title) continue;
|
||||
|
||||
const wHYlTdEls = Array.from(item.querySelectorAll(".wHYlTd"));
|
||||
let employer = clean(wHYlTdEls[0]?.textContent);
|
||||
let location: string | undefined;
|
||||
let via: string | undefined;
|
||||
|
||||
const viaBlock = wHYlTdEls.find((el) =>
|
||||
/\bvia\b/i.test(el.textContent ?? ""),
|
||||
);
|
||||
if (viaBlock) {
|
||||
const text = clean(viaBlock.textContent);
|
||||
const match = text.match(/^(.*?)\s*(?:•|·)\s*via\s+(.+)$/i);
|
||||
if (match) {
|
||||
location = match[1] || undefined;
|
||||
via = match[2] || undefined;
|
||||
} else {
|
||||
via = clean(text.replace(/^via\s*/i, ""));
|
||||
}
|
||||
}
|
||||
|
||||
if (!employer || !title || employer === title) {
|
||||
// Selector rot fallback: scan leaf text nodes structurally.
|
||||
const leafTexts = Array.from(item.querySelectorAll("div, span"))
|
||||
.filter((el) => el.children.length === 0)
|
||||
.map((el) => clean(el.textContent))
|
||||
.filter((text) => text && text !== title && !/^[•·|,]+$/.test(text));
|
||||
employer = employer || leafTexts[0] || "Unknown Employer";
|
||||
if (!via) {
|
||||
const viaText = leafTexts.find((text) => /^via\b/i.test(text));
|
||||
via = viaText ? clean(viaText.replace(/^via\s*/i, "")) : undefined;
|
||||
location =
|
||||
location ?? leafTexts.slice(1).find((text) => text !== viaText);
|
||||
}
|
||||
}
|
||||
|
||||
const shareUrlRaw = item.getAttribute("data-share-url") ?? undefined;
|
||||
const docId = extractDocId(shareUrlRaw ?? null);
|
||||
|
||||
cards.push({
|
||||
docId,
|
||||
title,
|
||||
employer: employer || "Unknown Employer",
|
||||
location,
|
||||
via,
|
||||
shareUrl: shareUrlRaw,
|
||||
});
|
||||
}
|
||||
|
||||
return cards;
|
||||
}
|
||||
|
||||
/**
|
||||
* Scopes apply-link extraction to the clicked job's own `data-encoded-docid`
|
||||
* block. Google's detail viewer can have adjacent/related jobs' apply links
|
||||
* present in the DOM simultaneously, so a document-wide "first apply link"
|
||||
* scan can silently pick up the wrong job's link.
|
||||
*/
|
||||
function extractDetailScript(docId: string | undefined): ScrapedDetail {
|
||||
function clean(value: string | null | undefined): string {
|
||||
return (value ?? "").replace(/\s+/g, " ").trim();
|
||||
}
|
||||
|
||||
function collectApplyLinks(
|
||||
root: ParentNode,
|
||||
): Array<{ href: string; text: string }> {
|
||||
const links: Array<{ href: string; text: string }> = [];
|
||||
for (const anchor of Array.from(root.querySelectorAll("a[href]"))) {
|
||||
const href = (anchor as HTMLAnchorElement).href;
|
||||
const text =
|
||||
clean(anchor.textContent) || clean(anchor.getAttribute("aria-label"));
|
||||
if (!href) continue;
|
||||
if (
|
||||
!/apply/i.test(text) &&
|
||||
!/apply/i.test(anchor.getAttribute("aria-label") ?? "")
|
||||
) {
|
||||
continue;
|
||||
}
|
||||
if (
|
||||
href.includes("google.com/search") ||
|
||||
href.includes("accounts.google.com")
|
||||
) {
|
||||
continue;
|
||||
}
|
||||
links.push({ href, text });
|
||||
}
|
||||
return links;
|
||||
}
|
||||
|
||||
let applyLinks: Array<{ href: string; text: string }> = [];
|
||||
if (docId) {
|
||||
const scoped = document.querySelector(`[data-encoded-docid="${docId}"]`);
|
||||
if (scoped) applyLinks = collectApplyLinks(scoped);
|
||||
}
|
||||
if (applyLinks.length === 0) {
|
||||
applyLinks = collectApplyLinks(document);
|
||||
}
|
||||
|
||||
// `.textContent` also pulls in text from nested <style>/<script> tags
|
||||
// (Google embeds scoped theme <style> blocks inside content divs) and from
|
||||
// the job-list sidebar itself (which concatenates every visible card's
|
||||
// text and easily outsizes a real description) — exclude both.
|
||||
const blocks = Array.from(document.querySelectorAll("div"))
|
||||
.filter(
|
||||
(el) =>
|
||||
el.children.length <= 3 && !el.querySelector("style, script, .EimVGf"),
|
||||
)
|
||||
.map((el) => clean(el.textContent))
|
||||
.filter((text) => text.length > 200);
|
||||
blocks.sort((a, b) => b.length - a.length);
|
||||
|
||||
return {
|
||||
applyLinks,
|
||||
description: blocks[0]?.slice(0, 5_000),
|
||||
};
|
||||
}
|
||||
|
||||
async function openCardAndExtractDetail(
|
||||
page: Page,
|
||||
selectors: readonly string[],
|
||||
index: number,
|
||||
docId: string | undefined,
|
||||
): Promise<ScrapedDetail | null> {
|
||||
const elementHandle = await page.evaluateHandle(
|
||||
(args: { selectors: readonly string[]; idx: number }) => {
|
||||
for (const selector of args.selectors) {
|
||||
const items = document.querySelectorAll(selector);
|
||||
if (items.length > 0) return items[args.idx] ?? null;
|
||||
}
|
||||
return null;
|
||||
},
|
||||
{ selectors, idx: index },
|
||||
);
|
||||
|
||||
const element = elementHandle.asElement();
|
||||
if (!element) return null;
|
||||
|
||||
await element.scrollIntoViewIfNeeded();
|
||||
// A single click is unreliable here: clicking a card while a *different*
|
||||
// card's detail pane is already open frequently just closes the open pane
|
||||
// instead of switching to the new one (confirmed by watching real runs —
|
||||
// it alternates open/closed rather than opening the newly clicked card).
|
||||
// A double-click reopens reliably; verified across 6 sequential cards.
|
||||
await element.dblclick({ timeout: 5_000, delay: 80 });
|
||||
if (docId) {
|
||||
try {
|
||||
await page.waitForSelector(`[data-encoded-docid="${docId}"]`, {
|
||||
timeout: 6_000,
|
||||
});
|
||||
} catch {
|
||||
// fall through — extractDetailScript falls back to a document-wide
|
||||
// scan for apply links if the docid-scoped block never shows up.
|
||||
}
|
||||
}
|
||||
await page.waitForTimeout(500);
|
||||
return page.evaluate(extractDetailScript, docId);
|
||||
}
|
||||
|
||||
async function saveDebugArtifacts(page: Page): Promise<void> {
|
||||
try {
|
||||
await mkdir(dirname(DEBUG_HTML_PATH), { recursive: true });
|
||||
await writeFile(DEBUG_HTML_PATH, await page.content(), "utf-8");
|
||||
await page.screenshot({ path: DEBUG_SCREENSHOT_PATH, fullPage: true });
|
||||
} catch (error) {
|
||||
console.error(
|
||||
`[google-jobs] failed to save debug artifacts: ${error instanceof Error ? error.message : String(error)}`,
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
async function scrapeTerm(
|
||||
context: BrowserContext,
|
||||
term: string,
|
||||
remainingCap: number,
|
||||
options: { headless: boolean; unblockTimeoutMs: number },
|
||||
): Promise<ScrapedJob[]> {
|
||||
const page = await context.newPage();
|
||||
const out: ScrapedJob[] = [];
|
||||
|
||||
try {
|
||||
await page.goto(buildSearchUrl(term), {
|
||||
waitUntil: "domcontentloaded",
|
||||
timeout: 45_000,
|
||||
});
|
||||
await dismissConsent(page);
|
||||
await page.waitForTimeout(1_500);
|
||||
|
||||
if (await isBlockedPage(page)) {
|
||||
const solvedByHuman =
|
||||
!options.headless &&
|
||||
(await waitForHumanToUnblock(page, options.unblockTimeoutMs));
|
||||
if (!solvedByHuman) {
|
||||
throw new GoogleBlockedError(
|
||||
"Google blocked this request (CAPTCHA / unusual traffic detection). This is common from shared, VPN, or datacenter IPs. Try again later, reduce run frequency, run from a residential network, set GOOGLE_JOBS_PROXY, or set GOOGLE_JOBS_HEADLESS=false to solve it yourself.",
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
const listReady = await waitForJobsList(page);
|
||||
if (!listReady) {
|
||||
if (await isEmptyJobsVertical(page)) {
|
||||
console.error(
|
||||
`[google-jobs] Google's Jobs tab reported no matches for "${term}" (not a block or selector issue — try a broader/more common phrasing).`,
|
||||
);
|
||||
} else {
|
||||
console.error(
|
||||
`[google-jobs] Jobs list did not appear for "${term}" (url: ${page.url()}). Saved a screenshot + HTML dump to ${DEBUG_SCREENSHOT_PATH} for inspection.`,
|
||||
);
|
||||
await saveDebugArtifacts(page);
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
await growJobsList(page, CARD_SELECTORS, remainingCap);
|
||||
|
||||
const cards = await page.evaluate(extractCardsScript, CARD_SELECTORS);
|
||||
const limit = Math.min(cards.length, remainingCap);
|
||||
|
||||
for (let index = 0; index < limit; index += 1) {
|
||||
const card = cards[index];
|
||||
// Google's own permalink into this specific listing — always available
|
||||
// from the list view, so results stay unique/useful even if the detail
|
||||
// click below fails for this card.
|
||||
const shareUrl = card.shareUrl;
|
||||
try {
|
||||
const detail = await openCardAndExtractDetail(
|
||||
page,
|
||||
CARD_SELECTORS,
|
||||
index,
|
||||
card.docId,
|
||||
);
|
||||
const applyLink = detail?.applyLinks[0];
|
||||
const jobUrl = shareUrl ?? applyLink?.href ?? page.url();
|
||||
|
||||
out.push({
|
||||
title: card.title,
|
||||
employer: card.employer,
|
||||
jobUrl,
|
||||
applicationLink: applyLink?.href,
|
||||
location: card.location,
|
||||
jobDescription: detail?.description,
|
||||
postedVia: card.via,
|
||||
isRemote: /remote/i.test(`${card.location ?? ""} ${card.title}`),
|
||||
});
|
||||
} catch (error) {
|
||||
console.error(
|
||||
`[google-jobs] card ${index} for "${term}" failed: ${
|
||||
error instanceof Error ? error.message : String(error)
|
||||
}`,
|
||||
);
|
||||
if (shareUrl) {
|
||||
// Detail click failed, but we still have a unique, valid link to
|
||||
// the listing from the card itself — don't drop the job entirely.
|
||||
out.push({
|
||||
title: card.title,
|
||||
employer: card.employer,
|
||||
jobUrl: shareUrl,
|
||||
location: card.location,
|
||||
postedVia: card.via,
|
||||
isRemote: /remote/i.test(`${card.location ?? ""} ${card.title}`),
|
||||
});
|
||||
}
|
||||
}
|
||||
await page.waitForTimeout(300 + Math.floor(Math.random() * 400));
|
||||
}
|
||||
} finally {
|
||||
await page.close();
|
||||
}
|
||||
|
||||
return out;
|
||||
}
|
||||
|
||||
async function main(): Promise<void> {
|
||||
const terms = parseTerms(process.env.GOOGLE_JOBS_SEARCH_TERMS);
|
||||
const maxJobsEnv = Number.parseInt(
|
||||
process.env.GOOGLE_JOBS_MAX_JOBS ?? "30",
|
||||
10,
|
||||
);
|
||||
const cap = Number.isFinite(maxJobsEnv)
|
||||
? Math.max(1, Math.min(maxJobsEnv, 150))
|
||||
: 30;
|
||||
// Headed mode lets a human solve a CAPTCHA in the visible window; only
|
||||
// useful when run directly on a machine with a display, not on a headless
|
||||
// cron/server pipeline.
|
||||
const headless = process.env.GOOGLE_JOBS_HEADLESS !== "false";
|
||||
const unblockTimeoutMs = Number.parseInt(
|
||||
process.env.GOOGLE_JOBS_UNBLOCK_TIMEOUT_MS ?? "300000",
|
||||
10,
|
||||
);
|
||||
|
||||
const browser = await launchBrowser(headless);
|
||||
const hasSavedState = await access(STATE_PATH)
|
||||
.then(() => true)
|
||||
.catch(() => false);
|
||||
const context = await browser.newContext(
|
||||
hasSavedState ? { storageState: STATE_PATH } : {},
|
||||
);
|
||||
// tsx/esbuild compiles this file with `keepNames`, which injects
|
||||
// `__name(fn, "fn")` calls into any named function it transpiles —
|
||||
// including ones later serialized into `page.evaluate()` scripts via
|
||||
// `Function.prototype.toString()`. That helper only exists in our Node
|
||||
// process, not in the browser page, so without this no-op polyfill any
|
||||
// evaluate script with a nested named function throws
|
||||
// `ReferenceError: __name is not defined`.
|
||||
await context.addInitScript(() => {
|
||||
(window as unknown as { __name?: (fn: unknown) => unknown }).__name = (
|
||||
fn: unknown,
|
||||
) => fn;
|
||||
});
|
||||
|
||||
const all: ScrapedJob[] = [];
|
||||
const seenUrls = new Set<string>();
|
||||
|
||||
try {
|
||||
for (const term of terms) {
|
||||
if (all.length >= cap) break;
|
||||
try {
|
||||
const rows = await scrapeTerm(context, term, cap - all.length, {
|
||||
headless,
|
||||
unblockTimeoutMs,
|
||||
});
|
||||
for (const row of rows) {
|
||||
if (all.length >= cap) break;
|
||||
if (seenUrls.has(row.jobUrl)) continue;
|
||||
seenUrls.add(row.jobUrl);
|
||||
all.push(row);
|
||||
}
|
||||
} catch (error) {
|
||||
if (error instanceof GoogleBlockedError) {
|
||||
// IP-level block affects the whole run; stop trying further terms
|
||||
// and let it propagate so the process exits non-zero with a clear message.
|
||||
throw error;
|
||||
}
|
||||
console.error(
|
||||
`[google-jobs] term "${term}" failed: ${
|
||||
error instanceof Error ? error.message : String(error)
|
||||
}`,
|
||||
);
|
||||
}
|
||||
}
|
||||
} finally {
|
||||
// Persist cookies/session (even a partially solved one) so future
|
||||
// headless/automated runs can reuse it and hit fewer CAPTCHA walls.
|
||||
await mkdir(dirname(STATE_PATH), { recursive: true });
|
||||
await context.storageState({ path: STATE_PATH }).catch(() => undefined);
|
||||
await browser.close();
|
||||
}
|
||||
|
||||
await mkdir(dirname(OUTPUT_PATH), { recursive: true });
|
||||
await writeFile(OUTPUT_PATH, JSON.stringify(all, null, 2), "utf-8");
|
||||
}
|
||||
|
||||
main().catch((error) => {
|
||||
console.error(error instanceof Error ? error.message : String(error));
|
||||
process.exit(1);
|
||||
});
|
||||
@@ -0,0 +1,168 @@
|
||||
import { spawn, spawnSync } from "node:child_process";
|
||||
import { readFile } from "node:fs/promises";
|
||||
import { createRequire } from "node:module";
|
||||
import { dirname, join } from "node:path";
|
||||
import { fileURLToPath } from "node:url";
|
||||
import { envForExtractorSubprocess } from "@shared/extractor-subprocess-env.js";
|
||||
import type { CreateJobInput } from "@shared/types/jobs";
|
||||
|
||||
const srcDir = dirname(fileURLToPath(import.meta.url));
|
||||
const EXTRACTOR_DIR = join(srcDir, "..");
|
||||
const OUTPUT_PATH = join(EXTRACTOR_DIR, "storage/jobs.json");
|
||||
const require = createRequire(import.meta.url);
|
||||
const TSX_CLI_PATH = (() => {
|
||||
try {
|
||||
return require.resolve("tsx/dist/cli.mjs");
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
})();
|
||||
|
||||
/** Browser scrape can take a while (per-term navigation + per-card clicks). */
|
||||
const SUBPROCESS_TIMEOUT_MS = 4 * 60 * 1000;
|
||||
|
||||
interface ScrapedJob {
|
||||
title?: string;
|
||||
employer?: string;
|
||||
jobUrl?: string;
|
||||
applicationLink?: string;
|
||||
location?: string;
|
||||
jobDescription?: string;
|
||||
postedVia?: string;
|
||||
isRemote?: boolean;
|
||||
}
|
||||
|
||||
export interface RunGoogleJobsOptions {
|
||||
searchTerms?: string[];
|
||||
maxJobs?: number;
|
||||
}
|
||||
|
||||
export interface GoogleJobsResult {
|
||||
success: boolean;
|
||||
jobs: CreateJobInput[];
|
||||
error?: string;
|
||||
}
|
||||
|
||||
function canRunNpmCommand(): boolean {
|
||||
const result = spawnSync("npm", ["--version"], { stdio: "ignore" });
|
||||
return !result.error && result.status === 0;
|
||||
}
|
||||
|
||||
function mapJob(row: ScrapedJob): CreateJobInput | null {
|
||||
const jobUrl = row.jobUrl?.trim();
|
||||
if (!jobUrl) return null;
|
||||
return {
|
||||
source: "google-jobs",
|
||||
title: row.title?.trim() || "Unknown Title",
|
||||
employer: row.employer?.trim() || "Unknown Employer",
|
||||
jobUrl,
|
||||
applicationLink: row.applicationLink?.trim() || jobUrl,
|
||||
location: row.location?.trim() || undefined,
|
||||
jobDescription: row.jobDescription?.trim() || undefined,
|
||||
isRemote: row.isRemote ?? undefined,
|
||||
};
|
||||
}
|
||||
|
||||
export async function runGoogleJobs(
|
||||
options: RunGoogleJobsOptions = {},
|
||||
): Promise<GoogleJobsResult> {
|
||||
const searchTerms =
|
||||
options.searchTerms && options.searchTerms.length > 0
|
||||
? options.searchTerms
|
||||
: ["software engineer"];
|
||||
const maxJobs = options.maxJobs ?? 30;
|
||||
|
||||
const useNpmCommand = canRunNpmCommand();
|
||||
if (!TSX_CLI_PATH && !useNpmCommand) {
|
||||
return {
|
||||
success: false,
|
||||
jobs: [],
|
||||
error: "Unable to execute Google Jobs extractor (npm/tsx unavailable)",
|
||||
};
|
||||
}
|
||||
|
||||
try {
|
||||
await new Promise<void>((resolve, reject) => {
|
||||
const extractorEnv = envForExtractorSubprocess({
|
||||
...process.env,
|
||||
GOOGLE_JOBS_SEARCH_TERMS: JSON.stringify(searchTerms),
|
||||
GOOGLE_JOBS_MAX_JOBS: String(maxJobs),
|
||||
});
|
||||
|
||||
const child = TSX_CLI_PATH
|
||||
? spawn(process.execPath, [TSX_CLI_PATH, join(srcDir, "main.ts")], {
|
||||
cwd: EXTRACTOR_DIR,
|
||||
env: extractorEnv,
|
||||
stdio: ["ignore", "pipe", "pipe"],
|
||||
})
|
||||
: spawn("npx", ["tsx", join(srcDir, "main.ts")], {
|
||||
cwd: EXTRACTOR_DIR,
|
||||
env: extractorEnv,
|
||||
stdio: ["ignore", "pipe", "pipe"],
|
||||
});
|
||||
|
||||
let stderr = "";
|
||||
child.stderr?.on("data", (chunk: Buffer) => {
|
||||
stderr += chunk.toString();
|
||||
});
|
||||
|
||||
const timeout = setTimeout(() => {
|
||||
child.kill("SIGKILL");
|
||||
reject(new Error("Google Jobs browser scrape timed out"));
|
||||
}, SUBPROCESS_TIMEOUT_MS);
|
||||
|
||||
child.on("error", (error) => {
|
||||
clearTimeout(timeout);
|
||||
reject(error);
|
||||
});
|
||||
child.on("close", (code) => {
|
||||
clearTimeout(timeout);
|
||||
if (code === 0) {
|
||||
resolve();
|
||||
return;
|
||||
}
|
||||
reject(
|
||||
new Error(
|
||||
stderr.trim() ||
|
||||
`Google Jobs browser scrape exited with code ${code}`,
|
||||
),
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
const raw = await readFile(OUTPUT_PATH, "utf-8");
|
||||
const parsed = JSON.parse(raw) as unknown;
|
||||
const rows = Array.isArray(parsed) ? parsed : [];
|
||||
|
||||
const jobs: CreateJobInput[] = [];
|
||||
const seen = new Set<string>();
|
||||
for (const value of rows) {
|
||||
if (!value || typeof value !== "object" || Array.isArray(value)) continue;
|
||||
const mapped = mapJob(value as ScrapedJob);
|
||||
if (!mapped) continue;
|
||||
const key = mapped.jobUrl;
|
||||
if (seen.has(key)) continue;
|
||||
seen.add(key);
|
||||
jobs.push(mapped);
|
||||
}
|
||||
|
||||
const { filterExpiredJobListings } = await import(
|
||||
"@shared/job-listing-probe.js"
|
||||
);
|
||||
const { kept, dropped } = await filterExpiredJobListings(jobs, {
|
||||
concurrency: 4,
|
||||
timeoutMs: 5_000,
|
||||
});
|
||||
|
||||
if (dropped > 0) {
|
||||
console.info(
|
||||
`[google-jobs] Dropped ${dropped} expired listing(s) after URL probe`,
|
||||
);
|
||||
}
|
||||
|
||||
return { success: true, jobs: kept };
|
||||
} catch (error) {
|
||||
const message = error instanceof Error ? error.message : "Unknown error";
|
||||
return { success: false, jobs: [], error: message };
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,17 @@
|
||||
{
|
||||
"compilerOptions": {
|
||||
"module": "ESNext",
|
||||
"moduleResolution": "bundler",
|
||||
"target": "ES2022",
|
||||
"outDir": "dist",
|
||||
"strict": true,
|
||||
"noUnusedLocals": false,
|
||||
"lib": ["ES2022", "DOM"],
|
||||
"types": ["node"],
|
||||
"baseUrl": ".",
|
||||
"paths": {
|
||||
"@shared/*": ["../../shared/src/*"]
|
||||
}
|
||||
},
|
||||
"include": ["./manifest.ts", "./src/**/*"]
|
||||
}
|
||||
@@ -116,11 +116,32 @@ router.addHandler(
|
||||
const title = (await titleLocator.textContent())?.trim() ?? null;
|
||||
const jobUrl = toAbsolute(await titleLocator.getAttribute("href"));
|
||||
|
||||
const employerImg = article.locator("figure img");
|
||||
const employer = (await employerImg.getAttribute("alt"))?.trim() ?? null;
|
||||
|
||||
const employerAnchor = article.locator("figure a");
|
||||
const employerUrl = toAbsolute(await employerAnchor.getAttribute("href"));
|
||||
// Employer logo/link are optional — some cards omit <figure img>, and a
|
||||
// hard getAttribute timeout used to fail the whole region page (0 jobs)
|
||||
// even after we had already counted cards.
|
||||
let employer: string | null = null;
|
||||
let employerUrl: string | null = null;
|
||||
try {
|
||||
const employerImg = article.locator("figure img").first();
|
||||
if ((await employerImg.count()) > 0) {
|
||||
employer =
|
||||
(
|
||||
await employerImg.getAttribute("alt", { timeout: 2000 })
|
||||
)?.trim() ?? null;
|
||||
}
|
||||
} catch {
|
||||
// optional
|
||||
}
|
||||
try {
|
||||
const employerAnchor = article.locator("figure a").first();
|
||||
if ((await employerAnchor.count()) > 0) {
|
||||
employerUrl = toAbsolute(
|
||||
await employerAnchor.getAttribute("href", { timeout: 2000 }),
|
||||
);
|
||||
}
|
||||
} catch {
|
||||
// optional
|
||||
}
|
||||
|
||||
let disciplines: string | null = null;
|
||||
try {
|
||||
|
||||
@@ -14,8 +14,10 @@ import type {
|
||||
import type { CreateJobInput } from "@shared/types/jobs";
|
||||
|
||||
const API_URL = "https://himalayas.app/jobs/api";
|
||||
const PAGE_SIZE = 50;
|
||||
const MAX_PAGES = 5;
|
||||
/** Himalayas silently caps `limit` at 20; requesting more makes us think the
|
||||
* feed ended early (`raw.length < PAGE_SIZE`) and stop after the first page. */
|
||||
const PAGE_SIZE = 20;
|
||||
const MAX_PAGES = 10;
|
||||
|
||||
interface HimalayasJob {
|
||||
title?: string;
|
||||
|
||||
@@ -111,12 +111,6 @@ function parseWorkplaceTypes(
|
||||
}
|
||||
}
|
||||
|
||||
function encodeSearchState(searchState: unknown): string {
|
||||
const json = JSON.stringify(searchState);
|
||||
const urlEncodedJson = encodeURIComponent(json);
|
||||
return Buffer.from(urlEncodedJson, "utf-8").toString("base64");
|
||||
}
|
||||
|
||||
function asRecord(value: unknown): Record<string, unknown> | null {
|
||||
if (!value || typeof value !== "object" || Array.isArray(value)) return null;
|
||||
return value as Record<string, unknown>;
|
||||
@@ -230,7 +224,18 @@ function extractResultsBatch(payload: unknown): RawHiringCafeJob[] {
|
||||
}
|
||||
|
||||
const payloadRecord = asRecord(payload);
|
||||
const results = payloadRecord?.results;
|
||||
if (!payloadRecord) return [];
|
||||
|
||||
const pageProps = asRecord(payloadRecord.pageProps);
|
||||
const ssrHits = pageProps?.ssrHits ?? payloadRecord.ssrHits;
|
||||
if (Array.isArray(ssrHits)) {
|
||||
return ssrHits.filter(
|
||||
(item): item is RawHiringCafeJob =>
|
||||
Boolean(item) && typeof item === "object" && !Array.isArray(item),
|
||||
);
|
||||
}
|
||||
|
||||
const results = payloadRecord.results;
|
||||
if (!Array.isArray(results)) return [];
|
||||
|
||||
return results.filter(
|
||||
@@ -242,7 +247,22 @@ function extractResultsBatch(payload: unknown): RawHiringCafeJob[] {
|
||||
function parseTotalCount(payload: unknown): number | null {
|
||||
const payloadRecord = asRecord(payload);
|
||||
if (!payloadRecord) return null;
|
||||
return toNumberOrNull(payloadRecord.total);
|
||||
|
||||
const pageProps = asRecord(payloadRecord.pageProps);
|
||||
return (
|
||||
toNumberOrNull(pageProps?.ssrTotalCount) ??
|
||||
toNumberOrNull(payloadRecord.ssrTotalCount) ??
|
||||
toNumberOrNull(payloadRecord.total)
|
||||
);
|
||||
}
|
||||
|
||||
function isLastSsrPage(payload: unknown): boolean {
|
||||
const payloadRecord = asRecord(payload);
|
||||
if (!payloadRecord) return true;
|
||||
|
||||
const pageProps = asRecord(payloadRecord.pageProps);
|
||||
const isLastPage = pageProps?.ssrIsLastPage ?? payloadRecord.ssrIsLastPage;
|
||||
return isLastPage === true;
|
||||
}
|
||||
|
||||
function buildCityLocationId(input: string): string {
|
||||
@@ -501,23 +521,37 @@ function createCitySearchState(args: {
|
||||
};
|
||||
}
|
||||
|
||||
async function callHiringCafeApi(
|
||||
async function readBuildIdFromPage(page: Page): Promise<string> {
|
||||
const buildId = await page.evaluate(() => {
|
||||
const element = document.querySelector("#__NEXT_DATA__");
|
||||
if (!element?.textContent) return null;
|
||||
const data = JSON.parse(element.textContent) as { buildId?: string };
|
||||
return data.buildId ?? null;
|
||||
});
|
||||
if (!buildId) {
|
||||
throw new Error("Hiring Cafe page did not expose Next.js buildId");
|
||||
}
|
||||
return buildId;
|
||||
}
|
||||
|
||||
async function fetchSsrSearchPage(
|
||||
page: Page,
|
||||
endpoint: string,
|
||||
params: Record<string, string>,
|
||||
buildId: string,
|
||||
searchState: Record<string, unknown>,
|
||||
pageNo: number,
|
||||
): Promise<unknown> {
|
||||
const response = await page.evaluate(
|
||||
async ({ endpointArg, paramsArg }) => {
|
||||
const url = new URL(endpointArg, window.location.origin);
|
||||
for (const [key, value] of Object.entries(paramsArg)) {
|
||||
url.searchParams.set(key, value);
|
||||
}
|
||||
async ({ buildIdArg, searchStateArg, pageNoArg }) => {
|
||||
const params = new URLSearchParams();
|
||||
params.set("searchState", JSON.stringify(searchStateArg));
|
||||
params.set("page", String(pageNoArg));
|
||||
const url = `/_next/data/${buildIdArg}/index.json?${params.toString()}`;
|
||||
|
||||
const res = await fetch(url.toString(), {
|
||||
method: "GET",
|
||||
const res = await fetch(url, {
|
||||
credentials: "include",
|
||||
headers: {
|
||||
Accept: "application/json, text/plain, */*",
|
||||
Accept: "application/json",
|
||||
"x-nextjs-data": "1",
|
||||
},
|
||||
});
|
||||
|
||||
@@ -539,7 +573,7 @@ async function callHiringCafeApi(
|
||||
|
||||
return output;
|
||||
},
|
||||
{ endpointArg: endpoint, paramsArg: params },
|
||||
{ buildIdArg: buildId, searchStateArg: searchState, pageNoArg: pageNo },
|
||||
);
|
||||
|
||||
const result = response as BrowserApiResponse;
|
||||
@@ -547,14 +581,14 @@ async function callHiringCafeApi(
|
||||
if (!result.ok) {
|
||||
const snippet = result.responseText.slice(0, 250);
|
||||
throw new Error(
|
||||
`Hiring Cafe API ${endpoint} failed (${result.status} ${result.statusText}): ${snippet}`,
|
||||
`Hiring Cafe SSR search failed (${result.status} ${result.statusText}): ${snippet}`,
|
||||
);
|
||||
}
|
||||
|
||||
if (result.data === null) {
|
||||
const snippet = result.responseText.slice(0, 250);
|
||||
throw new Error(
|
||||
`Hiring Cafe API ${endpoint} returned non-JSON response: ${snippet}`,
|
||||
`Hiring Cafe SSR search returned non-JSON response: ${snippet}`,
|
||||
);
|
||||
}
|
||||
|
||||
@@ -626,6 +660,8 @@ async function run(): Promise<void> {
|
||||
await initializePage();
|
||||
}
|
||||
|
||||
const buildId = await readBuildIdFromPage(page);
|
||||
|
||||
const countryLocation = resolveHiringCafeCountryLocation(country);
|
||||
const countryLong =
|
||||
countryLocation?.address_components[0]?.long_name ?? "United Kingdom";
|
||||
@@ -664,40 +700,23 @@ async function run(): Promise<void> {
|
||||
dateFetchedPastNDays,
|
||||
workplaceTypes,
|
||||
});
|
||||
const encodedSearchState = encodeSearchState(searchState);
|
||||
|
||||
let totalAvailable: number | null = null;
|
||||
try {
|
||||
const countPayload = await callHiringCafeApi(
|
||||
page,
|
||||
"/api/search-jobs/get-total-count",
|
||||
{
|
||||
s: encodedSearchState,
|
||||
},
|
||||
);
|
||||
totalAvailable = parseTotalCount(countPayload);
|
||||
} catch (error) {
|
||||
const message = error instanceof Error ? error.message : String(error);
|
||||
console.warn(
|
||||
`Hiring Cafe count request failed for term '${searchTerm}': ${message}`,
|
||||
);
|
||||
}
|
||||
|
||||
const termTarget =
|
||||
totalAvailable !== null
|
||||
? Math.min(maxJobsPerTerm, totalAvailable)
|
||||
: maxJobsPerTerm;
|
||||
let termTarget = maxJobsPerTerm;
|
||||
|
||||
let pageNo = 0;
|
||||
let termCollected = 0;
|
||||
|
||||
while (termCollected < termTarget && pageNo < PAGE_LIMIT) {
|
||||
const size = Math.min(1000, termTarget - termCollected);
|
||||
const jobsPayload = await callHiringCafeApi(page, "/api/search-jobs", {
|
||||
size: String(size),
|
||||
page: String(pageNo),
|
||||
s: encodedSearchState,
|
||||
});
|
||||
const jobsPayload = await fetchSsrSearchPage(
|
||||
page,
|
||||
buildId,
|
||||
searchState,
|
||||
pageNo,
|
||||
);
|
||||
|
||||
const totalAvailable = parseTotalCount(jobsPayload);
|
||||
if (totalAvailable !== null) {
|
||||
termTarget = Math.min(maxJobsPerTerm, totalAvailable);
|
||||
}
|
||||
|
||||
const batch = extractResultsBatch(jobsPayload);
|
||||
if (batch.length === 0) break;
|
||||
@@ -727,7 +746,7 @@ async function run(): Promise<void> {
|
||||
totalCollected: termCollected,
|
||||
});
|
||||
|
||||
if (batch.length < size) break;
|
||||
if (isLastSsrPage(jobsPayload)) break;
|
||||
pageNo += 1;
|
||||
}
|
||||
|
||||
|
||||
@@ -0,0 +1,221 @@
|
||||
/**
|
||||
* Huntflow public career sites — SSR HTML listing + optional vacancy detail pages.
|
||||
*
|
||||
* https://{tenant}.huntflow.io/
|
||||
*/
|
||||
|
||||
import type {
|
||||
ExtractorManifest,
|
||||
ExtractorRunResult,
|
||||
} from "@shared/types/extractors";
|
||||
import type { CreateJobInput } from "@shared/types/jobs";
|
||||
|
||||
interface HuntflowListing {
|
||||
slug: string;
|
||||
title: string;
|
||||
location?: string;
|
||||
}
|
||||
|
||||
function asString(value: unknown): string | undefined {
|
||||
if (typeof value !== "string") return undefined;
|
||||
const trimmed = value.trim();
|
||||
return trimmed ? trimmed : undefined;
|
||||
}
|
||||
|
||||
function readTenants(raw: string | undefined): string[] {
|
||||
if (!raw) return [];
|
||||
try {
|
||||
const parsed = JSON.parse(raw);
|
||||
if (Array.isArray(parsed)) {
|
||||
return parsed
|
||||
.map((entry) => (typeof entry === "string" ? entry.trim() : ""))
|
||||
.filter(Boolean);
|
||||
}
|
||||
} catch {
|
||||
// fall through
|
||||
}
|
||||
return raw
|
||||
.split(/[\n,;|]+/)
|
||||
.map((entry) => entry.trim())
|
||||
.filter(Boolean);
|
||||
}
|
||||
|
||||
function decodeHtmlEntities(value: string): string {
|
||||
return value
|
||||
.replace(/&/g, "&")
|
||||
.replace(/</g, "<")
|
||||
.replace(/>/g, ">")
|
||||
.replace(/"/g, '"')
|
||||
.replace(/'/g, "'");
|
||||
}
|
||||
|
||||
function stripHtml(html: string): string {
|
||||
return decodeHtmlEntities(html)
|
||||
.replace(/<[^>]+>/g, " ")
|
||||
.replace(/\s+/g, " ")
|
||||
.trim();
|
||||
}
|
||||
|
||||
function parseListings(html: string): HuntflowListing[] {
|
||||
const listings: HuntflowListing[] = [];
|
||||
const pattern =
|
||||
/<article class="_item_[^"]*">[\s\S]*?<a href="(\/vacancy\/[^"]+)"[^>]*>([^<]+)<\/a>[\s\S]*?<div class="_info_[^"]*">([^<]*)<\/div>/g;
|
||||
|
||||
for (const match of html.matchAll(pattern)) {
|
||||
const slug = match[1]?.replace(/^\/vacancy\//, "").replace(/\/$/, "");
|
||||
const title = match[2]?.trim();
|
||||
if (!slug || !title) continue;
|
||||
listings.push({
|
||||
slug,
|
||||
title: decodeHtmlEntities(title),
|
||||
location: match[3]?.trim() || undefined,
|
||||
});
|
||||
}
|
||||
|
||||
return listings;
|
||||
}
|
||||
|
||||
function parseVacancyDetail(html: string): {
|
||||
description?: string;
|
||||
location?: string;
|
||||
} {
|
||||
const positionMatch = html.match(/<h1 class="_position_[^"]*">([^<]+)<\/h1>/);
|
||||
const infoMatch = html.match(
|
||||
/<div class="_infoWrapper_[^"]*">[\s\S]*?<div>([^<]+)<\/div>/,
|
||||
);
|
||||
|
||||
const sections: string[] = [];
|
||||
for (const match of html.matchAll(
|
||||
/<div class="_content_1phzm_2">[\s\S]*?<!--\[-->([\s\S]*?)<!--\]-->/g,
|
||||
)) {
|
||||
const text = stripHtml(match[1] ?? "");
|
||||
if (text) sections.push(text);
|
||||
}
|
||||
|
||||
return {
|
||||
location: infoMatch?.[1]?.trim() || undefined,
|
||||
description:
|
||||
sections.length > 0
|
||||
? sections.join("\n\n")
|
||||
: positionMatch?.[1]?.trim() || undefined,
|
||||
};
|
||||
}
|
||||
|
||||
function matchesTerm(values: string[], term: string): boolean {
|
||||
const lower = term.toLowerCase();
|
||||
return values.some((value) => value.toLowerCase().includes(lower));
|
||||
}
|
||||
|
||||
async function fetchText(url: string): Promise<string> {
|
||||
const response = await fetch(url, {
|
||||
headers: { Accept: "text/html", "User-Agent": "JobOps/1.0" },
|
||||
});
|
||||
if (!response.ok) {
|
||||
throw new Error(`Huntflow request failed (${response.status}) for ${url}`);
|
||||
}
|
||||
return response.text();
|
||||
}
|
||||
|
||||
function tenantOrigin(tenant: string): string {
|
||||
const host = tenant.includes(".") ? tenant : `${tenant}.huntflow.io`;
|
||||
return host.startsWith("http") ? host : `https://${host}`;
|
||||
}
|
||||
|
||||
export const manifest: ExtractorManifest = {
|
||||
id: "huntflow",
|
||||
displayName: "Huntflow (ATS)",
|
||||
providesSources: ["huntflow"],
|
||||
async run(context): Promise<ExtractorRunResult> {
|
||||
if (context.shouldCancel?.()) return { success: true, jobs: [] };
|
||||
|
||||
const tenants = readTenants(context.settings.huntflowTenants);
|
||||
if (tenants.length === 0) {
|
||||
return {
|
||||
success: true,
|
||||
jobs: [],
|
||||
error:
|
||||
"No Huntflow tenants configured. Set HUNTFLOW_TENANTS or huntflowTenants (comma- or newline-separated subdomains, e.g. apicworld).",
|
||||
};
|
||||
}
|
||||
|
||||
const terms = context.searchTerms.length > 0 ? context.searchTerms : [];
|
||||
const enrichDetails = true;
|
||||
const seen = new Set<string>();
|
||||
const out: CreateJobInput[] = [];
|
||||
|
||||
try {
|
||||
for (let i = 0; i < tenants.length; i += 1) {
|
||||
if (context.shouldCancel?.()) break;
|
||||
const tenant = tenants[i];
|
||||
const origin = tenantOrigin(tenant);
|
||||
|
||||
context.onProgress?.({
|
||||
phase: "list",
|
||||
termsProcessed: i,
|
||||
termsTotal: tenants.length,
|
||||
currentUrl: origin,
|
||||
detail: `Huntflow: ${tenant} (${i + 1}/${tenants.length})`,
|
||||
});
|
||||
|
||||
const indexHtml = await fetchText(`${origin}/`);
|
||||
const employer = tenant
|
||||
.split(".")[0]
|
||||
.split(/[-_]/)
|
||||
.map((part) => part.charAt(0).toUpperCase() + part.slice(1))
|
||||
.join(" ");
|
||||
|
||||
for (const listing of parseListings(indexHtml)) {
|
||||
if (context.shouldCancel?.()) break;
|
||||
|
||||
const haystack = [listing.title, listing.location ?? ""];
|
||||
if (
|
||||
terms.length > 0 &&
|
||||
!terms.some((term) => matchesTerm(haystack, term))
|
||||
) {
|
||||
continue;
|
||||
}
|
||||
|
||||
const jobUrl = `${origin}/vacancy/${listing.slug}`;
|
||||
let location = listing.location ?? "Unknown";
|
||||
let jobDescription: string | undefined;
|
||||
let isRemote = location.toLowerCase().includes("remote");
|
||||
|
||||
if (enrichDetails) {
|
||||
try {
|
||||
const detailHtml = await fetchText(jobUrl);
|
||||
const detail = parseVacancyDetail(detailHtml);
|
||||
if (detail.location) location = detail.location;
|
||||
if (detail.description) jobDescription = detail.description;
|
||||
isRemote = location.toLowerCase().includes("remote");
|
||||
} catch {
|
||||
// keep listing row when detail fetch fails
|
||||
}
|
||||
}
|
||||
|
||||
const key = jobUrl;
|
||||
if (seen.has(key)) continue;
|
||||
seen.add(key);
|
||||
|
||||
out.push({
|
||||
source: "huntflow",
|
||||
sourceJobId: listing.slug,
|
||||
title: listing.title,
|
||||
employer,
|
||||
jobUrl,
|
||||
applicationLink: jobUrl,
|
||||
location,
|
||||
isRemote,
|
||||
jobDescription,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
return { success: true, jobs: out };
|
||||
} catch (error) {
|
||||
const message = error instanceof Error ? error.message : "Unknown error";
|
||||
return { success: false, jobs: out, error: message };
|
||||
}
|
||||
},
|
||||
};
|
||||
|
||||
export default manifest;
|
||||
@@ -0,0 +1,17 @@
|
||||
{
|
||||
"name": "huntflow-extractor",
|
||||
"version": "0.0.1",
|
||||
"type": "module",
|
||||
"description": "Huntflow public career-site extractor",
|
||||
"main": "manifest.ts",
|
||||
"dependencies": {
|
||||
"job-ops-shared": "^1.0.0"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@types/node": "^24.0.0",
|
||||
"typescript": "~5.9.0"
|
||||
},
|
||||
"scripts": {
|
||||
"check:types": "tsc --noEmit"
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,17 @@
|
||||
{
|
||||
"compilerOptions": {
|
||||
"module": "ESNext",
|
||||
"moduleResolution": "bundler",
|
||||
"target": "ES2022",
|
||||
"outDir": "dist",
|
||||
"strict": true,
|
||||
"noUnusedLocals": false,
|
||||
"lib": ["ES2022", "DOM"],
|
||||
"types": ["node"],
|
||||
"baseUrl": ".",
|
||||
"paths": {
|
||||
"@shared/*": ["../../shared/src/*"]
|
||||
}
|
||||
},
|
||||
"include": ["./manifest.ts", "./src/**/*"]
|
||||
}
|
||||
@@ -1,3 +1,8 @@
|
||||
# python-jobspy requires Python 3.10+ (wheels not published for 3.9 and below).
|
||||
python-jobspy
|
||||
#
|
||||
# Pinned to 1.1.82: scrape_jobs.py monkeypatches Glassdoor._fetch_jobs_page
|
||||
# to work around an upstream bug where any partial GraphQL error (even for
|
||||
# unrelated fields) discards a whole page of valid job data. If bumping this
|
||||
# version, re-check that method still matches before removing the pin.
|
||||
python-jobspy==1.1.82
|
||||
pandas
|
||||
|
||||
@@ -1,12 +1,81 @@
|
||||
import csv
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
from concurrent.futures import ThreadPoolExecutor, as_completed
|
||||
from pathlib import Path
|
||||
|
||||
import pandas as pd
|
||||
from jobspy import scrape_jobs
|
||||
from jobspy.exception import GlassdoorException
|
||||
from jobspy.glassdoor import Glassdoor
|
||||
from jobspy.glassdoor.util import get_cursor_for_page
|
||||
|
||||
PROGRESS_PREFIX = "JOBOPS_PROGRESS "
|
||||
_glassdoor_log = logging.getLogger("JobSpy:Glassdoor")
|
||||
|
||||
|
||||
def _patched_glassdoor_fetch_jobs_page(
|
||||
self, scraper_input, location_id, location_type, page_num, cursor
|
||||
):
|
||||
"""Replaces python-jobspy's Glassdoor._fetch_jobs_page (as of 1.1.82).
|
||||
|
||||
Upstream treats *any* `errors` entry in Glassdoor's GraphQL response as
|
||||
fatal and discards the whole page, even when the job listings we need
|
||||
parsed fine. In practice Glassdoor's backend intermittently (but
|
||||
consistently, as of writing) returns a partial error for an unrelated
|
||||
field — observed: `jobsPageSeoData` failing with
|
||||
`SUBREQUEST_HTTP_ERROR` / "dns error" on Glassdoor's own infra — while
|
||||
`data.jobListings.jobListings` still comes back with real results. The
|
||||
upstream check throws that good data away, so every Glassdoor run
|
||||
returns 0 jobs. This patch only bails out when the job listings payload
|
||||
itself is actually missing.
|
||||
"""
|
||||
jobs = []
|
||||
self.scraper_input = scraper_input
|
||||
try:
|
||||
payload = self._add_payload(location_id, location_type, page_num, cursor)
|
||||
response = self.session.post(
|
||||
f"{self.base_url}/graph",
|
||||
timeout_seconds=15,
|
||||
data=payload,
|
||||
)
|
||||
if response.status_code != 200:
|
||||
raise GlassdoorException(
|
||||
f"bad response status code: {response.status_code}"
|
||||
)
|
||||
res_json = response.json()[0]
|
||||
job_listings = res_json.get("data", {}).get("jobListings")
|
||||
if job_listings is None:
|
||||
raise ValueError("Error encountered in API response")
|
||||
if "errors" in res_json:
|
||||
_glassdoor_log.warning(
|
||||
"Glassdoor GraphQL returned partial errors for unrelated "
|
||||
"fields; continuing with the job listings that did parse: %s",
|
||||
res_json["errors"],
|
||||
)
|
||||
except (GlassdoorException, ValueError, Exception) as e:
|
||||
_glassdoor_log.error(f"Glassdoor: {str(e)}")
|
||||
return jobs, None
|
||||
|
||||
jobs_data = job_listings["jobListings"]
|
||||
|
||||
with ThreadPoolExecutor(max_workers=self.jobs_per_page) as executor:
|
||||
future_to_job_data = {
|
||||
executor.submit(self._process_job, job): job for job in jobs_data
|
||||
}
|
||||
for future in as_completed(future_to_job_data):
|
||||
try:
|
||||
job_post = future.result()
|
||||
if job_post:
|
||||
jobs.append(job_post)
|
||||
except Exception as exc:
|
||||
raise GlassdoorException(f"Glassdoor generated an exception: {exc}")
|
||||
|
||||
return jobs, get_cursor_for_page(job_listings["paginationCursors"], page_num + 1)
|
||||
|
||||
|
||||
Glassdoor._fetch_jobs_page = _patched_glassdoor_fetch_jobs_page
|
||||
COUNTRY_ALIASES = {
|
||||
"uk": "united kingdom",
|
||||
"united kingdom": "united kingdom",
|
||||
|
||||
@@ -6,11 +6,11 @@ import { createInterface } from "node:readline";
|
||||
import { fileURLToPath } from "node:url";
|
||||
import { resolveSearchCities } from "@shared/search-cities.js";
|
||||
import type { CreateJobInput, JobSource } from "@shared/types/jobs";
|
||||
import { normalizeIsRemote } from "@shared/work-arrangement.js";
|
||||
import {
|
||||
toNumberOrNull,
|
||||
toStringOrNull,
|
||||
} from "@shared/utils/type-conversion.js";
|
||||
import { normalizeIsRemote } from "@shared/work-arrangement.js";
|
||||
|
||||
const srcDir = dirname(fileURLToPath(import.meta.url));
|
||||
const EXTRACTOR_DIR = join(srcDir, "..");
|
||||
|
||||
@@ -0,0 +1,216 @@
|
||||
/**
|
||||
* Teamtailor public career sites — RSS feed per tenant.
|
||||
*
|
||||
* https://{company}.teamtailor.com/jobs.rss
|
||||
*/
|
||||
|
||||
import type {
|
||||
ExtractorManifest,
|
||||
ExtractorRunResult,
|
||||
} from "@shared/types/extractors";
|
||||
import type { CreateJobInput } from "@shared/types/jobs";
|
||||
|
||||
interface TeamtailorItem {
|
||||
title?: string;
|
||||
link?: string;
|
||||
guid?: string;
|
||||
description?: string;
|
||||
pubDate?: string;
|
||||
remoteStatus?: string;
|
||||
companyName?: string;
|
||||
location?: string;
|
||||
department?: string;
|
||||
}
|
||||
|
||||
function asString(value: unknown): string | undefined {
|
||||
if (typeof value !== "string") return undefined;
|
||||
const trimmed = value.trim();
|
||||
return trimmed ? trimmed : undefined;
|
||||
}
|
||||
|
||||
function readCompanies(raw: string | undefined): string[] {
|
||||
if (!raw) return [];
|
||||
try {
|
||||
const parsed = JSON.parse(raw);
|
||||
if (Array.isArray(parsed)) {
|
||||
return parsed
|
||||
.map((entry) => (typeof entry === "string" ? entry.trim() : ""))
|
||||
.filter(Boolean);
|
||||
}
|
||||
} catch {
|
||||
// fall through
|
||||
}
|
||||
return raw
|
||||
.split(/[\n,;|]+/)
|
||||
.map((entry) => entry.trim())
|
||||
.filter(Boolean);
|
||||
}
|
||||
|
||||
function xmlText(xml: string, tag: string): string | undefined {
|
||||
const pattern = new RegExp(`<${tag}[^>]*>([\\s\\S]*?)</${tag}>`, "i");
|
||||
const match = xml.match(pattern);
|
||||
if (!match?.[1]) return undefined;
|
||||
return (
|
||||
match[1].replace(/<!\[CDATA\[([\s\S]*?)\]\]>/g, "$1").trim() || undefined
|
||||
);
|
||||
}
|
||||
|
||||
function decodeHtmlEntities(value: string): string {
|
||||
return value
|
||||
.replace(/&/g, "&")
|
||||
.replace(/</g, "<")
|
||||
.replace(/>/g, ">")
|
||||
.replace(/"/g, '"')
|
||||
.replace(/'/g, "'");
|
||||
}
|
||||
|
||||
function stripHtml(html: string): string {
|
||||
return decodeHtmlEntities(html)
|
||||
.replace(/<[^>]+>/g, " ")
|
||||
.replace(/\s+/g, " ")
|
||||
.trim();
|
||||
}
|
||||
|
||||
function parseItems(xml: string): TeamtailorItem[] {
|
||||
const items: TeamtailorItem[] = [];
|
||||
const blocks = xml.match(/<item>([\s\S]*?)<\/item>/gi) ?? [];
|
||||
|
||||
for (const raw of blocks) {
|
||||
const block = raw.replace(/^<item>/i, "").replace(/<\/item>$/i, "");
|
||||
items.push({
|
||||
title: xmlText(block, "title"),
|
||||
link: xmlText(block, "link"),
|
||||
guid: xmlText(block, "guid"),
|
||||
description: xmlText(block, "description"),
|
||||
pubDate: xmlText(block, "pubDate"),
|
||||
remoteStatus: xmlText(block, "remoteStatus"),
|
||||
companyName: xmlText(block, "company_name"),
|
||||
location: xmlText(block, "location"),
|
||||
department: xmlText(block, "department"),
|
||||
});
|
||||
}
|
||||
|
||||
return items;
|
||||
}
|
||||
|
||||
function matchesTerm(item: TeamtailorItem, term: string): boolean {
|
||||
const lower = term.toLowerCase();
|
||||
if (item.title?.toLowerCase().includes(lower)) return true;
|
||||
if (item.department?.toLowerCase().includes(lower)) return true;
|
||||
const description = item.description ? stripHtml(item.description) : "";
|
||||
if (description.toLowerCase().includes(lower)) return true;
|
||||
return false;
|
||||
}
|
||||
|
||||
function mapJob(
|
||||
item: TeamtailorItem,
|
||||
companySlug: string,
|
||||
): CreateJobInput | null {
|
||||
const jobUrl = asString(item.link);
|
||||
if (!jobUrl) return null;
|
||||
|
||||
const employer =
|
||||
asString(item.companyName) ??
|
||||
companySlug
|
||||
.split(/[-_]/)
|
||||
.map((part) => part.charAt(0).toUpperCase() + part.slice(1))
|
||||
.join(" ");
|
||||
|
||||
const remoteStatus = asString(item.remoteStatus)?.toLowerCase();
|
||||
const isRemote =
|
||||
remoteStatus === "fully" ||
|
||||
remoteStatus === "hybrid" ||
|
||||
item.location?.toLowerCase().includes("remote") === true;
|
||||
|
||||
return {
|
||||
source: "teamtailor",
|
||||
sourceJobId: item.guid ?? jobUrl,
|
||||
title: asString(item.title) ?? "Unknown Title",
|
||||
employer,
|
||||
jobUrl,
|
||||
applicationLink: jobUrl,
|
||||
location: asString(item.location) ?? "Unknown",
|
||||
isRemote,
|
||||
datePosted: asString(item.pubDate),
|
||||
jobDescription: item.description ? stripHtml(item.description) : undefined,
|
||||
jobFunction: asString(item.department),
|
||||
};
|
||||
}
|
||||
|
||||
async function fetchCompanyFeed(company: string): Promise<TeamtailorItem[]> {
|
||||
const url = `https://${encodeURIComponent(company)}.teamtailor.com/jobs.rss?per_page=200`;
|
||||
const response = await fetch(url, {
|
||||
headers: {
|
||||
Accept: "application/rss+xml, application/xml, text/xml",
|
||||
"User-Agent": "JobOps/1.0",
|
||||
},
|
||||
});
|
||||
if (response.status === 404) return [];
|
||||
if (!response.ok) {
|
||||
throw new Error(
|
||||
`Teamtailor RSS for "${company}" failed with status ${response.status}`,
|
||||
);
|
||||
}
|
||||
return parseItems(await response.text());
|
||||
}
|
||||
|
||||
export const manifest: ExtractorManifest = {
|
||||
id: "teamtailor",
|
||||
displayName: "Teamtailor (ATS)",
|
||||
providesSources: ["teamtailor"],
|
||||
async run(context): Promise<ExtractorRunResult> {
|
||||
if (context.shouldCancel?.()) return { success: true, jobs: [] };
|
||||
|
||||
const companies = readCompanies(context.settings.teamtailorCompanies);
|
||||
if (companies.length === 0) {
|
||||
return {
|
||||
success: true,
|
||||
jobs: [],
|
||||
error:
|
||||
"No Teamtailor companies configured. Set TEAMTAILOR_COMPANIES or teamtailorCompanies (comma- or newline-separated subdomains).",
|
||||
};
|
||||
}
|
||||
|
||||
const terms = context.searchTerms.length > 0 ? context.searchTerms : [];
|
||||
const seen = new Set<string>();
|
||||
const out: CreateJobInput[] = [];
|
||||
|
||||
try {
|
||||
for (let i = 0; i < companies.length; i += 1) {
|
||||
if (context.shouldCancel?.()) break;
|
||||
const company = companies[i];
|
||||
|
||||
context.onProgress?.({
|
||||
phase: "list",
|
||||
termsProcessed: i,
|
||||
termsTotal: companies.length,
|
||||
currentUrl: company,
|
||||
detail: `Teamtailor: ${company} (${i + 1}/${companies.length})`,
|
||||
});
|
||||
|
||||
const items = await fetchCompanyFeed(company);
|
||||
for (const item of items) {
|
||||
if (
|
||||
terms.length > 0 &&
|
||||
!terms.some((term) => matchesTerm(item, term))
|
||||
) {
|
||||
continue;
|
||||
}
|
||||
const mapped = mapJob(item, company);
|
||||
if (!mapped) continue;
|
||||
const key = mapped.sourceJobId || mapped.jobUrl;
|
||||
if (seen.has(key)) continue;
|
||||
seen.add(key);
|
||||
out.push(mapped);
|
||||
}
|
||||
}
|
||||
|
||||
return { success: true, jobs: out };
|
||||
} catch (error) {
|
||||
const message = error instanceof Error ? error.message : "Unknown error";
|
||||
return { success: false, jobs: out, error: message };
|
||||
}
|
||||
},
|
||||
};
|
||||
|
||||
export default manifest;
|
||||
@@ -0,0 +1,17 @@
|
||||
{
|
||||
"name": "teamtailor-extractor",
|
||||
"version": "0.0.1",
|
||||
"type": "module",
|
||||
"description": "Teamtailor public RSS career-site extractor",
|
||||
"main": "manifest.ts",
|
||||
"dependencies": {
|
||||
"job-ops-shared": "^1.0.0"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@types/node": "^24.0.0",
|
||||
"typescript": "~5.9.0"
|
||||
},
|
||||
"scripts": {
|
||||
"check:types": "tsc --noEmit"
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,17 @@
|
||||
{
|
||||
"compilerOptions": {
|
||||
"module": "ESNext",
|
||||
"moduleResolution": "bundler",
|
||||
"target": "ES2022",
|
||||
"outDir": "dist",
|
||||
"strict": true,
|
||||
"noUnusedLocals": false,
|
||||
"lib": ["ES2022", "DOM"],
|
||||
"types": ["node"],
|
||||
"baseUrl": ".",
|
||||
"paths": {
|
||||
"@shared/*": ["../../shared/src/*"]
|
||||
}
|
||||
},
|
||||
"include": ["./manifest.ts", "./src/**/*"]
|
||||
}
|
||||
@@ -0,0 +1,259 @@
|
||||
/**
|
||||
* TestDevJobs — QA-focused board (Gridsome SSR HTML + embedded job state).
|
||||
*
|
||||
* https://testdevjobs.com/software-testing-jobs/
|
||||
*/
|
||||
|
||||
import type {
|
||||
ExtractorManifest,
|
||||
ExtractorRunResult,
|
||||
} from "@shared/types/extractors";
|
||||
import type { CreateJobInput } from "@shared/types/jobs";
|
||||
|
||||
const ORIGIN = "https://testdevjobs.com";
|
||||
const LIST_PATH = "/software-testing-jobs/";
|
||||
|
||||
interface ListingJob {
|
||||
path: string;
|
||||
title: string;
|
||||
employer: string;
|
||||
datePosted?: string;
|
||||
location?: string;
|
||||
jobType?: string;
|
||||
}
|
||||
|
||||
interface TestDevJobState {
|
||||
id?: string;
|
||||
jobTitle?: string;
|
||||
path?: string;
|
||||
jobDescription?: string;
|
||||
joblocation?: string;
|
||||
jobType?: string;
|
||||
salary?: string;
|
||||
isRemote?: boolean;
|
||||
applyLink?: string;
|
||||
companyName?: string;
|
||||
jobPosted?: string;
|
||||
}
|
||||
|
||||
function asString(value: unknown): string | undefined {
|
||||
if (typeof value !== "string") return undefined;
|
||||
const trimmed = value.trim();
|
||||
return trimmed ? trimmed : undefined;
|
||||
}
|
||||
|
||||
function decodeHtmlEntities(value: string): string {
|
||||
return value
|
||||
.replace(/&/g, "&")
|
||||
.replace(/</g, "<")
|
||||
.replace(/>/g, ">")
|
||||
.replace(/"/g, '"')
|
||||
.replace(/'/g, "'");
|
||||
}
|
||||
|
||||
function stripHtml(html: string): string {
|
||||
return decodeHtmlEntities(html)
|
||||
.replace(/<[^>]+>/g, " ")
|
||||
.replace(/\s+/g, " ")
|
||||
.trim();
|
||||
}
|
||||
|
||||
function readMaxPages(raw: string | undefined): number {
|
||||
const parsed = raw ? Number.parseInt(raw, 10) : Number.NaN;
|
||||
if (!Number.isFinite(parsed)) return 3;
|
||||
return Math.min(Math.max(parsed, 1), 20);
|
||||
}
|
||||
|
||||
function parseListingJobs(html: string): ListingJob[] {
|
||||
const jobs: ListingJob[] = [];
|
||||
const blocks = html.match(
|
||||
/<div class="job-tile-wrapper[\s\S]*?<\/div>\s*<\/div>\s*<\/div>/g,
|
||||
);
|
||||
|
||||
for (const block of blocks ?? []) {
|
||||
const pathMatch = block.match(/href="(\/job\/[^"]+)"/);
|
||||
const titleMatch = block.match(/class="ml-5 jobtitle[^"]*"[^>]*>([^<]+)</);
|
||||
const employerMatch = block.match(
|
||||
/class="mb-0 comptitle[^"]*"[^>]*>([^<]+)</,
|
||||
);
|
||||
const dateMatch = block.match(/itemprop="datePosted"[^>]*>([^<]+)</);
|
||||
const locationMatch = block.match(
|
||||
/itemprop="addressLocality"[^>]*>([^<]+)</,
|
||||
);
|
||||
const typeMatch = block.match(/itemprop="employmentType"[^>]*>([^<]+)</);
|
||||
|
||||
const path = pathMatch?.[1];
|
||||
const title = titleMatch?.[1]?.trim();
|
||||
if (!path || !title) continue;
|
||||
|
||||
jobs.push({
|
||||
path,
|
||||
title: decodeHtmlEntities(title),
|
||||
employer: decodeHtmlEntities(
|
||||
employerMatch?.[1]?.trim() ?? "Unknown Employer",
|
||||
),
|
||||
datePosted: dateMatch?.[1]?.trim(),
|
||||
location: locationMatch?.[1]?.trim(),
|
||||
jobType: typeMatch?.[1]?.trim(),
|
||||
});
|
||||
}
|
||||
|
||||
return jobs;
|
||||
}
|
||||
|
||||
function parseInitialState(html: string): TestDevJobState | null {
|
||||
const match = html.match(/window\.__INITIAL_STATE__=({[\s\S]*?});/);
|
||||
if (!match?.[1]) return null;
|
||||
|
||||
try {
|
||||
const parsed = JSON.parse(match[1]) as {
|
||||
data?: { job?: TestDevJobState };
|
||||
};
|
||||
return parsed.data?.job ?? null;
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
function matchesTerm(values: string[], term: string): boolean {
|
||||
const lower = term.toLowerCase();
|
||||
return values.some((value) => value.toLowerCase().includes(lower));
|
||||
}
|
||||
|
||||
async function fetchText(url: string): Promise<string> {
|
||||
const response = await fetch(url, {
|
||||
headers: { Accept: "text/html", "User-Agent": "JobOps/1.0" },
|
||||
});
|
||||
if (!response.ok) {
|
||||
throw new Error(
|
||||
`TestDevJobs request failed (${response.status}) for ${url}`,
|
||||
);
|
||||
}
|
||||
return response.text();
|
||||
}
|
||||
|
||||
function listingUrl(page: number): string {
|
||||
if (page <= 1) return `${ORIGIN}${LIST_PATH}`;
|
||||
return `${ORIGIN}${LIST_PATH}${page}/`;
|
||||
}
|
||||
|
||||
export const manifest: ExtractorManifest = {
|
||||
id: "testdevjobs",
|
||||
displayName: "TestDevJobs",
|
||||
providesSources: ["testdevjobs"],
|
||||
async run(context): Promise<ExtractorRunResult> {
|
||||
if (context.shouldCancel?.()) return { success: true, jobs: [] };
|
||||
|
||||
const maxJobs = context.settings.testdevjobsMaxJobsPerTerm
|
||||
? Number.parseInt(context.settings.testdevjobsMaxJobsPerTerm, 10)
|
||||
: 100;
|
||||
const cap = Number.isFinite(maxJobs)
|
||||
? Math.min(Math.max(maxJobs, 1), 500)
|
||||
: 100;
|
||||
const maxPages = readMaxPages(context.settings.testdevjobsMaxPages);
|
||||
const terms = context.searchTerms.length > 0 ? context.searchTerms : [];
|
||||
const enrichDetails = true;
|
||||
|
||||
const seen = new Set<string>();
|
||||
const out: CreateJobInput[] = [];
|
||||
|
||||
try {
|
||||
for (let page = 1; page <= maxPages; page += 1) {
|
||||
if (context.shouldCancel?.()) break;
|
||||
if (out.length >= cap) break;
|
||||
|
||||
const url = listingUrl(page);
|
||||
context.onProgress?.({
|
||||
phase: "list",
|
||||
termsProcessed: page - 1,
|
||||
termsTotal: maxPages,
|
||||
currentUrl: url,
|
||||
detail: `TestDevJobs: listing page ${page}/${maxPages}`,
|
||||
});
|
||||
|
||||
const html = await fetchText(url);
|
||||
const listings = parseListingJobs(html);
|
||||
if (listings.length === 0) break;
|
||||
|
||||
for (const listing of listings) {
|
||||
if (context.shouldCancel?.()) break;
|
||||
if (out.length >= cap) break;
|
||||
|
||||
const haystack = [
|
||||
listing.title,
|
||||
listing.employer,
|
||||
listing.location ?? "",
|
||||
listing.jobType ?? "",
|
||||
];
|
||||
if (
|
||||
terms.length > 0 &&
|
||||
!terms.some((term) => matchesTerm(haystack, term))
|
||||
) {
|
||||
continue;
|
||||
}
|
||||
|
||||
let jobUrl = `${ORIGIN}${listing.path}`;
|
||||
let applicationLink = jobUrl;
|
||||
let jobDescription: string | undefined;
|
||||
let location = listing.location;
|
||||
let jobType = listing.jobType;
|
||||
let datePosted = listing.datePosted;
|
||||
let isRemote = listing.location?.toLowerCase().includes("remote");
|
||||
|
||||
if (enrichDetails) {
|
||||
try {
|
||||
const detailHtml = await fetchText(jobUrl);
|
||||
const state = parseInitialState(detailHtml);
|
||||
if (state) {
|
||||
if (state.path) jobUrl = `${ORIGIN}${state.path}`;
|
||||
if (state.applyLink) applicationLink = state.applyLink;
|
||||
if (state.jobDescription) jobDescription = state.jobDescription;
|
||||
if (state.joblocation) location = state.joblocation;
|
||||
if (state.jobType) jobType = state.jobType;
|
||||
if (state.jobPosted) datePosted = state.jobPosted;
|
||||
if (typeof state.isRemote === "boolean")
|
||||
isRemote = state.isRemote;
|
||||
}
|
||||
} catch {
|
||||
// keep listing row when detail fetch fails
|
||||
}
|
||||
}
|
||||
|
||||
const key = listing.path;
|
||||
if (seen.has(key)) continue;
|
||||
seen.add(key);
|
||||
|
||||
out.push({
|
||||
source: "testdevjobs",
|
||||
sourceJobId: key.split("/").filter(Boolean).pop(),
|
||||
title: listing.title,
|
||||
employer: listing.employer,
|
||||
jobUrl,
|
||||
applicationLink,
|
||||
location: location ?? "Unknown",
|
||||
isRemote,
|
||||
datePosted,
|
||||
jobDescription,
|
||||
jobType,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
context.onProgress?.({
|
||||
phase: "list",
|
||||
termsProcessed: maxPages,
|
||||
termsTotal: maxPages,
|
||||
currentUrl: `${ORIGIN}${LIST_PATH}`,
|
||||
jobPagesProcessed: out.length,
|
||||
detail: `TestDevJobs: ${out.length} jobs`,
|
||||
});
|
||||
|
||||
return { success: true, jobs: out };
|
||||
} catch (error) {
|
||||
const message = error instanceof Error ? error.message : "Unknown error";
|
||||
return { success: false, jobs: out, error: message };
|
||||
}
|
||||
},
|
||||
};
|
||||
|
||||
export default manifest;
|
||||
@@ -0,0 +1,17 @@
|
||||
{
|
||||
"name": "testdevjobs-extractor",
|
||||
"version": "0.0.1",
|
||||
"type": "module",
|
||||
"description": "TestDevJobs QA board extractor",
|
||||
"main": "manifest.ts",
|
||||
"dependencies": {
|
||||
"job-ops-shared": "^1.0.0"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@types/node": "^24.0.0",
|
||||
"typescript": "~5.9.0"
|
||||
},
|
||||
"scripts": {
|
||||
"check:types": "tsc --noEmit"
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,17 @@
|
||||
{
|
||||
"compilerOptions": {
|
||||
"module": "ESNext",
|
||||
"moduleResolution": "bundler",
|
||||
"target": "ES2022",
|
||||
"outDir": "dist",
|
||||
"strict": true,
|
||||
"noUnusedLocals": false,
|
||||
"lib": ["ES2022", "DOM"],
|
||||
"types": ["node"],
|
||||
"baseUrl": ".",
|
||||
"paths": {
|
||||
"@shared/*": ["../../shared/src/*"]
|
||||
}
|
||||
},
|
||||
"include": ["./manifest.ts", "./src/**/*"]
|
||||
}
|
||||
@@ -0,0 +1,49 @@
|
||||
import type {
|
||||
ExtractorManifest,
|
||||
ExtractorRunResult,
|
||||
} from "@shared/types/extractors";
|
||||
import { runWellfound } from "./src/run.js";
|
||||
|
||||
export const manifest: ExtractorManifest = {
|
||||
id: "wellfound",
|
||||
displayName: "Wellfound",
|
||||
providesSources: ["wellfound"],
|
||||
async run(context): Promise<ExtractorRunResult> {
|
||||
if (context.shouldCancel?.()) return { success: true, jobs: [] };
|
||||
|
||||
const parsedMax = context.settings.wellfoundMaxJobsPerTerm
|
||||
? Number.parseInt(context.settings.wellfoundMaxJobsPerTerm, 10)
|
||||
: Number.NaN;
|
||||
const maxJobs = Number.isFinite(parsedMax) ? Math.max(1, parsedMax) : 50;
|
||||
|
||||
context.onProgress?.({
|
||||
phase: "list",
|
||||
termsProcessed: 0,
|
||||
termsTotal: 1,
|
||||
currentUrl: "https://wellfound.com/jobs",
|
||||
detail: "Wellfound: launching browser scrape",
|
||||
});
|
||||
|
||||
const result = await runWellfound({
|
||||
searchTerms: context.searchTerms,
|
||||
maxJobs,
|
||||
});
|
||||
|
||||
if (!result.success) {
|
||||
return { success: false, jobs: [], error: result.error };
|
||||
}
|
||||
|
||||
context.onProgress?.({
|
||||
phase: "list",
|
||||
termsProcessed: 1,
|
||||
termsTotal: 1,
|
||||
currentUrl: "https://wellfound.com/jobs",
|
||||
jobPagesProcessed: result.jobs.length,
|
||||
detail: `Wellfound: ${result.jobs.length} jobs`,
|
||||
});
|
||||
|
||||
return { success: true, jobs: result.jobs };
|
||||
},
|
||||
};
|
||||
|
||||
export default manifest;
|
||||
@@ -0,0 +1,22 @@
|
||||
{
|
||||
"name": "wellfound-extractor",
|
||||
"version": "0.0.1",
|
||||
"type": "module",
|
||||
"description": "Wellfound startup job board extractor (browser-backed)",
|
||||
"main": "manifest.ts",
|
||||
"dependencies": {
|
||||
"camoufox-js": "^0.8.0",
|
||||
"job-ops-shared": "^1.0.0",
|
||||
"playwright": "^1.57.0",
|
||||
"tsx": "^4.4.0"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@types/node": "^24.0.0",
|
||||
"typescript": "~5.9.0"
|
||||
},
|
||||
"scripts": {
|
||||
"start": "tsx src/main.ts",
|
||||
"check:types": "tsc --noEmit",
|
||||
"get-binaries": "camoufox-js fetch"
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,171 @@
|
||||
import { mkdir, writeFile } from "node:fs/promises";
|
||||
import { dirname, join } from "node:path";
|
||||
import { fileURLToPath } from "node:url";
|
||||
import { launchOptions } from "camoufox-js";
|
||||
import { firefox } from "playwright";
|
||||
|
||||
const __dirname = dirname(fileURLToPath(import.meta.url));
|
||||
const OUTPUT_PATH = join(__dirname, "../storage/jobs.json");
|
||||
|
||||
interface ScrapedJob {
|
||||
title: string;
|
||||
employer: string;
|
||||
jobUrl: string;
|
||||
location?: string;
|
||||
isRemote?: boolean;
|
||||
}
|
||||
|
||||
function slugifyTerm(term: string): string {
|
||||
return term
|
||||
.trim()
|
||||
.toLowerCase()
|
||||
.replace(/[^a-z0-9]+/g, "-")
|
||||
.replace(/^-+|-+$/g, "");
|
||||
}
|
||||
|
||||
function parseTerms(raw: string | undefined): string[] {
|
||||
if (!raw) return ["software-engineer"];
|
||||
try {
|
||||
const parsed = JSON.parse(raw) as unknown;
|
||||
if (Array.isArray(parsed)) {
|
||||
return parsed.map((entry) => slugifyTerm(String(entry))).filter(Boolean);
|
||||
}
|
||||
} catch {
|
||||
// fall through
|
||||
}
|
||||
return raw
|
||||
.split(/[\n|,]+/)
|
||||
.map((entry) => slugifyTerm(entry))
|
||||
.filter(Boolean);
|
||||
}
|
||||
|
||||
function buildSearchUrl(roleSlug: string): string {
|
||||
return `https://wellfound.com/role/l/${encodeURIComponent(roleSlug)}/remote`;
|
||||
}
|
||||
|
||||
async function scrapeRole(
|
||||
roleSlug: string,
|
||||
maxJobs: number,
|
||||
): Promise<ScrapedJob[]> {
|
||||
const browser = await firefox.launch(await launchOptions({ headless: true }));
|
||||
const page = await browser.newPage();
|
||||
|
||||
try {
|
||||
await page.goto(buildSearchUrl(roleSlug), {
|
||||
waitUntil: "domcontentloaded",
|
||||
timeout: 90_000,
|
||||
});
|
||||
await page.waitForTimeout(4_000);
|
||||
|
||||
const jobs = await page.evaluate(() => {
|
||||
const results: Array<{
|
||||
title: string;
|
||||
employer: string;
|
||||
jobUrl: string;
|
||||
location?: string;
|
||||
}> = [];
|
||||
|
||||
const scripts = Array.from(
|
||||
document.querySelectorAll('script[type="application/ld+json"]'),
|
||||
);
|
||||
for (const script of scripts) {
|
||||
try {
|
||||
const data = JSON.parse(script.textContent ?? "null") as unknown;
|
||||
const rows = Array.isArray(data) ? data : [data];
|
||||
for (const row of rows) {
|
||||
if (!row || typeof row !== "object") continue;
|
||||
const record = row as Record<string, unknown>;
|
||||
if (record["@type"] !== "JobPosting") continue;
|
||||
const title = String(record.title ?? "").trim();
|
||||
const jobUrl = String(record.url ?? record.sameAs ?? "").trim();
|
||||
const org = record.hiringOrganization as
|
||||
| Record<string, unknown>
|
||||
| undefined;
|
||||
const employer = String(org?.name ?? "Unknown Employer").trim();
|
||||
const location = String(
|
||||
(record.jobLocation as Record<string, unknown> | undefined)
|
||||
?.name ??
|
||||
record.jobLocation ??
|
||||
"",
|
||||
).trim();
|
||||
if (!title || !jobUrl) continue;
|
||||
results.push({
|
||||
title,
|
||||
employer,
|
||||
jobUrl,
|
||||
location: location || undefined,
|
||||
});
|
||||
}
|
||||
} catch {
|
||||
// ignore malformed JSON-LD blocks
|
||||
}
|
||||
}
|
||||
|
||||
if (results.length === 0) {
|
||||
for (const anchor of Array.from(
|
||||
document.querySelectorAll('a[href*="/job/"], a[href*="/jobs/"]'),
|
||||
)) {
|
||||
const href = anchor.getAttribute("href") ?? "";
|
||||
if (!href.includes("/job")) continue;
|
||||
const jobUrl = href.startsWith("http")
|
||||
? href
|
||||
: `https://wellfound.com${href.startsWith("/") ? href : `/${href}`}`;
|
||||
const title = (anchor.textContent ?? "").trim();
|
||||
if (!title || title.length < 4) continue;
|
||||
results.push({
|
||||
title,
|
||||
employer: "Unknown Employer",
|
||||
jobUrl,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
return results;
|
||||
});
|
||||
|
||||
const seen = new Set<string>();
|
||||
const out: ScrapedJob[] = [];
|
||||
for (const job of jobs) {
|
||||
if (out.length >= maxJobs) break;
|
||||
if (seen.has(job.jobUrl)) continue;
|
||||
seen.add(job.jobUrl);
|
||||
out.push({
|
||||
...job,
|
||||
isRemote: true,
|
||||
});
|
||||
}
|
||||
|
||||
return out;
|
||||
} finally {
|
||||
await browser.close();
|
||||
}
|
||||
}
|
||||
|
||||
async function main(): Promise<void> {
|
||||
const terms = parseTerms(process.env.WELLFOUND_SEARCH_TERMS);
|
||||
const maxJobs = Number.parseInt(process.env.WELLFOUND_MAX_JOBS ?? "50", 10);
|
||||
const cap = Number.isFinite(maxJobs)
|
||||
? Math.max(1, Math.min(maxJobs, 200))
|
||||
: 50;
|
||||
|
||||
const all: ScrapedJob[] = [];
|
||||
const seen = new Set<string>();
|
||||
|
||||
for (const term of terms) {
|
||||
const rows = await scrapeRole(term, cap);
|
||||
for (const row of rows) {
|
||||
if (all.length >= cap) break;
|
||||
if (seen.has(row.jobUrl)) continue;
|
||||
seen.add(row.jobUrl);
|
||||
all.push(row);
|
||||
}
|
||||
}
|
||||
|
||||
await mkdir(dirname(OUTPUT_PATH), { recursive: true });
|
||||
await writeFile(OUTPUT_PATH, JSON.stringify(all, null, 2), "utf-8");
|
||||
}
|
||||
|
||||
main().catch((error) => {
|
||||
console.error(error instanceof Error ? error.message : String(error));
|
||||
process.exit(1);
|
||||
});
|
||||
@@ -0,0 +1,139 @@
|
||||
import { spawn, spawnSync } from "node:child_process";
|
||||
import { readFile } from "node:fs/promises";
|
||||
import { createRequire } from "node:module";
|
||||
import { dirname, join } from "node:path";
|
||||
import { fileURLToPath } from "node:url";
|
||||
import { envForExtractorSubprocess } from "@shared/extractor-subprocess-env.js";
|
||||
import type { CreateJobInput } from "@shared/types/jobs";
|
||||
|
||||
const srcDir = dirname(fileURLToPath(import.meta.url));
|
||||
const EXTRACTOR_DIR = join(srcDir, "..");
|
||||
const OUTPUT_PATH = join(EXTRACTOR_DIR, "storage/jobs.json");
|
||||
const require = createRequire(import.meta.url);
|
||||
const TSX_CLI_PATH = (() => {
|
||||
try {
|
||||
return require.resolve("tsx/dist/cli.mjs");
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
})();
|
||||
|
||||
interface ScrapedJob {
|
||||
title?: string;
|
||||
employer?: string;
|
||||
jobUrl?: string;
|
||||
location?: string;
|
||||
isRemote?: boolean;
|
||||
}
|
||||
|
||||
export interface RunWellfoundOptions {
|
||||
searchTerms?: string[];
|
||||
maxJobs?: number;
|
||||
}
|
||||
|
||||
export interface WellfoundResult {
|
||||
success: boolean;
|
||||
jobs: CreateJobInput[];
|
||||
error?: string;
|
||||
}
|
||||
|
||||
function canRunNpmCommand(): boolean {
|
||||
const result = spawnSync("npm", ["--version"], { stdio: "ignore" });
|
||||
return !result.error && result.status === 0;
|
||||
}
|
||||
|
||||
function mapJob(row: ScrapedJob): CreateJobInput | null {
|
||||
const jobUrl = row.jobUrl?.trim();
|
||||
if (!jobUrl) return null;
|
||||
return {
|
||||
source: "wellfound",
|
||||
sourceJobId: jobUrl.split("/").filter(Boolean).pop(),
|
||||
title: row.title?.trim() || "Unknown Title",
|
||||
employer: row.employer?.trim() || "Unknown Employer",
|
||||
jobUrl,
|
||||
applicationLink: jobUrl,
|
||||
location: row.location?.trim() || "Remote",
|
||||
isRemote: row.isRemote ?? true,
|
||||
};
|
||||
}
|
||||
|
||||
export async function runWellfound(
|
||||
options: RunWellfoundOptions = {},
|
||||
): Promise<WellfoundResult> {
|
||||
const searchTerms =
|
||||
options.searchTerms && options.searchTerms.length > 0
|
||||
? options.searchTerms
|
||||
: ["software engineer"];
|
||||
const maxJobs = options.maxJobs ?? 50;
|
||||
|
||||
const useNpmCommand = canRunNpmCommand();
|
||||
if (!TSX_CLI_PATH && !useNpmCommand) {
|
||||
return {
|
||||
success: false,
|
||||
jobs: [],
|
||||
error: "Unable to execute Wellfound extractor (npm/tsx unavailable)",
|
||||
};
|
||||
}
|
||||
|
||||
try {
|
||||
await new Promise<void>((resolve, reject) => {
|
||||
const extractorEnv = envForExtractorSubprocess({
|
||||
...process.env,
|
||||
WELLFOUND_SEARCH_TERMS: JSON.stringify(searchTerms),
|
||||
WELLFOUND_MAX_JOBS: String(maxJobs),
|
||||
});
|
||||
|
||||
const child = TSX_CLI_PATH
|
||||
? spawn(process.execPath, [TSX_CLI_PATH, join(srcDir, "main.ts")], {
|
||||
cwd: EXTRACTOR_DIR,
|
||||
env: extractorEnv,
|
||||
stdio: ["ignore", "pipe", "pipe"],
|
||||
})
|
||||
: spawn("npx", ["tsx", join(srcDir, "main.ts")], {
|
||||
cwd: EXTRACTOR_DIR,
|
||||
env: extractorEnv,
|
||||
stdio: ["ignore", "pipe", "pipe"],
|
||||
});
|
||||
|
||||
let stderr = "";
|
||||
child.stderr?.on("data", (chunk: Buffer) => {
|
||||
stderr += chunk.toString();
|
||||
});
|
||||
|
||||
child.on("error", reject);
|
||||
child.on("close", (code) => {
|
||||
if (code === 0) {
|
||||
resolve();
|
||||
return;
|
||||
}
|
||||
reject(
|
||||
new Error(
|
||||
stderr.trim() ||
|
||||
`Wellfound browser scrape exited with code ${code}`,
|
||||
),
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
const raw = await readFile(OUTPUT_PATH, "utf-8");
|
||||
const parsed = JSON.parse(raw) as unknown;
|
||||
const rows = Array.isArray(parsed) ? parsed : [];
|
||||
|
||||
const jobs: CreateJobInput[] = [];
|
||||
const seen = new Set<string>();
|
||||
for (const value of rows) {
|
||||
if (!value || typeof value !== "object" || Array.isArray(value)) continue;
|
||||
const mapped = mapJob(value as ScrapedJob);
|
||||
if (!mapped) continue;
|
||||
const key = mapped.jobUrl;
|
||||
if (seen.has(key)) continue;
|
||||
seen.add(key);
|
||||
jobs.push(mapped);
|
||||
}
|
||||
|
||||
return { success: true, jobs };
|
||||
} catch (error) {
|
||||
const message = error instanceof Error ? error.message : "Unknown error";
|
||||
return { success: false, jobs: [], error: message };
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,17 @@
|
||||
{
|
||||
"compilerOptions": {
|
||||
"module": "ESNext",
|
||||
"moduleResolution": "bundler",
|
||||
"target": "ES2022",
|
||||
"outDir": "dist",
|
||||
"strict": true,
|
||||
"noUnusedLocals": false,
|
||||
"lib": ["ES2022", "DOM"],
|
||||
"types": ["node"],
|
||||
"baseUrl": ".",
|
||||
"paths": {
|
||||
"@shared/*": ["../../shared/src/*"]
|
||||
}
|
||||
},
|
||||
"include": ["./manifest.ts", "./src/**/*"]
|
||||
}
|
||||
@@ -0,0 +1,149 @@
|
||||
/**
|
||||
* Working Nomads — public JSON feed.
|
||||
*
|
||||
* https://www.workingnomads.com/api/exposed_jobs/
|
||||
*/
|
||||
|
||||
import type {
|
||||
ExtractorManifest,
|
||||
ExtractorRunResult,
|
||||
} from "@shared/types/extractors";
|
||||
import type { CreateJobInput } from "@shared/types/jobs";
|
||||
|
||||
const API_URL = "https://www.workingnomads.com/api/exposed_jobs/";
|
||||
|
||||
interface WorkingNomadsJob {
|
||||
url?: string;
|
||||
title?: string;
|
||||
description?: string;
|
||||
company_name?: string;
|
||||
category_name?: string;
|
||||
tags?: string;
|
||||
location?: string;
|
||||
pub_date?: string;
|
||||
}
|
||||
|
||||
function asString(value: unknown): string | undefined {
|
||||
if (typeof value !== "string") return undefined;
|
||||
const trimmed = value.trim();
|
||||
return trimmed ? trimmed : undefined;
|
||||
}
|
||||
|
||||
function stripHtml(html: string): string {
|
||||
return html
|
||||
.replace(/<[^>]+>/g, " ")
|
||||
.replace(/ /g, " ")
|
||||
.replace(/&/g, "&")
|
||||
.replace(/</g, "<")
|
||||
.replace(/>/g, ">")
|
||||
.replace(/\s+/g, " ")
|
||||
.trim();
|
||||
}
|
||||
|
||||
function matchesTerm(job: WorkingNomadsJob, term: string): boolean {
|
||||
const lower = term.toLowerCase();
|
||||
if (job.title?.toLowerCase().includes(lower)) return true;
|
||||
if (job.company_name?.toLowerCase().includes(lower)) return true;
|
||||
if (job.category_name?.toLowerCase().includes(lower)) return true;
|
||||
if (job.tags?.toLowerCase().includes(lower)) return true;
|
||||
const description = job.description ? stripHtml(job.description) : "";
|
||||
if (description.toLowerCase().includes(lower)) return true;
|
||||
return false;
|
||||
}
|
||||
|
||||
function mapJob(raw: WorkingNomadsJob): CreateJobInput | null {
|
||||
const jobUrl = asString(raw.url);
|
||||
if (!jobUrl) return null;
|
||||
|
||||
const title = asString(raw.title) ?? "Unknown Title";
|
||||
const employer = asString(raw.company_name) ?? "Unknown Employer";
|
||||
const location = asString(raw.location) ?? "Remote";
|
||||
const description = raw.description ? stripHtml(raw.description) : undefined;
|
||||
|
||||
return {
|
||||
source: "workingnomads",
|
||||
sourceJobId: jobUrl.split("/").filter(Boolean).pop(),
|
||||
title,
|
||||
employer,
|
||||
jobUrl,
|
||||
applicationLink: jobUrl,
|
||||
location,
|
||||
isRemote: true,
|
||||
datePosted: asString(raw.pub_date),
|
||||
jobDescription: description,
|
||||
companyIndustry: asString(raw.category_name),
|
||||
disciplines: asString(raw.tags),
|
||||
};
|
||||
}
|
||||
|
||||
export const manifest: ExtractorManifest = {
|
||||
id: "workingnomads",
|
||||
displayName: "Working Nomads",
|
||||
providesSources: ["workingnomads"],
|
||||
async run(context): Promise<ExtractorRunResult> {
|
||||
if (context.shouldCancel?.()) return { success: true, jobs: [] };
|
||||
|
||||
const maxJobs = context.settings.workingnomadsMaxJobsPerTerm
|
||||
? Number.parseInt(context.settings.workingnomadsMaxJobsPerTerm, 10)
|
||||
: 100;
|
||||
const cap = Number.isFinite(maxJobs)
|
||||
? Math.min(Math.max(maxJobs, 1), 500)
|
||||
: 100;
|
||||
|
||||
const terms = context.searchTerms.length > 0 ? context.searchTerms : [];
|
||||
|
||||
context.onProgress?.({
|
||||
phase: "list",
|
||||
termsProcessed: 0,
|
||||
termsTotal: 1,
|
||||
currentUrl: API_URL,
|
||||
detail: "Working Nomads: fetching exposed_jobs API",
|
||||
});
|
||||
|
||||
try {
|
||||
const response = await fetch(API_URL, {
|
||||
headers: { Accept: "application/json", "User-Agent": "JobOps/1.0" },
|
||||
});
|
||||
if (!response.ok) {
|
||||
throw new Error(
|
||||
`Working Nomads request failed with status ${response.status}`,
|
||||
);
|
||||
}
|
||||
|
||||
const body = (await response.json()) as unknown;
|
||||
const rows = Array.isArray(body) ? body : [];
|
||||
|
||||
const seen = new Set<string>();
|
||||
const out: CreateJobInput[] = [];
|
||||
|
||||
for (const row of rows as WorkingNomadsJob[]) {
|
||||
if (out.length >= cap) break;
|
||||
if (terms.length > 0 && !terms.some((term) => matchesTerm(row, term))) {
|
||||
continue;
|
||||
}
|
||||
const mapped = mapJob(row);
|
||||
if (!mapped) continue;
|
||||
const key = mapped.sourceJobId || mapped.jobUrl;
|
||||
if (seen.has(key)) continue;
|
||||
seen.add(key);
|
||||
out.push(mapped);
|
||||
}
|
||||
|
||||
context.onProgress?.({
|
||||
phase: "list",
|
||||
termsProcessed: 1,
|
||||
termsTotal: 1,
|
||||
currentUrl: API_URL,
|
||||
jobPagesProcessed: out.length,
|
||||
detail: `Working Nomads: ${out.length} matched (${rows.length} total)`,
|
||||
});
|
||||
|
||||
return { success: true, jobs: out };
|
||||
} catch (error) {
|
||||
const message = error instanceof Error ? error.message : "Unknown error";
|
||||
return { success: false, jobs: [], error: message };
|
||||
}
|
||||
},
|
||||
};
|
||||
|
||||
export default manifest;
|
||||
@@ -0,0 +1,17 @@
|
||||
{
|
||||
"name": "workingnomads-extractor",
|
||||
"version": "0.0.1",
|
||||
"type": "module",
|
||||
"description": "Working Nomads public JSON feed extractor",
|
||||
"main": "manifest.ts",
|
||||
"dependencies": {
|
||||
"job-ops-shared": "^1.0.0"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@types/node": "^24.0.0",
|
||||
"typescript": "~5.9.0"
|
||||
},
|
||||
"scripts": {
|
||||
"check:types": "tsc --noEmit"
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,17 @@
|
||||
{
|
||||
"compilerOptions": {
|
||||
"module": "ESNext",
|
||||
"moduleResolution": "bundler",
|
||||
"target": "ES2022",
|
||||
"outDir": "dist",
|
||||
"strict": true,
|
||||
"noUnusedLocals": false,
|
||||
"lib": ["ES2022", "DOM"],
|
||||
"types": ["node"],
|
||||
"baseUrl": ".",
|
||||
"paths": {
|
||||
"@shared/*": ["../../shared/src/*"]
|
||||
}
|
||||
},
|
||||
"include": ["./manifest.ts", "./src/**/*"]
|
||||
}
|
||||
@@ -9,7 +9,6 @@
|
||||
<link rel="preconnect" href="https://fonts.googleapis.com">
|
||||
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
|
||||
<link href="https://fonts.googleapis.com/css2?family=Inter:wght@400;500;600;700&display=swap" rel="stylesheet">
|
||||
<script defer src="https://umami.dakheera47.com/script.js" data-website-id="0dc42ed1-87c3-4ac0-9409-5a9b9588fe66"></script>
|
||||
</head>
|
||||
<body>
|
||||
<div id="root"></div>
|
||||
|
||||
@@ -91,7 +91,7 @@ describe("App demo banner", () => {
|
||||
});
|
||||
|
||||
render(
|
||||
<MemoryRouter initialEntries={["/overview"]}>
|
||||
<MemoryRouter initialEntries={["/jobs/discovered"]}>
|
||||
<App />
|
||||
</MemoryRouter>,
|
||||
);
|
||||
@@ -114,7 +114,7 @@ describe("App demo banner", () => {
|
||||
});
|
||||
|
||||
render(
|
||||
<MemoryRouter initialEntries={["/overview"]}>
|
||||
<MemoryRouter initialEntries={["/jobs/discovered"]}>
|
||||
<App />
|
||||
</MemoryRouter>,
|
||||
);
|
||||
@@ -135,7 +135,7 @@ describe("App demo banner", () => {
|
||||
});
|
||||
|
||||
render(
|
||||
<MemoryRouter initialEntries={["/overview"]}>
|
||||
<MemoryRouter initialEntries={["/jobs/discovered"]}>
|
||||
<App />
|
||||
</MemoryRouter>,
|
||||
);
|
||||
|
||||
@@ -14,29 +14,31 @@ import { BasicAuthPrompt } from "./components/BasicAuthPrompt";
|
||||
import { OnboardingGate } from "./components/OnboardingGate";
|
||||
import { useDemoInfo } from "./hooks/useDemoInfo";
|
||||
import { GmailOauthCallbackPage } from "./pages/GmailOauthCallbackPage";
|
||||
import { HomePage } from "./pages/HomePage";
|
||||
import { InProgressBoardPage } from "./pages/InProgressBoardPage";
|
||||
import { JobPage } from "./pages/JobPage";
|
||||
import { OrchestratorPage } from "./pages/OrchestratorPage";
|
||||
import { SettingsPage } from "./pages/SettingsPage";
|
||||
import { TracerLinksPage } from "./pages/TracerLinksPage";
|
||||
import { TrackingInboxPage } from "./pages/TrackingInboxPage";
|
||||
import { VisaSponsorsPage } from "./pages/VisaSponsorsPage";
|
||||
|
||||
/** Backwards-compatibility redirects: old URL paths -> new URL paths */
|
||||
const REDIRECTS: Array<{ from: string; to: string }> = [
|
||||
{ from: "/", to: "/jobs/ready" },
|
||||
{ from: "/home", to: "/overview" },
|
||||
{ from: "/", to: "/jobs/discovered" },
|
||||
{ from: "/home", to: "/jobs/discovered" },
|
||||
{ from: "/overview", to: "/jobs/discovered" },
|
||||
{ from: "/ready", to: "/jobs/ready" },
|
||||
{ from: "/ready/:jobId", to: "/jobs/ready/:jobId" },
|
||||
{ from: "/discovered", to: "/jobs/discovered" },
|
||||
{ from: "/discovered/:jobId", to: "/jobs/discovered/:jobId" },
|
||||
{ from: "/applied", to: "/jobs/applied" },
|
||||
{ from: "/applied/:jobId", to: "/jobs/applied/:jobId" },
|
||||
{ from: "/in-progress", to: "/applications/in-progress" },
|
||||
{ from: "/in-progress/:jobId", to: "/applications/in-progress" },
|
||||
{ from: "/jobs/in_progress", to: "/applications/in-progress" },
|
||||
{ from: "/jobs/in_progress/:jobId", to: "/applications/in-progress" },
|
||||
// Slim product: kanban / inbox / tracer / visa browser redirect into Jobs.
|
||||
{ from: "/in-progress", to: "/jobs/discovered" },
|
||||
{ from: "/in-progress/:jobId", to: "/jobs/discovered" },
|
||||
{ from: "/jobs/in_progress", to: "/jobs/discovered" },
|
||||
{ from: "/jobs/in_progress/:jobId", to: "/jobs/discovered" },
|
||||
{ from: "/applications/in-progress", to: "/jobs/discovered" },
|
||||
{ from: "/applications/in-progress/:jobId", to: "/jobs/discovered" },
|
||||
{ from: "/tracking-inbox", to: "/jobs/discovered" },
|
||||
{ from: "/tracer-links", to: "/jobs/discovered" },
|
||||
{ from: "/visa-sponsors", to: "/jobs/discovered" },
|
||||
{ from: "/all", to: "/jobs/all" },
|
||||
{ from: "/all/:jobId", to: "/jobs/all/:jobId" },
|
||||
];
|
||||
@@ -134,24 +136,13 @@ export const App: React.FC = () => {
|
||||
/>
|
||||
))}
|
||||
|
||||
{/* Application routes */}
|
||||
<Route path="/overview" element={<HomePage />} />
|
||||
{/* Application routes — slim product: Jobs + Settings */}
|
||||
<Route
|
||||
path="/oauth/gmail/callback"
|
||||
element={<GmailOauthCallbackPage />}
|
||||
/>
|
||||
<Route path="/job/:id" element={<JobPage />} />
|
||||
<Route
|
||||
path="/applications/in-progress"
|
||||
element={<InProgressBoardPage />}
|
||||
/>
|
||||
<Route path="/settings" element={<SettingsPage />} />
|
||||
<Route path="/tracer-links" element={<TracerLinksPage />} />
|
||||
<Route path="/visa-sponsors" element={<VisaSponsorsPage />} />
|
||||
<Route
|
||||
path="/tracking-inbox"
|
||||
element={<TrackingInboxPage />}
|
||||
/>
|
||||
<Route path="/jobs/:tab" element={<OrchestratorPage />} />
|
||||
<Route
|
||||
path="/jobs/:tab/:jobId"
|
||||
|
||||
@@ -25,6 +25,7 @@ import type {
|
||||
JobsListResponse,
|
||||
JobsRevisionResponse,
|
||||
JobTracerLinksResponse,
|
||||
KeywordSet,
|
||||
ManualJobDraft,
|
||||
ManualJobFetchResponse,
|
||||
ManualJobInferenceResponse,
|
||||
@@ -39,7 +40,6 @@ import type {
|
||||
ProfileStatusResponse,
|
||||
ResumeProfile,
|
||||
ResumeProjectCatalogItem,
|
||||
RxResumeMode,
|
||||
SearchProfile,
|
||||
StageEvent,
|
||||
StageEventMetadata,
|
||||
@@ -393,11 +393,16 @@ export async function activateSearchProfileForBasicAuthUser(
|
||||
return;
|
||||
}
|
||||
if (!listPayload.ok || !Array.isArray(listPayload.data)) return;
|
||||
const match = listPayload.data.find(
|
||||
(row) => (row.data?.basicAuthUser ?? "").trim() === username.trim(),
|
||||
);
|
||||
if (!match) return;
|
||||
await fetch(`${API_BASE}/profiles/${match.id}/activate`, {
|
||||
const normalizedUsername = username.trim().toLowerCase();
|
||||
const matches = listPayload.data.filter((row) => {
|
||||
const aliases = (row.data?.basicAuthUser ?? "")
|
||||
.split(",")
|
||||
.map((part) => part.trim().toLowerCase())
|
||||
.filter(Boolean);
|
||||
return aliases.includes(normalizedUsername);
|
||||
});
|
||||
if (matches.length !== 1) return;
|
||||
await fetch(`${API_BASE}/profiles/${matches[0].id}/activate`, {
|
||||
method: "POST",
|
||||
headers: {
|
||||
"Content-Type": "application/json",
|
||||
@@ -1533,19 +1538,6 @@ export async function getProfileProjects(): Promise<
|
||||
export async function getResumeProjectsCatalog(): Promise<
|
||||
ResumeProjectCatalogItem[]
|
||||
> {
|
||||
try {
|
||||
const settings = await getSettings();
|
||||
if (settings.rxresumeBaseResumeId) {
|
||||
return await getRxResumeProjects(
|
||||
settings.rxresumeBaseResumeId,
|
||||
undefined,
|
||||
settings.rxresumeMode?.value,
|
||||
);
|
||||
}
|
||||
} catch {
|
||||
// fall through to profile-based projects
|
||||
}
|
||||
|
||||
return getProfileProjects();
|
||||
}
|
||||
|
||||
@@ -1586,19 +1578,6 @@ export async function getLlmModels(input?: {
|
||||
return data.models;
|
||||
}
|
||||
|
||||
export async function validateRxresume(input?: {
|
||||
mode?: "v4" | "v5";
|
||||
email?: string;
|
||||
password?: string;
|
||||
apiKey?: string;
|
||||
baseUrl?: string;
|
||||
}): Promise<ValidationResult> {
|
||||
return fetchApi<ValidationResult>("/onboarding/validate/rxresume", {
|
||||
method: "POST",
|
||||
body: JSON.stringify(input ?? {}),
|
||||
});
|
||||
}
|
||||
|
||||
export async function validateResumeConfig(): Promise<ValidationResult> {
|
||||
return fetchApi<ValidationResult>("/onboarding/validate/resume");
|
||||
}
|
||||
@@ -1612,29 +1591,6 @@ export async function updateSettings(
|
||||
});
|
||||
}
|
||||
|
||||
export async function getRxResumes(
|
||||
mode?: RxResumeMode,
|
||||
): Promise<{ id: string; name: string }[]> {
|
||||
const query = mode ? `?mode=${encodeURIComponent(mode)}` : "";
|
||||
const data = await fetchApi<{ resumes: { id: string; name: string }[] }>(
|
||||
`/settings/rx-resumes${query}`,
|
||||
);
|
||||
return data.resumes;
|
||||
}
|
||||
|
||||
export async function getRxResumeProjects(
|
||||
resumeId: string,
|
||||
signal?: AbortSignal,
|
||||
mode?: RxResumeMode,
|
||||
): Promise<ResumeProjectCatalogItem[]> {
|
||||
const query = mode ? `?mode=${encodeURIComponent(mode)}` : "";
|
||||
const data = await fetchApi<{ projects: ResumeProjectCatalogItem[] }>(
|
||||
`/settings/rx-resumes/${encodeURIComponent(resumeId)}/projects${query}`,
|
||||
{ signal },
|
||||
);
|
||||
return data.projects;
|
||||
}
|
||||
|
||||
// Database API
|
||||
export async function clearDatabase(): Promise<{
|
||||
message: string;
|
||||
@@ -1662,6 +1618,30 @@ export async function deleteJobsByStatus(status: string): Promise<{
|
||||
});
|
||||
}
|
||||
|
||||
export async function deleteDiscoveredMatchingArchived(): Promise<{
|
||||
message: string;
|
||||
count: number;
|
||||
}> {
|
||||
return fetchApi<{
|
||||
message: string;
|
||||
count: number;
|
||||
}>("/jobs/duplicates/discovered", {
|
||||
method: "DELETE",
|
||||
});
|
||||
}
|
||||
|
||||
export async function expireDeadListings(): Promise<{
|
||||
message: string;
|
||||
count: number;
|
||||
}> {
|
||||
return fetchApi<{
|
||||
message: string;
|
||||
count: number;
|
||||
}>("/jobs/expire-dead-listings", {
|
||||
method: "POST",
|
||||
});
|
||||
}
|
||||
|
||||
export async function deleteJobsBelowScore(threshold: number): Promise<{
|
||||
message: string;
|
||||
count: number;
|
||||
@@ -1790,3 +1770,41 @@ export async function generateProfileFromResume(): Promise<JobSearchProfile> {
|
||||
method: "POST",
|
||||
});
|
||||
}
|
||||
|
||||
// Keyword sets API
|
||||
export async function listKeywordSets(): Promise<KeywordSet[]> {
|
||||
return fetchApi<KeywordSet[]>("/keyword-sets");
|
||||
}
|
||||
|
||||
export async function createKeywordSet(input: {
|
||||
name: string;
|
||||
terms?: string[];
|
||||
}): Promise<KeywordSet> {
|
||||
return fetchApi<KeywordSet>("/keyword-sets", {
|
||||
method: "POST",
|
||||
body: JSON.stringify(input),
|
||||
});
|
||||
}
|
||||
|
||||
export async function updateKeywordSet(
|
||||
id: string,
|
||||
input: { name?: string; terms?: string[] },
|
||||
): Promise<KeywordSet> {
|
||||
return fetchApi<KeywordSet>(`/keyword-sets/${encodeURIComponent(id)}`, {
|
||||
method: "PATCH",
|
||||
body: JSON.stringify(input),
|
||||
});
|
||||
}
|
||||
|
||||
export async function deleteKeywordSet(id: string): Promise<void> {
|
||||
await fetchApi<void>(`/keyword-sets/${encodeURIComponent(id)}`, {
|
||||
method: "DELETE",
|
||||
});
|
||||
}
|
||||
|
||||
export async function activateKeywordSet(id: string): Promise<KeywordSet> {
|
||||
return fetchApi<KeywordSet>(
|
||||
`/keyword-sets/${encodeURIComponent(id)}/activate`,
|
||||
{ method: "POST" },
|
||||
);
|
||||
}
|
||||
|
||||
@@ -20,6 +20,7 @@ import {
|
||||
} from "@/components/ui/tooltip";
|
||||
import { cn, formatDate, sourceLabel } from "@/lib/utils";
|
||||
import { useSettings } from "../hooks/useSettings";
|
||||
import { SponsorshipSignalsPills } from "./SponsorshipSignalsPills";
|
||||
import {
|
||||
getJobStatusIndicator,
|
||||
getTracerStatusIndicator,
|
||||
@@ -255,6 +256,10 @@ export const JobHeader: React.FC<JobHeaderProps> = ({
|
||||
onCheck={onCheckSponsor}
|
||||
/>
|
||||
)}
|
||||
<SponsorshipSignalsPills
|
||||
sponsorshipSignals={job.sponsorshipSignals}
|
||||
size="xs"
|
||||
/>
|
||||
</div>
|
||||
<ScoreMeter score={job.suitabilityScore} />
|
||||
</div>
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
import * as api from "@client/api";
|
||||
import { useSettings } from "@client/hooks/useSettings";
|
||||
import { fireEvent, screen, waitFor } from "@testing-library/react";
|
||||
import { screen, waitFor } from "@testing-library/react";
|
||||
import type React from "react";
|
||||
import { beforeEach, describe, expect, it, vi } from "vitest";
|
||||
import { renderWithQueryClient } from "../test/renderWithQueryClient";
|
||||
@@ -12,8 +12,6 @@ const render = (ui: Parameters<typeof renderWithQueryClient>[0]) =>
|
||||
vi.mock("@client/api", () => ({
|
||||
getDemoInfo: vi.fn(),
|
||||
validateLlm: vi.fn(),
|
||||
validateRxresume: vi.fn(),
|
||||
validateResumeConfig: vi.fn(),
|
||||
updateSettings: vi.fn(),
|
||||
}));
|
||||
|
||||
@@ -36,10 +34,6 @@ vi.mock("@client/pages/settings/components/SettingsInput", () => ({
|
||||
),
|
||||
}));
|
||||
|
||||
vi.mock("@client/pages/settings/components/BaseResumeSelection", () => ({
|
||||
BaseResumeSelection: () => <div>Base resume selection</div>,
|
||||
}));
|
||||
|
||||
vi.mock("@/components/ui/alert-dialog", () => ({
|
||||
AlertDialog: ({ children }: { children: React.ReactNode }) => (
|
||||
<div>{children}</div>
|
||||
@@ -58,19 +52,6 @@ vi.mock("@/components/ui/alert-dialog", () => ({
|
||||
),
|
||||
}));
|
||||
|
||||
vi.mock("@/components/ui/tabs", () => ({
|
||||
Tabs: ({ children }: { children: React.ReactNode }) => <div>{children}</div>,
|
||||
TabsContent: ({ children }: { children: React.ReactNode }) => (
|
||||
<div>{children}</div>
|
||||
),
|
||||
TabsList: ({ children }: { children: React.ReactNode }) => (
|
||||
<div>{children}</div>
|
||||
),
|
||||
TabsTrigger: ({ children }: { children: React.ReactNode }) => (
|
||||
<button type="button">{children}</button>
|
||||
),
|
||||
}));
|
||||
|
||||
vi.mock("@/components/ui/select", () => ({
|
||||
Select: ({ children }: { children: React.ReactNode }) => (
|
||||
<div>{children}</div>
|
||||
@@ -105,11 +86,6 @@ const settingsResponse = {
|
||||
settings: {
|
||||
llmProvider: { value: "openrouter", default: "openrouter", override: null },
|
||||
llmApiKeyHint: null,
|
||||
rxresumeEmail: "",
|
||||
rxresumeUrl: "",
|
||||
rxresumeApiKeyHint: null,
|
||||
rxresumePasswordHint: null,
|
||||
rxresumeBaseResumeId: null,
|
||||
localResumeProfilePath: null,
|
||||
localResumeFileConfigured: false,
|
||||
},
|
||||
@@ -131,19 +107,11 @@ describe("OnboardingGate", () => {
|
||||
vi.mocked(useSettings).mockReturnValue(settingsResponse as any);
|
||||
});
|
||||
|
||||
it("renders the gate once validations complete and any fail", async () => {
|
||||
it("renders the gate when LLM validation fails", async () => {
|
||||
vi.mocked(api.validateLlm).mockResolvedValue({
|
||||
valid: false,
|
||||
message: "Invalid",
|
||||
});
|
||||
vi.mocked(api.validateRxresume).mockResolvedValue({
|
||||
valid: true,
|
||||
message: null,
|
||||
});
|
||||
vi.mocked(api.validateResumeConfig).mockResolvedValue({
|
||||
valid: true,
|
||||
message: null,
|
||||
});
|
||||
|
||||
render(<OnboardingGate />);
|
||||
|
||||
@@ -151,28 +119,17 @@ describe("OnboardingGate", () => {
|
||||
await waitFor(() => {
|
||||
expect(screen.getByText("Welcome to Job Ops")).toBeInTheDocument();
|
||||
});
|
||||
expect(
|
||||
screen.queryByLabelText("Local resume path"),
|
||||
).not.toBeInTheDocument();
|
||||
expect(screen.queryByText("Resume file")).not.toBeInTheDocument();
|
||||
});
|
||||
|
||||
it("hides the gate when all validations succeed", async () => {
|
||||
vi.mocked(useSettings).mockReturnValue({
|
||||
...settingsResponse,
|
||||
settings: {
|
||||
...settingsResponse.settings,
|
||||
rxresumeApiKeyHint: "abcd1234",
|
||||
},
|
||||
} as any);
|
||||
it("hides the gate when LLM validation succeeds", async () => {
|
||||
vi.mocked(api.validateLlm).mockResolvedValue({
|
||||
valid: true,
|
||||
message: null,
|
||||
});
|
||||
vi.mocked(api.validateRxresume).mockResolvedValue({
|
||||
valid: true,
|
||||
message: null,
|
||||
});
|
||||
vi.mocked(api.validateResumeConfig).mockResolvedValue({
|
||||
valid: true,
|
||||
message: null,
|
||||
});
|
||||
|
||||
render(<OnboardingGate />);
|
||||
|
||||
@@ -180,13 +137,12 @@ describe("OnboardingGate", () => {
|
||||
expect(screen.queryByText("Welcome to Job Ops")).not.toBeInTheDocument();
|
||||
});
|
||||
|
||||
it("hides the gate for Ollama when local resume file is configured on the server", async () => {
|
||||
it("hides the gate for providers without API keys", async () => {
|
||||
vi.mocked(useSettings).mockReturnValue({
|
||||
...settingsResponse,
|
||||
settings: {
|
||||
...settingsResponse.settings,
|
||||
llmProvider: { value: "ollama", default: "ollama", override: null },
|
||||
localResumeFileConfigured: true,
|
||||
},
|
||||
} as any);
|
||||
|
||||
@@ -195,79 +151,6 @@ describe("OnboardingGate", () => {
|
||||
await waitFor(() => {
|
||||
expect(screen.queryByText("Welcome to Job Ops")).not.toBeInTheDocument();
|
||||
});
|
||||
expect(api.validateRxresume).not.toHaveBeenCalled();
|
||||
expect(api.validateResumeConfig).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("skips LLM key validation for providers without API keys", async () => {
|
||||
vi.mocked(useSettings).mockReturnValue({
|
||||
...settingsResponse,
|
||||
settings: {
|
||||
...settingsResponse.settings,
|
||||
llmProvider: { value: "ollama", default: "ollama", override: null },
|
||||
},
|
||||
} as any);
|
||||
vi.mocked(api.validateRxresume).mockResolvedValue({
|
||||
valid: false,
|
||||
message: "Missing",
|
||||
});
|
||||
vi.mocked(api.validateResumeConfig).mockResolvedValue({
|
||||
valid: true,
|
||||
message: null,
|
||||
});
|
||||
|
||||
render(<OnboardingGate />);
|
||||
|
||||
await waitFor(() => expect(api.validateResumeConfig).toHaveBeenCalled());
|
||||
expect(api.validateLlm).not.toHaveBeenCalled();
|
||||
expect(api.validateRxresume).not.toHaveBeenCalled();
|
||||
await waitFor(() => {
|
||||
expect(screen.getByText("Welcome to Job Ops")).toBeInTheDocument();
|
||||
});
|
||||
expect(screen.queryByText("LLM API key")).not.toBeInTheDocument();
|
||||
});
|
||||
|
||||
it("renders the RxResume URL field and includes it in validation", async () => {
|
||||
vi.mocked(useSettings).mockReturnValue({
|
||||
...settingsResponse,
|
||||
settings: {
|
||||
...settingsResponse.settings,
|
||||
rxresumeUrl: "https://resume.example.com",
|
||||
rxresumeApiKeyHint: "abcd1234",
|
||||
},
|
||||
} as any);
|
||||
vi.mocked(api.validateLlm).mockResolvedValue({
|
||||
valid: false,
|
||||
message: "Invalid",
|
||||
});
|
||||
vi.mocked(api.validateRxresume).mockResolvedValue({
|
||||
valid: true,
|
||||
message: null,
|
||||
});
|
||||
vi.mocked(api.validateResumeConfig).mockResolvedValue({
|
||||
valid: true,
|
||||
message: null,
|
||||
});
|
||||
|
||||
render(<OnboardingGate />);
|
||||
|
||||
await waitFor(() =>
|
||||
expect(screen.getByLabelText("RxResume URL")).toBeInTheDocument(),
|
||||
);
|
||||
await waitFor(() =>
|
||||
expect(api.validateRxresume).toHaveBeenCalledWith(
|
||||
expect.objectContaining({
|
||||
baseUrl: "https://resume.example.com",
|
||||
}),
|
||||
),
|
||||
);
|
||||
|
||||
fireEvent.change(screen.getByLabelText("RxResume URL"), {
|
||||
target: { value: "https://self-hosted.example.com" },
|
||||
});
|
||||
|
||||
expect(
|
||||
screen.getByDisplayValue("https://self-hosted.example.com"),
|
||||
).toBeInTheDocument();
|
||||
});
|
||||
});
|
||||
|
||||
@@ -1,15 +1,6 @@
|
||||
import * as api from "@client/api";
|
||||
import { ReactiveResumeConfigPanel } from "@client/components/ReactiveResumeConfigPanel";
|
||||
import { useDemoInfo } from "@client/hooks/useDemoInfo";
|
||||
import { useRxResumeConfigState } from "@client/hooks/useRxResumeConfigState";
|
||||
import { useSettings } from "@client/hooks/useSettings";
|
||||
import {
|
||||
getInitialRxResumeMode,
|
||||
getRxResumeCredentialDrafts,
|
||||
getRxResumeMissingCredentialLabels,
|
||||
validateAndMaybePersistRxResumeMode,
|
||||
} from "@client/lib/rxresume-config";
|
||||
import { BaseResumeSelection } from "@client/pages/settings/components/BaseResumeSelection";
|
||||
import { SettingsInput } from "@client/pages/settings/components/SettingsInput";
|
||||
import {
|
||||
getLlmProviderConfig,
|
||||
@@ -19,10 +10,10 @@ import {
|
||||
} from "@client/pages/settings/utils";
|
||||
import { getDefaultModelForProvider } from "@shared/settings-registry";
|
||||
import type { UpdateSettingsInput } from "@shared/settings-schema.js";
|
||||
import type { RxResumeMode, ValidationResult } from "@shared/types.js";
|
||||
import type { ValidationResult } from "@shared/types.js";
|
||||
import { Check } from "lucide-react";
|
||||
import type React from "react";
|
||||
import { useCallback, useEffect, useMemo, useState } from "react";
|
||||
import { useCallback, useEffect, useState } from "react";
|
||||
import { Controller, useForm } from "react-hook-form";
|
||||
import { toast } from "sonner";
|
||||
import {
|
||||
@@ -48,22 +39,14 @@ import {
|
||||
SelectTrigger,
|
||||
SelectValue,
|
||||
} from "@/components/ui/select";
|
||||
import { Tabs, TabsContent, TabsList, TabsTrigger } from "@/components/ui/tabs";
|
||||
import { cn } from "@/lib/utils";
|
||||
|
||||
type ValidationState = ValidationResult & { checked: boolean };
|
||||
type TimestampedValidationState = ValidationState & { testedAt: number | null };
|
||||
|
||||
type OnboardingFormData = {
|
||||
llmProvider: string;
|
||||
llmBaseUrl: string;
|
||||
llmApiKey: string;
|
||||
rxresumeMode: RxResumeMode;
|
||||
rxresumeEmail: string;
|
||||
rxresumeUrl: string;
|
||||
rxresumePassword: string;
|
||||
rxresumeApiKey: string;
|
||||
rxresumeBaseResumeId: string | null;
|
||||
};
|
||||
|
||||
const EMPTY_VALIDATION_STATE: ValidationState = {
|
||||
@@ -72,27 +55,6 @@ const EMPTY_VALIDATION_STATE: ValidationState = {
|
||||
checked: false,
|
||||
};
|
||||
|
||||
const EMPTY_TIMESTAMPED_VALIDATION_STATE: TimestampedValidationState = {
|
||||
...EMPTY_VALIDATION_STATE,
|
||||
testedAt: null,
|
||||
};
|
||||
|
||||
function getStepPrimaryLabel(input: {
|
||||
currentStep: string | null;
|
||||
llmValidated: boolean;
|
||||
rxresumeValidated: boolean;
|
||||
baseResumeValidated: boolean;
|
||||
}): string {
|
||||
const toLabel = (isValidated: boolean): string =>
|
||||
isValidated ? "Revalidate" : "Validate";
|
||||
|
||||
if (input.currentStep === "llm") return toLabel(input.llmValidated);
|
||||
if (input.currentStep === "rxresume") return toLabel(input.rxresumeValidated);
|
||||
if (input.currentStep === "baseresume")
|
||||
return toLabel(input.baseResumeValidated);
|
||||
return "Validate";
|
||||
}
|
||||
|
||||
export const OnboardingGate: React.FC = () => {
|
||||
const {
|
||||
settings,
|
||||
@@ -100,41 +62,11 @@ export const OnboardingGate: React.FC = () => {
|
||||
refreshSettings,
|
||||
} = useSettings();
|
||||
|
||||
/** Skip RxResume onboarding when Vite flag is set, server reports a local resume file, or Settings has a local path. */
|
||||
const skipRxResumeOnboarding = useMemo(
|
||||
() =>
|
||||
import.meta.env.VITE_SKIP_RXRESUME_ONBOARDING === "true" ||
|
||||
Boolean(settings?.localResumeFileConfigured) ||
|
||||
Boolean(settings?.localResumeProfilePath?.trim()),
|
||||
[settings?.localResumeFileConfigured, settings?.localResumeProfilePath],
|
||||
);
|
||||
const {
|
||||
storedRxResume,
|
||||
getBaseResumeIdForMode,
|
||||
setBaseResumeIdForMode,
|
||||
syncBaseResumeIdsForMode,
|
||||
} = useRxResumeConfigState(settings);
|
||||
|
||||
const [isSavingEnv, setIsSavingEnv] = useState(false);
|
||||
const [isValidatingLlm, setIsValidatingLlm] = useState(false);
|
||||
const [isValidatingRxresume, setIsValidatingRxresume] = useState(false);
|
||||
const [isValidatingBaseResume, setIsValidatingBaseResume] = useState(false);
|
||||
const [llmValidation, setLlmValidation] = useState<ValidationState>(
|
||||
EMPTY_VALIDATION_STATE,
|
||||
);
|
||||
const [rxresumeValidation, setRxresumeValidation] = useState<ValidationState>(
|
||||
EMPTY_VALIDATION_STATE,
|
||||
);
|
||||
const [rxresumeVersionValidations, setRxresumeVersionValidations] = useState<{
|
||||
v4: TimestampedValidationState;
|
||||
v5: TimestampedValidationState;
|
||||
}>({
|
||||
v4: EMPTY_TIMESTAMPED_VALIDATION_STATE,
|
||||
v5: EMPTY_TIMESTAMPED_VALIDATION_STATE,
|
||||
});
|
||||
const [baseResumeValidation, setBaseResumeValidation] =
|
||||
useState<ValidationState>(EMPTY_VALIDATION_STATE);
|
||||
const [currentStep, setCurrentStep] = useState<string | null>(null);
|
||||
const demoInfo = useDemoInfo();
|
||||
const demoMode = demoInfo?.demoMode ?? false;
|
||||
|
||||
@@ -144,12 +76,6 @@ export const OnboardingGate: React.FC = () => {
|
||||
llmProvider: "",
|
||||
llmBaseUrl: "",
|
||||
llmApiKey: "",
|
||||
rxresumeMode: "v5",
|
||||
rxresumeEmail: "",
|
||||
rxresumeUrl: "",
|
||||
rxresumePassword: "",
|
||||
rxresumeApiKey: "",
|
||||
rxresumeBaseResumeId: null,
|
||||
},
|
||||
});
|
||||
|
||||
@@ -187,26 +113,6 @@ export const OnboardingGate: React.FC = () => {
|
||||
}
|
||||
}, [getValues, settings?.llmProvider]);
|
||||
|
||||
const validateBaseResume = useCallback(async () => {
|
||||
setIsValidatingBaseResume(true);
|
||||
try {
|
||||
const result = await api.validateResumeConfig();
|
||||
setBaseResumeValidation({ ...result, checked: true });
|
||||
return result;
|
||||
} catch (error) {
|
||||
const message =
|
||||
error instanceof Error
|
||||
? error.message
|
||||
: "Base resume validation failed";
|
||||
const result = { valid: false, message };
|
||||
setBaseResumeValidation({ ...result, checked: true });
|
||||
return result;
|
||||
} finally {
|
||||
setIsValidatingBaseResume(false);
|
||||
}
|
||||
}, []);
|
||||
|
||||
const rxresumeModeValue = watch("rxresumeMode");
|
||||
const selectedProvider = normalizeLlmProvider(
|
||||
llmProvider || settings?.llmProvider?.value || "openrouter",
|
||||
);
|
||||
@@ -220,234 +126,68 @@ export const OnboardingGate: React.FC = () => {
|
||||
|
||||
const llmKeyHint = settings?.llmApiKeyHint ?? null;
|
||||
const hasLlmKey = Boolean(llmKeyHint);
|
||||
const rxresumeModeCurrent = (rxresumeModeValue ||
|
||||
settings?.rxresumeMode?.value ||
|
||||
"v5") as RxResumeMode;
|
||||
const hasCheckedValidations =
|
||||
(requiresLlmKey ? llmValidation.checked : true) &&
|
||||
(skipRxResumeOnboarding
|
||||
? true
|
||||
: rxresumeValidation.checked && baseResumeValidation.checked);
|
||||
const llmValidated = requiresLlmKey ? llmValidation.valid : true;
|
||||
|
||||
const hasCheckedValidations = requiresLlmKey ? llmValidation.checked : true;
|
||||
|
||||
const shouldOpen =
|
||||
!demoMode &&
|
||||
Boolean(settings && !settingsLoading) &&
|
||||
hasCheckedValidations &&
|
||||
!(
|
||||
llmValidated &&
|
||||
(skipRxResumeOnboarding
|
||||
? true
|
||||
: rxresumeValidation.valid && baseResumeValidation.valid)
|
||||
);
|
||||
!llmValidated;
|
||||
|
||||
const validateRxresumeVersion = useCallback(
|
||||
async (
|
||||
version: "v4" | "v5",
|
||||
): Promise<ValidationResult & { checked: true; testedAt: number }> => {
|
||||
const values = getValues();
|
||||
const draftCredentials = getRxResumeCredentialDrafts(values);
|
||||
const testedAt = Date.now();
|
||||
const result = await validateAndMaybePersistRxResumeMode({
|
||||
mode: version,
|
||||
stored: storedRxResume,
|
||||
draft: draftCredentials,
|
||||
validate: api.validateRxresume,
|
||||
getPrecheckMessage: (failure) =>
|
||||
failure === "missing-v5-api-key"
|
||||
? "v5 API key required. Add a v5 API key, then test again."
|
||||
: "v4 email and password required. Add both credentials, then test again.",
|
||||
getValidationErrorMessage: (error, mode) =>
|
||||
error instanceof Error
|
||||
? error.message
|
||||
: `RxResume ${mode} validation failed`,
|
||||
});
|
||||
return { ...result.validation, checked: true, testedAt };
|
||||
},
|
||||
[getValues, storedRxResume],
|
||||
);
|
||||
|
||||
const validateRxresume = useCallback(async () => {
|
||||
const values = getValues();
|
||||
const selectedMode = values.rxresumeMode;
|
||||
|
||||
setIsValidatingRxresume(true);
|
||||
try {
|
||||
const versionResult = await validateRxresumeVersion(selectedMode);
|
||||
setRxresumeVersionValidations((current) => ({
|
||||
...current,
|
||||
[selectedMode]: versionResult,
|
||||
}));
|
||||
|
||||
const result: ValidationResult = {
|
||||
valid: versionResult.valid,
|
||||
message: versionResult.message,
|
||||
};
|
||||
setRxresumeValidation({ ...result, checked: true });
|
||||
return result;
|
||||
} finally {
|
||||
setIsValidatingRxresume(false);
|
||||
}
|
||||
}, [getValues, validateRxresumeVersion]);
|
||||
|
||||
// Initialize form values from settings
|
||||
useEffect(() => {
|
||||
if (settings) {
|
||||
const initialMode = getInitialRxResumeMode({
|
||||
savedMode: (settings.rxresumeMode?.value ??
|
||||
null) as RxResumeMode | null,
|
||||
hasV4: storedRxResume.hasV4,
|
||||
hasV5: storedRxResume.hasV5,
|
||||
});
|
||||
const selectedId = syncBaseResumeIdsForMode(initialMode);
|
||||
reset({
|
||||
llmProvider: settings.llmProvider?.value || "",
|
||||
llmBaseUrl: settings.llmBaseUrl?.value || "",
|
||||
llmApiKey: "",
|
||||
rxresumeMode: initialMode,
|
||||
rxresumeEmail: "",
|
||||
rxresumeUrl: settings.rxresumeUrl ?? "",
|
||||
rxresumePassword: "",
|
||||
rxresumeApiKey: "",
|
||||
rxresumeBaseResumeId: selectedId,
|
||||
});
|
||||
}
|
||||
}, [
|
||||
settings,
|
||||
reset,
|
||||
storedRxResume.hasV4,
|
||||
storedRxResume.hasV5,
|
||||
syncBaseResumeIdsForMode,
|
||||
]);
|
||||
}, [settings, reset]);
|
||||
|
||||
// Clear base URL when provider doesn't require it
|
||||
useEffect(() => {
|
||||
if (!showBaseUrl) {
|
||||
setValue("llmBaseUrl", "");
|
||||
}
|
||||
}, [showBaseUrl, setValue]);
|
||||
|
||||
// Reset LLM validation when provider changes
|
||||
useEffect(() => {
|
||||
if (!selectedProvider) return;
|
||||
setLlmValidation({ valid: false, message: null, checked: false });
|
||||
}, [selectedProvider]);
|
||||
|
||||
const steps = useMemo(
|
||||
() =>
|
||||
skipRxResumeOnboarding
|
||||
? [
|
||||
{
|
||||
id: "llm",
|
||||
label: "LLM Provider",
|
||||
subtitle: "Provider + credentials",
|
||||
complete: llmValidated,
|
||||
disabled: false,
|
||||
},
|
||||
]
|
||||
: [
|
||||
{
|
||||
id: "llm",
|
||||
label: "LLM Provider",
|
||||
subtitle: "Provider + credentials",
|
||||
complete: llmValidated,
|
||||
disabled: false,
|
||||
},
|
||||
{
|
||||
id: "rxresume",
|
||||
label: "Connect Reactive Resume",
|
||||
subtitle: "Version + credentials",
|
||||
complete: rxresumeValidation.valid,
|
||||
disabled: false,
|
||||
},
|
||||
{
|
||||
id: "baseresume",
|
||||
label: "Select Template Resume",
|
||||
subtitle: "Template selection",
|
||||
complete: baseResumeValidation.valid,
|
||||
disabled: !rxresumeValidation.valid,
|
||||
},
|
||||
],
|
||||
[
|
||||
skipRxResumeOnboarding,
|
||||
llmValidated,
|
||||
rxresumeValidation.valid,
|
||||
baseResumeValidation.valid,
|
||||
],
|
||||
);
|
||||
|
||||
const defaultStep = steps.find((step) => !step.complete)?.id ?? steps[0]?.id;
|
||||
|
||||
useEffect(() => {
|
||||
if (!shouldOpen) return;
|
||||
if (!currentStep && defaultStep) {
|
||||
setCurrentStep(defaultStep);
|
||||
}
|
||||
}, [currentStep, defaultStep, shouldOpen]);
|
||||
|
||||
const runAllValidations = useCallback(async () => {
|
||||
const runLlmValidation = useCallback(async () => {
|
||||
if (!settings) return;
|
||||
const validations: Promise<ValidationResult>[] = [];
|
||||
if (requiresLlmKey) {
|
||||
validations.push(validateLlm());
|
||||
await validateLlm();
|
||||
} else {
|
||||
setLlmValidation({ valid: true, message: null, checked: true });
|
||||
}
|
||||
if (!skipRxResumeOnboarding) {
|
||||
validations.push(validateRxresume(), validateBaseResume());
|
||||
} else {
|
||||
setRxresumeValidation({ valid: true, message: null, checked: true });
|
||||
setBaseResumeValidation({ valid: true, message: null, checked: true });
|
||||
}
|
||||
}, [settings, requiresLlmKey, validateLlm]);
|
||||
|
||||
const results = await Promise.allSettled(validations);
|
||||
|
||||
const failed = results.find((result) => result.status === "rejected");
|
||||
if (failed) {
|
||||
const reason = failed.status === "rejected" ? failed.reason : null;
|
||||
const message =
|
||||
reason instanceof Error ? reason.message : "Validation checks failed";
|
||||
toast.error(message);
|
||||
}
|
||||
}, [
|
||||
settings,
|
||||
requiresLlmKey,
|
||||
skipRxResumeOnboarding,
|
||||
validateLlm,
|
||||
validateRxresume,
|
||||
validateBaseResume,
|
||||
]);
|
||||
|
||||
// Run validations on mount when needed
|
||||
useEffect(() => {
|
||||
if (demoMode) return;
|
||||
if (!settings || settingsLoading) return;
|
||||
const needsValidation =
|
||||
(requiresLlmKey ? !llmValidation.checked : false) ||
|
||||
(skipRxResumeOnboarding
|
||||
? false
|
||||
: !rxresumeValidation.checked || !baseResumeValidation.checked);
|
||||
if (!needsValidation) return;
|
||||
void runAllValidations();
|
||||
if (requiresLlmKey ? llmValidation.checked : true) return;
|
||||
void runLlmValidation();
|
||||
}, [
|
||||
settings,
|
||||
settingsLoading,
|
||||
requiresLlmKey,
|
||||
llmValidation.checked,
|
||||
rxresumeValidation.checked,
|
||||
baseResumeValidation.checked,
|
||||
runAllValidations,
|
||||
runLlmValidation,
|
||||
demoMode,
|
||||
skipRxResumeOnboarding,
|
||||
]);
|
||||
|
||||
const handleSaveLlm = async (): Promise<boolean> => {
|
||||
const handleSaveLlm = async () => {
|
||||
const values = getValues();
|
||||
const apiKeyValue = values.llmApiKey.trim();
|
||||
const baseUrlValue = values.llmBaseUrl.trim();
|
||||
|
||||
if (requiresLlmKey && !apiKeyValue && !hasLlmKey) {
|
||||
toast.info("Add your LLM API key to continue");
|
||||
return false;
|
||||
return;
|
||||
}
|
||||
|
||||
try {
|
||||
@@ -457,7 +197,7 @@ export const OnboardingGate: React.FC = () => {
|
||||
|
||||
if (!validation.valid) {
|
||||
toast.error(validation.message || "LLM validation failed");
|
||||
return false;
|
||||
return;
|
||||
}
|
||||
|
||||
const update: Partial<UpdateSettingsInput> = {
|
||||
@@ -484,165 +224,19 @@ export const OnboardingGate: React.FC = () => {
|
||||
? `Default for ${providerConfig.label}: ${defaultModel}.`
|
||||
: "Select the model manually in Settings > Model.",
|
||||
});
|
||||
return true;
|
||||
} catch (error) {
|
||||
const message =
|
||||
error instanceof Error ? error.message : "Failed to save LLM settings";
|
||||
toast.error(message);
|
||||
return false;
|
||||
} finally {
|
||||
setIsSavingEnv(false);
|
||||
}
|
||||
};
|
||||
|
||||
const handleSaveRxresume = async (): Promise<boolean> => {
|
||||
const values = getValues();
|
||||
const modeValue = values.rxresumeMode;
|
||||
const draftCredentials = getRxResumeCredentialDrafts(values);
|
||||
const missing = getRxResumeMissingCredentialLabels({
|
||||
mode: modeValue,
|
||||
stored: storedRxResume,
|
||||
draft: draftCredentials,
|
||||
});
|
||||
const isBusy = isSavingEnv || settingsLoading || isValidatingLlm;
|
||||
const progressValue = llmValidated ? 100 : 0;
|
||||
|
||||
if (missing.length > 0) {
|
||||
toast.info("Almost there", {
|
||||
description: `Missing: ${missing.join(", ")}`,
|
||||
});
|
||||
return false;
|
||||
}
|
||||
|
||||
try {
|
||||
setIsValidatingRxresume(true);
|
||||
const result = await validateAndMaybePersistRxResumeMode({
|
||||
mode: modeValue,
|
||||
stored: storedRxResume,
|
||||
draft: draftCredentials,
|
||||
validate: api.validateRxresume,
|
||||
persist: async (update) => {
|
||||
setIsSavingEnv(true);
|
||||
try {
|
||||
await api.updateSettings(update);
|
||||
await refreshSettings();
|
||||
} finally {
|
||||
setIsSavingEnv(false);
|
||||
}
|
||||
},
|
||||
persistOnSuccess: true,
|
||||
getPrecheckMessage: (failure) =>
|
||||
failure === "missing-v5-api-key"
|
||||
? "v5 API key required. Add a v5 API key, then test again."
|
||||
: "v4 email and password required. Add both credentials, then test again.",
|
||||
getValidationErrorMessage: (error) =>
|
||||
error instanceof Error ? error.message : "RxResume validation failed",
|
||||
getPersistErrorMessage: (error) =>
|
||||
error instanceof Error
|
||||
? error.message
|
||||
: "Failed to save RxResume credentials",
|
||||
});
|
||||
|
||||
setRxresumeVersionValidations((current) => ({
|
||||
...current,
|
||||
[modeValue]: {
|
||||
...result.validation,
|
||||
checked: true,
|
||||
testedAt: Date.now(),
|
||||
},
|
||||
}));
|
||||
setRxresumeValidation({ ...result.validation, checked: true });
|
||||
|
||||
if (!result.validation.valid) {
|
||||
toast.error(result.validation.message || "RxResume validation failed");
|
||||
return false;
|
||||
}
|
||||
setValue("rxresumePassword", "");
|
||||
setValue("rxresumeApiKey", "");
|
||||
|
||||
toast.success("RxResume connected");
|
||||
return true;
|
||||
} catch (error) {
|
||||
const message =
|
||||
error instanceof Error
|
||||
? error.message
|
||||
: "Failed to save RxResume credentials";
|
||||
toast.error(message);
|
||||
return false;
|
||||
} finally {
|
||||
setIsValidatingRxresume(false);
|
||||
setIsSavingEnv(false);
|
||||
}
|
||||
};
|
||||
|
||||
const handleSaveBaseResume = async (): Promise<boolean> => {
|
||||
const values = getValues();
|
||||
|
||||
if (!values.rxresumeBaseResumeId) {
|
||||
toast.info("Select a base resume to continue");
|
||||
return false;
|
||||
}
|
||||
|
||||
try {
|
||||
setIsSavingEnv(true);
|
||||
await api.updateSettings({
|
||||
rxresumeMode: values.rxresumeMode,
|
||||
rxresumeBaseResumeId: values.rxresumeBaseResumeId,
|
||||
});
|
||||
const validation = await validateBaseResume();
|
||||
if (!validation.valid) {
|
||||
toast.error(validation.message || "Base resume validation failed");
|
||||
return false;
|
||||
}
|
||||
|
||||
await refreshSettings();
|
||||
toast.success("Base resume set");
|
||||
return true;
|
||||
} catch (error) {
|
||||
const message =
|
||||
error instanceof Error ? error.message : "Failed to save base resume";
|
||||
toast.error(message);
|
||||
return false;
|
||||
} finally {
|
||||
setIsSavingEnv(false);
|
||||
}
|
||||
};
|
||||
|
||||
const resolvedStepIndex = currentStep
|
||||
? steps.findIndex((step) => step.id === currentStep)
|
||||
: 0;
|
||||
const stepIndex = resolvedStepIndex >= 0 ? resolvedStepIndex : 0;
|
||||
const completedSteps = steps.filter((step) => step.complete).length;
|
||||
const progressValue =
|
||||
steps.length > 0 ? Math.round((completedSteps / steps.length) * 100) : 0;
|
||||
const isBusy =
|
||||
isSavingEnv ||
|
||||
settingsLoading ||
|
||||
isValidatingLlm ||
|
||||
isValidatingRxresume ||
|
||||
isValidatingBaseResume;
|
||||
const canGoBack = stepIndex > 0;
|
||||
|
||||
const handlePrimaryAction = async () => {
|
||||
if (!currentStep) return;
|
||||
if (currentStep === "llm") {
|
||||
await handleSaveLlm();
|
||||
return;
|
||||
}
|
||||
if (currentStep === "rxresume") {
|
||||
await handleSaveRxresume();
|
||||
return;
|
||||
}
|
||||
if (currentStep === "baseresume") {
|
||||
await handleSaveBaseResume();
|
||||
return;
|
||||
}
|
||||
};
|
||||
|
||||
const handleBack = () => {
|
||||
if (!canGoBack) return;
|
||||
setCurrentStep(steps[stepIndex - 1]?.id ?? currentStep);
|
||||
};
|
||||
|
||||
if (!shouldOpen || !currentStep) return null;
|
||||
if (!shouldOpen) return null;
|
||||
|
||||
return (
|
||||
<AlertDialog open>
|
||||
@@ -654,241 +248,134 @@ export const OnboardingGate: React.FC = () => {
|
||||
<AlertDialogHeader>
|
||||
<AlertDialogTitle>Welcome to Job Ops</AlertDialogTitle>
|
||||
<AlertDialogDescription>
|
||||
Let's get your workspace ready. Add your keys and resume once,
|
||||
then the pipeline can run end-to-end.
|
||||
Connect your LLM provider to run job scoring, summaries, and
|
||||
tailoring. You can add a resume file later in Settings if you need
|
||||
PDF export or resume-based scoring.
|
||||
</AlertDialogDescription>
|
||||
</AlertDialogHeader>
|
||||
|
||||
<Tabs value={currentStep} onValueChange={setCurrentStep}>
|
||||
<TabsList className="grid h-auto w-full grid-cols-1 gap-2 border-b border-border/60 bg-transparent p-0 text-left sm:grid-cols-3">
|
||||
{steps.map((step, index) => {
|
||||
const isActive = step.id === currentStep;
|
||||
const isComplete = step.complete;
|
||||
|
||||
return (
|
||||
<FieldLabel
|
||||
key={step.id}
|
||||
className={cn(
|
||||
"w-full [&>[data-slot=field]]:border-0 [&>[data-slot=field]]:p-0 [&>[data-slot=field]]:rounded-none",
|
||||
step.disabled && "opacity-50 cursor-not-allowed",
|
||||
)}
|
||||
>
|
||||
<TabsTrigger
|
||||
value={step.id}
|
||||
disabled={step.disabled}
|
||||
className={cn(
|
||||
"w-full rounded-md hover:bg-muted/60 border-b-2 border-transparent px-3 py-4 text-left shadow-none",
|
||||
isActive
|
||||
? "border-primary !bg-muted/60 text-foreground"
|
||||
: "text-muted-foreground",
|
||||
)}
|
||||
>
|
||||
<Field orientation="horizontal" className="items-start">
|
||||
<FieldContent>
|
||||
<FieldTitle>{step.label}</FieldTitle>
|
||||
<FieldDescription>{step.subtitle}</FieldDescription>
|
||||
</FieldContent>
|
||||
<span
|
||||
className={cn(
|
||||
"mt-0.5 flex h-6 w-6 items-center justify-center rounded-md text-xs font-semibold",
|
||||
isComplete
|
||||
? "bg-primary text-primary-foreground"
|
||||
: "bg-muted text-muted-foreground",
|
||||
)}
|
||||
>
|
||||
{isComplete ? (
|
||||
<Check className="h-3.5 w-3.5" />
|
||||
) : (
|
||||
index + 1
|
||||
)}
|
||||
</span>
|
||||
</Field>
|
||||
</TabsTrigger>
|
||||
</FieldLabel>
|
||||
);
|
||||
})}
|
||||
</TabsList>
|
||||
|
||||
<TabsContent value="llm" className="space-y-4 pt-6">
|
||||
<div>
|
||||
<p className="text-sm font-semibold">Connect LLM provider</p>
|
||||
<p className="text-xs text-muted-foreground">
|
||||
Used for job scoring, summaries, and tailoring.
|
||||
</p>
|
||||
</div>
|
||||
<div className="grid gap-4 md:grid-cols-2">
|
||||
<div className="space-y-2">
|
||||
<label htmlFor="llmProvider" className="text-sm font-medium">
|
||||
Provider
|
||||
</label>
|
||||
<Controller
|
||||
name="llmProvider"
|
||||
control={control}
|
||||
render={({ field }) => (
|
||||
<Select
|
||||
value={selectedProvider}
|
||||
onValueChange={(value) => {
|
||||
field.onChange(value);
|
||||
}}
|
||||
disabled={isSavingEnv}
|
||||
>
|
||||
<SelectTrigger id="llmProvider">
|
||||
<SelectValue placeholder="Select provider" />
|
||||
</SelectTrigger>
|
||||
<SelectContent>
|
||||
{LLM_PROVIDERS.map((provider) => (
|
||||
<SelectItem key={provider} value={provider}>
|
||||
{LLM_PROVIDER_LABELS[provider]}
|
||||
</SelectItem>
|
||||
))}
|
||||
</SelectContent>
|
||||
</Select>
|
||||
)}
|
||||
/>
|
||||
<p className="text-xs text-muted-foreground">
|
||||
{providerConfig.providerHint}
|
||||
</p>
|
||||
</div>
|
||||
{showBaseUrl && (
|
||||
<Controller
|
||||
name="llmBaseUrl"
|
||||
control={control}
|
||||
render={({ field }) => (
|
||||
<SettingsInput
|
||||
label="LLM base URL"
|
||||
inputProps={{
|
||||
name: "llmBaseUrl",
|
||||
value: field.value,
|
||||
onChange: field.onChange,
|
||||
}}
|
||||
placeholder={providerConfig.baseUrlPlaceholder}
|
||||
helper={providerConfig.baseUrlHelper}
|
||||
current={settings?.llmBaseUrl?.value || "—"}
|
||||
disabled={isSavingEnv}
|
||||
/>
|
||||
)}
|
||||
/>
|
||||
)}
|
||||
{showApiKey && (
|
||||
<Controller
|
||||
name="llmApiKey"
|
||||
control={control}
|
||||
render={({ field }) => (
|
||||
<SettingsInput
|
||||
label="LLM API key"
|
||||
inputProps={{
|
||||
name: "llmApiKey",
|
||||
value: field.value,
|
||||
onChange: field.onChange,
|
||||
}}
|
||||
type="password"
|
||||
placeholder="Enter key"
|
||||
helper={
|
||||
llmKeyHint
|
||||
? `${providerConfig.keyHelper}. Leave blank to use the saved key.`
|
||||
: providerConfig.keyHelper
|
||||
}
|
||||
disabled={isSavingEnv}
|
||||
/>
|
||||
)}
|
||||
/>
|
||||
)}
|
||||
</div>
|
||||
</TabsContent>
|
||||
|
||||
<TabsContent value="rxresume" className="space-y-4 pt-6">
|
||||
<ReactiveResumeConfigPanel
|
||||
mode={rxresumeModeCurrent}
|
||||
onModeChange={(mode) => {
|
||||
setValue("rxresumeMode", mode);
|
||||
setValue(
|
||||
"rxresumeBaseResumeId",
|
||||
getBaseResumeIdForMode(mode),
|
||||
);
|
||||
setRxresumeValidation((previous) => ({
|
||||
...EMPTY_VALIDATION_STATE,
|
||||
checked: previous.checked,
|
||||
}));
|
||||
}}
|
||||
disabled={isSavingEnv}
|
||||
showValidationStatus
|
||||
validationStatuses={rxresumeVersionValidations}
|
||||
intro={{
|
||||
title: "Link your RxResume account",
|
||||
description:
|
||||
"Used to export tailored PDFs. Choose between Reactive Resume version 4 and 5, and provide the credentials.",
|
||||
}}
|
||||
v5={{
|
||||
apiKey: watch("rxresumeApiKey"),
|
||||
onApiKeyChange: (value) => setValue("rxresumeApiKey", value),
|
||||
}}
|
||||
shared={{
|
||||
baseUrl: watch("rxresumeUrl"),
|
||||
onBaseUrlChange: (value) => setValue("rxresumeUrl", value),
|
||||
}}
|
||||
v4={{
|
||||
email: watch("rxresumeEmail"),
|
||||
onEmailChange: (value) => setValue("rxresumeEmail", value),
|
||||
password: watch("rxresumePassword"),
|
||||
onPasswordChange: (value) =>
|
||||
setValue("rxresumePassword", value),
|
||||
}}
|
||||
/>
|
||||
</TabsContent>
|
||||
|
||||
<TabsContent value="baseresume" className="space-y-4 pt-6">
|
||||
<div>
|
||||
<p className="text-sm font-semibold">
|
||||
Select your template resume
|
||||
</p>
|
||||
<p className="text-xs text-muted-foreground">
|
||||
Choose the resume you want to use as a template. The selected
|
||||
resume will be used as a template for tailoring.
|
||||
</p>
|
||||
</div>
|
||||
<Controller
|
||||
name="rxresumeBaseResumeId"
|
||||
control={control}
|
||||
render={({ field }) => (
|
||||
<BaseResumeSelection
|
||||
value={field.value}
|
||||
onValueChange={(value) => {
|
||||
const mode = (getValues("rxresumeMode") ??
|
||||
"v5") as RxResumeMode;
|
||||
setBaseResumeIdForMode(mode, value);
|
||||
field.onChange(value);
|
||||
}}
|
||||
hasRxResumeAccess={rxresumeValidation.valid}
|
||||
rxresumeMode={rxresumeModeCurrent}
|
||||
disabled={isSavingEnv}
|
||||
/>
|
||||
)}
|
||||
/>
|
||||
</TabsContent>
|
||||
</Tabs>
|
||||
|
||||
<div className="flex items-center justify-between">
|
||||
<Button
|
||||
variant="outline"
|
||||
onClick={handleBack}
|
||||
disabled={!canGoBack || isBusy}
|
||||
>
|
||||
Back
|
||||
</Button>
|
||||
<div className="flex items-center gap-2">
|
||||
<Button onClick={handlePrimaryAction} disabled={isBusy}>
|
||||
{isBusy
|
||||
? "Validating..."
|
||||
: getStepPrimaryLabel({
|
||||
currentStep,
|
||||
llmValidated,
|
||||
rxresumeValidated: rxresumeValidation.valid,
|
||||
baseResumeValidated: baseResumeValidation.valid,
|
||||
})}
|
||||
</Button>
|
||||
<FieldLabel className="w-full [&>[data-slot=field]]:border-0 [&>[data-slot=field]]:p-0 [&>[data-slot=field]]:rounded-none">
|
||||
<div className="w-full rounded-md border-b-2 border-primary bg-muted/60 px-3 py-4 text-left">
|
||||
<Field orientation="horizontal" className="items-start">
|
||||
<FieldContent>
|
||||
<FieldTitle>LLM Provider</FieldTitle>
|
||||
<FieldDescription>Provider + credentials</FieldDescription>
|
||||
</FieldContent>
|
||||
<span
|
||||
className={cn(
|
||||
"mt-0.5 flex h-6 w-6 items-center justify-center rounded-md text-xs font-semibold",
|
||||
llmValidated
|
||||
? "bg-primary text-primary-foreground"
|
||||
: "bg-muted text-muted-foreground",
|
||||
)}
|
||||
>
|
||||
{llmValidated ? <Check className="h-3.5 w-3.5" /> : "1"}
|
||||
</span>
|
||||
</Field>
|
||||
</div>
|
||||
</FieldLabel>
|
||||
|
||||
<div className="space-y-4 pt-2">
|
||||
<div>
|
||||
<p className="text-sm font-semibold">Connect LLM provider</p>
|
||||
<p className="text-xs text-muted-foreground">
|
||||
Used for job scoring, summaries, and tailoring.
|
||||
</p>
|
||||
</div>
|
||||
<div className="grid gap-4 md:grid-cols-2">
|
||||
<div className="space-y-2">
|
||||
<label htmlFor="llmProvider" className="text-sm font-medium">
|
||||
Provider
|
||||
</label>
|
||||
<Controller
|
||||
name="llmProvider"
|
||||
control={control}
|
||||
render={({ field }) => (
|
||||
<Select
|
||||
value={selectedProvider}
|
||||
onValueChange={(value) => {
|
||||
field.onChange(value);
|
||||
}}
|
||||
disabled={isSavingEnv}
|
||||
>
|
||||
<SelectTrigger id="llmProvider">
|
||||
<SelectValue placeholder="Select provider" />
|
||||
</SelectTrigger>
|
||||
<SelectContent>
|
||||
{LLM_PROVIDERS.map((provider) => (
|
||||
<SelectItem key={provider} value={provider}>
|
||||
{LLM_PROVIDER_LABELS[provider]}
|
||||
</SelectItem>
|
||||
))}
|
||||
</SelectContent>
|
||||
</Select>
|
||||
)}
|
||||
/>
|
||||
<p className="text-xs text-muted-foreground">
|
||||
{providerConfig.providerHint}
|
||||
</p>
|
||||
</div>
|
||||
{showBaseUrl && (
|
||||
<Controller
|
||||
name="llmBaseUrl"
|
||||
control={control}
|
||||
render={({ field }) => (
|
||||
<SettingsInput
|
||||
label="LLM base URL"
|
||||
inputProps={{
|
||||
name: "llmBaseUrl",
|
||||
value: field.value,
|
||||
onChange: field.onChange,
|
||||
}}
|
||||
placeholder={providerConfig.baseUrlPlaceholder}
|
||||
helper={providerConfig.baseUrlHelper}
|
||||
current={settings?.llmBaseUrl?.value || "—"}
|
||||
disabled={isSavingEnv}
|
||||
/>
|
||||
)}
|
||||
/>
|
||||
)}
|
||||
{showApiKey && (
|
||||
<Controller
|
||||
name="llmApiKey"
|
||||
control={control}
|
||||
render={({ field }) => (
|
||||
<SettingsInput
|
||||
label="LLM API key"
|
||||
inputProps={{
|
||||
name: "llmApiKey",
|
||||
value: field.value,
|
||||
onChange: field.onChange,
|
||||
}}
|
||||
type="password"
|
||||
placeholder="Enter key"
|
||||
helper={
|
||||
llmKeyHint
|
||||
? `${providerConfig.keyHelper}. Leave blank to use the saved key.`
|
||||
: providerConfig.keyHelper
|
||||
}
|
||||
disabled={isSavingEnv}
|
||||
/>
|
||||
)}
|
||||
/>
|
||||
)}
|
||||
</div>
|
||||
{llmValidation.checked && !llmValidation.valid ? (
|
||||
<p className="text-xs text-destructive">
|
||||
{llmValidation.message ?? "LLM validation failed."}
|
||||
</p>
|
||||
) : null}
|
||||
</div>
|
||||
|
||||
<div className="flex items-center justify-end">
|
||||
<Button onClick={handleSaveLlm} disabled={isBusy}>
|
||||
{isBusy
|
||||
? "Validating..."
|
||||
: llmValidated
|
||||
? "Revalidate"
|
||||
: "Connect"}
|
||||
</Button>
|
||||
</div>
|
||||
<Progress value={progressValue} className="h-2" />
|
||||
</div>
|
||||
</AlertDialogContent>
|
||||
|
||||
@@ -1,453 +0,0 @@
|
||||
import { BaseResumeSelection } from "@client/pages/settings/components/BaseResumeSelection";
|
||||
import { SettingsInput } from "@client/pages/settings/components/SettingsInput";
|
||||
import {
|
||||
toggleAiSelectable,
|
||||
toggleMustInclude,
|
||||
} from "@client/pages/settings/resume-projects-state";
|
||||
import type { ResumeProjectsSettingsInput } from "@shared/settings-schema.js";
|
||||
import type { ResumeProjectCatalogItem, RxResumeMode } from "@shared/types.js";
|
||||
import { AlertCircle, AlertTriangle } from "lucide-react";
|
||||
import type React from "react";
|
||||
import { Alert, AlertDescription, AlertTitle } from "@/components/ui/alert";
|
||||
import { Checkbox } from "@/components/ui/checkbox";
|
||||
import { Input } from "@/components/ui/input";
|
||||
import { Separator } from "@/components/ui/separator";
|
||||
import {
|
||||
Table,
|
||||
TableBody,
|
||||
TableCell,
|
||||
TableHead,
|
||||
TableHeader,
|
||||
TableRow,
|
||||
} from "@/components/ui/table";
|
||||
import { Tabs, TabsList, TabsTrigger } from "@/components/ui/tabs";
|
||||
import { clampInt } from "@/lib/utils";
|
||||
import { StatusIndicator } from "./StatusIndicator";
|
||||
|
||||
type VersionValidationState = {
|
||||
checked: boolean;
|
||||
valid: boolean;
|
||||
message?: string | null;
|
||||
status?: number | null;
|
||||
};
|
||||
|
||||
type ProjectSelectionConfig = {
|
||||
baseResumeId: string | null;
|
||||
onBaseResumeIdChange: (value: string | null) => void;
|
||||
projects: ResumeProjectCatalogItem[];
|
||||
value: ResumeProjectsSettingsInput | null | undefined;
|
||||
onChange: (next: ResumeProjectsSettingsInput) => void;
|
||||
lockedCount: number;
|
||||
maxProjectsTotal: number;
|
||||
isProjectsLoading: boolean;
|
||||
disabled: boolean;
|
||||
maxProjectsError?: string;
|
||||
};
|
||||
|
||||
type ReactiveResumeConfigPanelProps = {
|
||||
mode: RxResumeMode;
|
||||
onModeChange: (mode: RxResumeMode) => void;
|
||||
disabled?: boolean;
|
||||
hasRxResumeAccess?: boolean;
|
||||
showValidationStatus?: boolean;
|
||||
validationStatuses?: {
|
||||
v4: VersionValidationState;
|
||||
v5: VersionValidationState;
|
||||
};
|
||||
intro?: {
|
||||
title: string;
|
||||
description?: string;
|
||||
};
|
||||
v5: {
|
||||
apiKey: string;
|
||||
onApiKeyChange: (value: string) => void;
|
||||
error?: string;
|
||||
helper?: string;
|
||||
placeholder?: string;
|
||||
};
|
||||
shared: {
|
||||
baseUrl: string;
|
||||
onBaseUrlChange: (value: string) => void;
|
||||
baseUrlError?: string;
|
||||
baseUrlHelper?: string;
|
||||
baseUrlPlaceholder?: string;
|
||||
};
|
||||
v4: {
|
||||
email: string;
|
||||
onEmailChange: (value: string) => void;
|
||||
emailError?: string;
|
||||
password: string;
|
||||
onPasswordChange: (value: string) => void;
|
||||
passwordError?: string;
|
||||
emailPlaceholder?: string;
|
||||
passwordPlaceholder?: string;
|
||||
};
|
||||
projectSelection?: ProjectSelectionConfig;
|
||||
};
|
||||
|
||||
function renderStatusPill(label: string, state: VersionValidationState) {
|
||||
const statusLabel = state.checked
|
||||
? state.valid
|
||||
? "Connected"
|
||||
: "Failed"
|
||||
: "Not tested";
|
||||
const dotColor = state.checked
|
||||
? state.valid
|
||||
? "bg-emerald-500"
|
||||
: "bg-destructive"
|
||||
: "bg-muted-foreground";
|
||||
|
||||
return (
|
||||
<StatusIndicator
|
||||
label={`${label}: ${statusLabel}`}
|
||||
dotColor={dotColor}
|
||||
tooltip={
|
||||
state.checked && !state.valid && state.message
|
||||
? state.message
|
||||
: undefined
|
||||
}
|
||||
/>
|
||||
);
|
||||
}
|
||||
|
||||
function isAvailabilityWarning(state?: VersionValidationState): boolean {
|
||||
const status = state?.status ?? null;
|
||||
return status === 0 || (typeof status === "number" && status >= 500);
|
||||
}
|
||||
|
||||
export const ReactiveResumeConfigPanel: React.FC<
|
||||
ReactiveResumeConfigPanelProps
|
||||
> = ({
|
||||
mode,
|
||||
onModeChange,
|
||||
disabled = false,
|
||||
hasRxResumeAccess = false,
|
||||
showValidationStatus = false,
|
||||
validationStatuses,
|
||||
intro,
|
||||
shared,
|
||||
v5,
|
||||
v4,
|
||||
projectSelection,
|
||||
}) => {
|
||||
const canShowProjectSelection = Boolean(
|
||||
projectSelection && hasRxResumeAccess,
|
||||
);
|
||||
const selectedValidationStatus = validationStatuses?.[mode];
|
||||
const showInlineValidationAlert = Boolean(
|
||||
selectedValidationStatus?.checked &&
|
||||
!selectedValidationStatus.valid &&
|
||||
selectedValidationStatus.message,
|
||||
);
|
||||
const selectedValidationIsWarning =
|
||||
showInlineValidationAlert &&
|
||||
isAvailabilityWarning(selectedValidationStatus);
|
||||
const handleModeChange = (value: string) =>
|
||||
onModeChange(value === "v4" ? "v4" : "v5");
|
||||
|
||||
return (
|
||||
<div className="space-y-4">
|
||||
{intro ? (
|
||||
<div>
|
||||
<p className="text-sm font-semibold">{intro.title}</p>
|
||||
{intro.description ? (
|
||||
<p className="text-xs text-muted-foreground">{intro.description}</p>
|
||||
) : null}
|
||||
</div>
|
||||
) : null}
|
||||
|
||||
<Tabs value={mode} onValueChange={handleModeChange}>
|
||||
<TabsList className="grid h-auto w-full grid-cols-2">
|
||||
<TabsTrigger value="v5" disabled={disabled}>
|
||||
v5 (API key)
|
||||
</TabsTrigger>
|
||||
<TabsTrigger value="v4" disabled={disabled}>
|
||||
v4 (Email + Password)
|
||||
</TabsTrigger>
|
||||
</TabsList>
|
||||
</Tabs>
|
||||
|
||||
{showValidationStatus && selectedValidationStatus ? (
|
||||
<div className="flex flex-wrap items-center gap-2 text-xs w-full justify-between">
|
||||
{renderStatusPill(`${mode} status`, selectedValidationStatus)}
|
||||
</div>
|
||||
) : null}
|
||||
|
||||
{showInlineValidationAlert && selectedValidationStatus?.message ? (
|
||||
<Alert
|
||||
variant={selectedValidationIsWarning ? "default" : "destructive"}
|
||||
className={
|
||||
selectedValidationIsWarning
|
||||
? "border-amber-200 bg-amber-50 text-amber-950 [&>svg]:text-amber-700"
|
||||
: undefined
|
||||
}
|
||||
>
|
||||
{selectedValidationIsWarning ? (
|
||||
<AlertTriangle className="h-4 w-4" />
|
||||
) : (
|
||||
<AlertCircle className="h-4 w-4" />
|
||||
)}
|
||||
<AlertTitle>
|
||||
Reactive Resume {mode.toUpperCase()}{" "}
|
||||
{selectedValidationIsWarning ? "warning" : "error"}
|
||||
</AlertTitle>
|
||||
<AlertDescription>
|
||||
{selectedValidationStatus.message}
|
||||
</AlertDescription>
|
||||
</Alert>
|
||||
) : null}
|
||||
|
||||
{mode === "v5" ? (
|
||||
<div className="grid gap-4">
|
||||
<SettingsInput
|
||||
label="RxResume URL"
|
||||
inputProps={{
|
||||
name: "rxresumeUrl",
|
||||
value: shared.baseUrl,
|
||||
onChange: (event) =>
|
||||
shared.onBaseUrlChange(event.currentTarget.value),
|
||||
}}
|
||||
type="url"
|
||||
placeholder={
|
||||
shared.baseUrlPlaceholder ?? "https://resume.example.com"
|
||||
}
|
||||
helper={
|
||||
shared.baseUrlHelper ??
|
||||
"Leave blank to use the default for the selected mode (or the RXRESUME_URL environment override, if set)."
|
||||
}
|
||||
disabled={disabled}
|
||||
error={shared.baseUrlError}
|
||||
/>
|
||||
<SettingsInput
|
||||
label="v5 API key"
|
||||
inputProps={{
|
||||
name: "rxresumeApiKey",
|
||||
value: v5.apiKey,
|
||||
onChange: (event) => v5.onApiKeyChange(event.currentTarget.value),
|
||||
}}
|
||||
type="password"
|
||||
placeholder={v5.placeholder ?? "Enter v5 API key"}
|
||||
helper={v5.helper}
|
||||
disabled={disabled}
|
||||
error={v5.error}
|
||||
/>
|
||||
</div>
|
||||
) : (
|
||||
<div className="grid gap-4 md:grid-cols-2">
|
||||
<div className="md:col-span-2">
|
||||
<SettingsInput
|
||||
label="RxResume URL"
|
||||
inputProps={{
|
||||
name: "rxresumeUrl",
|
||||
value: shared.baseUrl,
|
||||
onChange: (event) =>
|
||||
shared.onBaseUrlChange(event.currentTarget.value),
|
||||
}}
|
||||
type="url"
|
||||
placeholder={
|
||||
shared.baseUrlPlaceholder ?? "https://resume.example.com"
|
||||
}
|
||||
helper={
|
||||
shared.baseUrlHelper ??
|
||||
"Leave blank to use the public cloud default for the selected mode."
|
||||
}
|
||||
disabled={disabled}
|
||||
error={shared.baseUrlError}
|
||||
/>
|
||||
</div>
|
||||
<SettingsInput
|
||||
label="v4 Email"
|
||||
inputProps={{
|
||||
name: "rxresumeEmail",
|
||||
value: v4.email,
|
||||
onChange: (event) => v4.onEmailChange(event.currentTarget.value),
|
||||
}}
|
||||
placeholder={v4.emailPlaceholder ?? "you@example.com"}
|
||||
disabled={disabled}
|
||||
error={v4.emailError}
|
||||
/>
|
||||
<SettingsInput
|
||||
label="v4 Password"
|
||||
inputProps={{
|
||||
name: "rxresumePassword",
|
||||
value: v4.password,
|
||||
onChange: (event) =>
|
||||
v4.onPasswordChange(event.currentTarget.value),
|
||||
}}
|
||||
type="password"
|
||||
placeholder={v4.passwordPlaceholder ?? "Enter v4 password"}
|
||||
disabled={disabled}
|
||||
error={v4.passwordError}
|
||||
/>
|
||||
</div>
|
||||
)}
|
||||
|
||||
{projectSelection ? (
|
||||
<>
|
||||
<Separator />
|
||||
|
||||
{!canShowProjectSelection ? (
|
||||
<div className="rounded-md border border-dashed border-muted-foreground/40 bg-muted/30 px-4 py-3 text-sm text-muted-foreground">
|
||||
Connect Reactive Resume and choose a template resume to configure
|
||||
resume projects.
|
||||
</div>
|
||||
) : (
|
||||
<div className="space-y-4">
|
||||
<BaseResumeSelection
|
||||
value={projectSelection.baseResumeId}
|
||||
onValueChange={projectSelection.onBaseResumeIdChange}
|
||||
hasRxResumeAccess={hasRxResumeAccess}
|
||||
rxresumeMode={mode}
|
||||
disabled={projectSelection.disabled}
|
||||
/>
|
||||
|
||||
{!projectSelection.baseResumeId ? (
|
||||
<div className="rounded-md border border-dashed border-muted-foreground/40 bg-muted/30 px-4 py-3 text-sm text-muted-foreground">
|
||||
Choose a PDF to configure resume projects.
|
||||
</div>
|
||||
) : (
|
||||
<>
|
||||
<div className="space-y-2">
|
||||
<div className="text-sm font-medium">
|
||||
Max projects to choose
|
||||
</div>
|
||||
<Input
|
||||
type="number"
|
||||
inputMode="numeric"
|
||||
min={projectSelection.lockedCount}
|
||||
max={projectSelection.maxProjectsTotal}
|
||||
value={projectSelection.value?.maxProjects ?? 0}
|
||||
onChange={(event) => {
|
||||
if (!projectSelection.value) return;
|
||||
const next = Number(event.target.value);
|
||||
const clamped = clampInt(
|
||||
next,
|
||||
projectSelection.lockedCount,
|
||||
projectSelection.maxProjectsTotal,
|
||||
);
|
||||
projectSelection.onChange({
|
||||
...projectSelection.value,
|
||||
maxProjects: clamped,
|
||||
});
|
||||
}}
|
||||
disabled={
|
||||
projectSelection.disabled ||
|
||||
projectSelection.isProjectsLoading ||
|
||||
!projectSelection.value
|
||||
}
|
||||
/>
|
||||
{projectSelection.maxProjectsError ? (
|
||||
<p className="text-xs text-destructive">
|
||||
{projectSelection.maxProjectsError}
|
||||
</p>
|
||||
) : null}
|
||||
</div>
|
||||
|
||||
<Table>
|
||||
<TableHeader>
|
||||
<TableRow>
|
||||
<TableHead className="text-xs whitespace-wrap sm:whitespace-nowrap">
|
||||
Project
|
||||
</TableHead>
|
||||
<TableHead className="text-xs whitespace-wrap sm:whitespace-nowrap">
|
||||
Visible in template
|
||||
</TableHead>
|
||||
<TableHead className="text-xs whitespace-wrap sm:whitespace-nowrap">
|
||||
Must Include
|
||||
</TableHead>
|
||||
<TableHead className="text-xs whitespace-wrap sm:whitespace-nowrap">
|
||||
AI selectable
|
||||
</TableHead>
|
||||
</TableRow>
|
||||
</TableHeader>
|
||||
|
||||
<TableBody>
|
||||
{projectSelection.projects.map((project) => {
|
||||
const value = projectSelection.value;
|
||||
const locked = Boolean(
|
||||
value?.lockedProjectIds.includes(project.id),
|
||||
);
|
||||
const aiSelectable = Boolean(
|
||||
value?.aiSelectableProjectIds.includes(project.id),
|
||||
);
|
||||
const projectMeta =
|
||||
mode === "v5"
|
||||
? project.date
|
||||
: [project.description, project.date]
|
||||
.filter(Boolean)
|
||||
.join(" - ");
|
||||
|
||||
return (
|
||||
<TableRow key={project.id}>
|
||||
<TableCell>
|
||||
<div className="space-y-0.5">
|
||||
<div className="font-medium">
|
||||
{project.name}
|
||||
</div>
|
||||
{projectMeta ? (
|
||||
<div className="text-xs text-muted-foreground">
|
||||
{projectMeta}
|
||||
</div>
|
||||
) : null}
|
||||
</div>
|
||||
</TableCell>
|
||||
<TableCell>
|
||||
{project.isVisibleInBase ? "Yes" : "No"}
|
||||
</TableCell>
|
||||
<TableCell>
|
||||
<Checkbox
|
||||
checked={locked}
|
||||
onCheckedChange={() => {
|
||||
if (!value) return;
|
||||
projectSelection.onChange(
|
||||
toggleMustInclude({
|
||||
settings: value,
|
||||
projectId: project.id,
|
||||
checked: !locked,
|
||||
maxProjectsTotal:
|
||||
projectSelection.maxProjectsTotal,
|
||||
}),
|
||||
);
|
||||
}}
|
||||
disabled={
|
||||
projectSelection.disabled ||
|
||||
projectSelection.isProjectsLoading ||
|
||||
!value
|
||||
}
|
||||
/>
|
||||
</TableCell>
|
||||
<TableCell>
|
||||
<Checkbox
|
||||
checked={locked ? true : aiSelectable}
|
||||
onCheckedChange={() => {
|
||||
if (!value) return;
|
||||
projectSelection.onChange(
|
||||
toggleAiSelectable({
|
||||
settings: value,
|
||||
projectId: project.id,
|
||||
checked: !aiSelectable,
|
||||
}),
|
||||
);
|
||||
}}
|
||||
disabled={
|
||||
projectSelection.disabled ||
|
||||
projectSelection.isProjectsLoading ||
|
||||
locked ||
|
||||
!value
|
||||
}
|
||||
/>
|
||||
</TableCell>
|
||||
</TableRow>
|
||||
);
|
||||
})}
|
||||
</TableBody>
|
||||
</Table>
|
||||
</>
|
||||
)}
|
||||
</div>
|
||||
)}
|
||||
</>
|
||||
) : null}
|
||||
</div>
|
||||
);
|
||||
};
|
||||
@@ -51,7 +51,6 @@ import { useRescoreJob } from "../hooks/useRescoreJob";
|
||||
import { FitAssessment, JobHeader, JobNotes, TailoredSummary } from ".";
|
||||
import { CoverLetterDisplay } from "./CoverLetterDisplay";
|
||||
import { TailorMode } from "./discovered-panel/TailorMode";
|
||||
import { GhostwriterDrawer } from "./ghostwriter/GhostwriterDrawer";
|
||||
import { JobDetailsEditDrawer } from "./JobDetailsEditDrawer";
|
||||
import { KbdHint } from "./KbdHint";
|
||||
import { OpenJobListingButton } from "./OpenJobListingButton";
|
||||
@@ -380,11 +379,6 @@ export const ReadyPanel: React.FC<ReadyPanelProps> = ({
|
||||
───────────────────────────────────────────────────────────────────── */}
|
||||
<div className="pb-4 border-b border-border/40">
|
||||
<div className="grid gap-2 sm:grid-cols-2">
|
||||
<GhostwriterDrawer
|
||||
job={job}
|
||||
triggerClassName="h-9 w-full justify-center gap-1 px-2 text-xs"
|
||||
/>
|
||||
|
||||
{/* Download PDF - primary artifact action */}
|
||||
<Button
|
||||
asChild
|
||||
|
||||
@@ -0,0 +1,74 @@
|
||||
import {
|
||||
parseSponsorshipSignals,
|
||||
SPONSORSHIP_GREEN_FLAG_IDS,
|
||||
SPONSORSHIP_RED_FLAG_IDS,
|
||||
SPONSORSHIP_SIGNAL_META,
|
||||
SPONSORSHIP_YELLOW_FLAG_IDS,
|
||||
type SponsorshipFlagTier,
|
||||
} from "@shared/sponsorship-signals";
|
||||
import type React from "react";
|
||||
import { Badge } from "@/components/ui/badge";
|
||||
import {
|
||||
Tooltip,
|
||||
TooltipContent,
|
||||
TooltipProvider,
|
||||
TooltipTrigger,
|
||||
} from "@/components/ui/tooltip";
|
||||
import { cn } from "@/lib/utils";
|
||||
|
||||
const flagTierClass: Record<SponsorshipFlagTier, string> = {
|
||||
green: "border-emerald-500/30 bg-emerald-500/10 text-emerald-200",
|
||||
yellow: "border-amber-500/40 bg-amber-500/10 text-amber-200",
|
||||
red: "border-rose-500/40 bg-rose-500/10 text-rose-200",
|
||||
};
|
||||
|
||||
interface SponsorshipSignalsPillsProps {
|
||||
sponsorshipSignals: string | null | undefined;
|
||||
className?: string;
|
||||
size?: "xs" | "sm";
|
||||
}
|
||||
|
||||
export const SponsorshipSignalsPills: React.FC<
|
||||
SponsorshipSignalsPillsProps
|
||||
> = ({ sponsorshipSignals, className, size = "sm" }) => {
|
||||
const signals = parseSponsorshipSignals(sponsorshipSignals);
|
||||
if (signals.length === 0) return null;
|
||||
|
||||
const ordered = [
|
||||
...SPONSORSHIP_RED_FLAG_IDS.filter((id) => signals.includes(id)),
|
||||
...SPONSORSHIP_YELLOW_FLAG_IDS.filter((id) => signals.includes(id)),
|
||||
...SPONSORSHIP_GREEN_FLAG_IDS.filter((id) => signals.includes(id)),
|
||||
];
|
||||
|
||||
return (
|
||||
<TooltipProvider>
|
||||
<div className={cn("flex flex-wrap items-center gap-1", className)}>
|
||||
{ordered.map((signal) => {
|
||||
const meta = SPONSORSHIP_SIGNAL_META[signal];
|
||||
return (
|
||||
<Tooltip key={signal} delayDuration={0}>
|
||||
<TooltipTrigger asChild>
|
||||
<Badge
|
||||
variant="outline"
|
||||
className={cn(
|
||||
"font-medium",
|
||||
size === "xs" ? "px-1 py-0 text-[10px]" : "text-xs",
|
||||
flagTierClass[meta.flagTier],
|
||||
)}
|
||||
>
|
||||
{meta.shortLabel}
|
||||
</Badge>
|
||||
</TooltipTrigger>
|
||||
<TooltipContent side="top" className="max-w-xs">
|
||||
<p className="text-xs font-medium">{meta.label}</p>
|
||||
<p className="text-xs text-muted-foreground">
|
||||
{meta.description}
|
||||
</p>
|
||||
</TooltipContent>
|
||||
</Tooltip>
|
||||
);
|
||||
})}
|
||||
</div>
|
||||
</TooltipProvider>
|
||||
);
|
||||
};
|
||||
@@ -1,24 +1,20 @@
|
||||
import {
|
||||
Columns3,
|
||||
Home,
|
||||
Inbox,
|
||||
LayoutDashboard,
|
||||
Link2,
|
||||
Settings,
|
||||
Shield,
|
||||
} from "lucide-react";
|
||||
import { LayoutDashboard, Settings } from "lucide-react";
|
||||
|
||||
export type NavLink = {
|
||||
to: string;
|
||||
label: string;
|
||||
icon: typeof Home;
|
||||
icon: typeof LayoutDashboard;
|
||||
activePaths?: string[];
|
||||
};
|
||||
|
||||
/**
|
||||
* Slim product nav: pipeline + scoring only.
|
||||
* Overview / kanban / inbox / tracer / visa browser stay routable for
|
||||
* bookmarks but are intentionally omitted from the chrome.
|
||||
*/
|
||||
export const NAV_LINKS: NavLink[] = [
|
||||
{ to: "/overview", label: "Overview", icon: Home },
|
||||
{
|
||||
to: "/jobs/ready",
|
||||
to: "/jobs/discovered",
|
||||
label: "Jobs",
|
||||
icon: LayoutDashboard,
|
||||
activePaths: [
|
||||
@@ -28,20 +24,6 @@ export const NAV_LINKS: NavLink[] = [
|
||||
"/jobs/all",
|
||||
],
|
||||
},
|
||||
{
|
||||
to: "/applications/in-progress",
|
||||
label: "In Progress",
|
||||
icon: Columns3,
|
||||
activePaths: ["/applications/in-progress"],
|
||||
},
|
||||
{ to: "/tracking-inbox", label: "Tracking Inbox", icon: Inbox },
|
||||
{
|
||||
to: "/tracer-links",
|
||||
label: "Tracer Links",
|
||||
icon: Link2,
|
||||
activePaths: ["/tracer-links"],
|
||||
},
|
||||
{ to: "/visa-sponsors", label: "Visa Sponsors", icon: Shield },
|
||||
{ to: "/settings", label: "Settings", icon: Settings },
|
||||
];
|
||||
|
||||
|
||||
@@ -1,55 +0,0 @@
|
||||
import {
|
||||
getRxResumeBaseResumeSelection,
|
||||
getStoredRxResumeCredentialAvailability,
|
||||
type RxResumeSettingsLike,
|
||||
} from "@client/lib/rxresume-config";
|
||||
import type { RxResumeMode } from "@shared/types.js";
|
||||
import { useCallback, useMemo, useState } from "react";
|
||||
|
||||
const EMPTY_IDS_BY_MODE: Record<RxResumeMode, string | null> = {
|
||||
v4: null,
|
||||
v5: null,
|
||||
};
|
||||
|
||||
export function useRxResumeConfigState(settings: RxResumeSettingsLike) {
|
||||
const storedRxResume = useMemo(
|
||||
() => getStoredRxResumeCredentialAvailability(settings),
|
||||
[settings],
|
||||
);
|
||||
const [baseResumeIdsByMode, setBaseResumeIdsByMode] =
|
||||
useState<Record<RxResumeMode, string | null>>(EMPTY_IDS_BY_MODE);
|
||||
|
||||
const syncBaseResumeIdsForMode = useCallback(
|
||||
(mode: RxResumeMode) => {
|
||||
const { idsByMode, selectedId } = getRxResumeBaseResumeSelection(
|
||||
settings,
|
||||
mode,
|
||||
);
|
||||
setBaseResumeIdsByMode(idsByMode);
|
||||
return selectedId;
|
||||
},
|
||||
[settings],
|
||||
);
|
||||
|
||||
const getBaseResumeIdForMode = useCallback(
|
||||
(mode: RxResumeMode) => baseResumeIdsByMode[mode] ?? null,
|
||||
[baseResumeIdsByMode],
|
||||
);
|
||||
|
||||
const setBaseResumeIdForMode = useCallback(
|
||||
(mode: RxResumeMode, value: string | null) => {
|
||||
setBaseResumeIdsByMode((prev) =>
|
||||
prev[mode] === value ? prev : { ...prev, [mode]: value },
|
||||
);
|
||||
},
|
||||
[],
|
||||
);
|
||||
|
||||
return {
|
||||
storedRxResume,
|
||||
baseResumeIdsByMode,
|
||||
syncBaseResumeIdsForMode,
|
||||
getBaseResumeIdForMode,
|
||||
setBaseResumeIdForMode,
|
||||
};
|
||||
}
|
||||
@@ -1,8 +1,17 @@
|
||||
import { PRIOR_SKIP_DISMISS_LOOKBACK_MS } from "@shared/job-fingerprint";
|
||||
import { createJob } from "@shared/testing/factories";
|
||||
import { describe, expect, it } from "vitest";
|
||||
import { buildDuplicateDismissHints } from "./job-dedup";
|
||||
|
||||
describe("buildDuplicateDismissHints", () => {
|
||||
const now = Date.parse("2026-06-10T12:00:00.000Z");
|
||||
const recentUpdatedAt = new Date(
|
||||
now - PRIOR_SKIP_DISMISS_LOOKBACK_MS + 86_400_000,
|
||||
).toISOString();
|
||||
const staleUpdatedAt = new Date(
|
||||
now - PRIOR_SKIP_DISMISS_LOOKBACK_MS - 86_400_000,
|
||||
).toISOString();
|
||||
|
||||
it("flags open jobs that match a skipped posting", () => {
|
||||
const jobs = [
|
||||
createJob({
|
||||
@@ -10,6 +19,7 @@ describe("buildDuplicateDismissHints", () => {
|
||||
employer: "Acme",
|
||||
title: "SDET",
|
||||
status: "skipped",
|
||||
updatedAt: recentUpdatedAt,
|
||||
}),
|
||||
createJob({
|
||||
id: "open-1",
|
||||
@@ -25,8 +35,71 @@ describe("buildDuplicateDismissHints", () => {
|
||||
}),
|
||||
];
|
||||
|
||||
const hints = buildDuplicateDismissHints(jobs);
|
||||
const hints = buildDuplicateDismissHints(jobs, now);
|
||||
expect(hints.get("open-1")).toBe("skipped");
|
||||
expect(hints.has("open-2")).toBe(false);
|
||||
});
|
||||
|
||||
it("matches employer prefix and plural title variants", () => {
|
||||
const jobs = [
|
||||
createJob({
|
||||
id: "skipped-1",
|
||||
employer: "CGI",
|
||||
title: "Automation Test Engineer",
|
||||
status: "skipped",
|
||||
updatedAt: recentUpdatedAt,
|
||||
}),
|
||||
createJob({
|
||||
id: "open-1",
|
||||
employer: ". CGI IT UK Limited",
|
||||
title: "Automation Test Engineers",
|
||||
status: "discovered",
|
||||
}),
|
||||
];
|
||||
|
||||
const hints = buildDuplicateDismissHints(jobs, now);
|
||||
expect(hints.get("open-1")).toBe("skipped");
|
||||
});
|
||||
|
||||
it("ignores skipped jobs older than the lookback window", () => {
|
||||
const jobs = [
|
||||
createJob({
|
||||
id: "skipped-1",
|
||||
employer: "Acme",
|
||||
title: "SDET",
|
||||
status: "skipped",
|
||||
updatedAt: staleUpdatedAt,
|
||||
}),
|
||||
createJob({
|
||||
id: "open-1",
|
||||
employer: "Acme",
|
||||
title: "SDET",
|
||||
status: "discovered",
|
||||
}),
|
||||
];
|
||||
|
||||
const hints = buildDuplicateDismissHints(jobs, now);
|
||||
expect(hints.has("open-1")).toBe(false);
|
||||
});
|
||||
|
||||
it("does not match different titles at the same employer", () => {
|
||||
const jobs = [
|
||||
createJob({
|
||||
id: "skipped-1",
|
||||
employer: "Cognizant",
|
||||
title: "Automation Test Engineer",
|
||||
status: "skipped",
|
||||
updatedAt: recentUpdatedAt,
|
||||
}),
|
||||
createJob({
|
||||
id: "open-1",
|
||||
employer: "Cognizant",
|
||||
title: "AI Automation Test Engineer",
|
||||
status: "discovered",
|
||||
}),
|
||||
];
|
||||
|
||||
const hints = buildDuplicateDismissHints(jobs, now);
|
||||
expect(hints.has("open-1")).toBe(false);
|
||||
});
|
||||
});
|
||||
|
||||
@@ -1,44 +1,60 @@
|
||||
import { collectJobDedupKeys } from "@shared/job-fingerprint";
|
||||
import {
|
||||
jobsMatchEmployerAndTitleDismiss,
|
||||
PRIOR_SKIP_DISMISS_LOOKBACK_MS,
|
||||
} from "@shared/job-fingerprint";
|
||||
import type { JobListItem, JobStatus } from "@shared/types";
|
||||
|
||||
export type DuplicateDismissReason = "skipped" | "applied";
|
||||
|
||||
const OPEN_JOB_STATUSES = new Set<JobStatus>([
|
||||
"discovered",
|
||||
"ready",
|
||||
"processing",
|
||||
]);
|
||||
|
||||
function isWithinDismissLookback(updatedAt: string, nowMs: number): boolean {
|
||||
const updatedMs = Date.parse(updatedAt);
|
||||
if (!Number.isFinite(updatedMs)) return false;
|
||||
return nowMs - updatedMs <= PRIOR_SKIP_DISMISS_LOOKBACK_MS;
|
||||
}
|
||||
|
||||
/**
|
||||
* Map open jobs to a prior skip/apply when employer+title or description matches.
|
||||
* Map open jobs to a prior skip/apply when the same company and title match
|
||||
* within the last 90 days (see {@link PRIOR_SKIP_DISMISS_LOOKBACK_MS}).
|
||||
*/
|
||||
export function buildDuplicateDismissHints(
|
||||
jobs: readonly JobListItem[],
|
||||
nowMs: number = Date.now(),
|
||||
): Map<string, DuplicateDismissReason> {
|
||||
const dismissedKeys = new Map<string, DuplicateDismissReason>();
|
||||
const dismissed: Array<{
|
||||
employer: string;
|
||||
title: string;
|
||||
reason: DuplicateDismissReason;
|
||||
}> = [];
|
||||
|
||||
for (const job of jobs) {
|
||||
if (job.status !== "skipped" && job.status !== "applied") continue;
|
||||
const reason: DuplicateDismissReason =
|
||||
job.status === "applied" ? "applied" : "skipped";
|
||||
for (const key of collectJobDedupKeys({
|
||||
if (!isWithinDismissLookback(job.updatedAt, nowMs)) continue;
|
||||
dismissed.push({
|
||||
employer: job.employer,
|
||||
title: job.title,
|
||||
})) {
|
||||
if (!dismissedKeys.has(key)) dismissedKeys.set(key, reason);
|
||||
}
|
||||
reason: job.status === "applied" ? "applied" : "skipped",
|
||||
});
|
||||
}
|
||||
|
||||
const hints = new Map<string, DuplicateDismissReason>();
|
||||
const openStatuses = new Set<JobStatus>([
|
||||
"discovered",
|
||||
"ready",
|
||||
"processing",
|
||||
]);
|
||||
|
||||
for (const job of jobs) {
|
||||
if (!openStatuses.has(job.status)) continue;
|
||||
for (const key of collectJobDedupKeys({
|
||||
employer: job.employer,
|
||||
title: job.title,
|
||||
})) {
|
||||
const reason = dismissedKeys.get(key);
|
||||
if (reason) {
|
||||
hints.set(job.id, reason);
|
||||
if (!OPEN_JOB_STATUSES.has(job.status)) continue;
|
||||
for (const prior of dismissed) {
|
||||
if (
|
||||
jobsMatchEmployerAndTitleDismiss({
|
||||
employerA: job.employer,
|
||||
titleA: job.title,
|
||||
employerB: prior.employer,
|
||||
titleB: prior.title,
|
||||
})
|
||||
) {
|
||||
hints.set(job.id, prior.reason);
|
||||
break;
|
||||
}
|
||||
}
|
||||
@@ -46,5 +62,3 @@ export function buildDuplicateDismissHints(
|
||||
|
||||
return hints;
|
||||
}
|
||||
|
||||
export { collectJobDedupKeys };
|
||||
|
||||
@@ -1,288 +0,0 @@
|
||||
import type { UpdateSettingsInput } from "@shared/settings-schema.js";
|
||||
import type { RxResumeMode, ValidationResult } from "@shared/types.js";
|
||||
|
||||
export type RxResumeSettingsLike =
|
||||
| {
|
||||
rxresumeMode?: { value?: string | null } | null;
|
||||
rxresumeEmail?: string | null;
|
||||
rxresumeUrl?: string | null;
|
||||
rxresumePasswordHint?: string | null;
|
||||
rxresumeApiKeyHint?: string | null;
|
||||
rxresumeBaseResumeId?: string | null;
|
||||
rxresumeBaseResumeIdV4?: string | null;
|
||||
rxresumeBaseResumeIdV5?: string | null;
|
||||
}
|
||||
| null
|
||||
| undefined;
|
||||
|
||||
export const RXRESUME_MODES = ["v4", "v5"] as const;
|
||||
|
||||
export const RXRESUME_PRECHECK_MESSAGES = {
|
||||
"missing-v4-email-password": "Add v4 email and password, then test again.",
|
||||
"missing-v5-api-key": "Add a v5 API key, then test again.",
|
||||
} as const;
|
||||
|
||||
export const coerceRxResumeMode = (
|
||||
value: unknown,
|
||||
fallback: RxResumeMode = "v5",
|
||||
): RxResumeMode => (value === "v4" || value === "v5" ? value : fallback);
|
||||
|
||||
export const getStoredRxResumeCredentialAvailability = (
|
||||
settings: RxResumeSettingsLike,
|
||||
) => {
|
||||
const email = Boolean(settings?.rxresumeEmail?.trim());
|
||||
const password = Boolean(settings?.rxresumePasswordHint);
|
||||
const apiKey = Boolean(settings?.rxresumeApiKeyHint);
|
||||
return { email, password, apiKey, hasV4: email && password, hasV5: apiKey };
|
||||
};
|
||||
|
||||
export const getInitialRxResumeMode = (input: {
|
||||
savedMode: RxResumeMode | null | undefined;
|
||||
hasV4: boolean;
|
||||
hasV5: boolean;
|
||||
}): RxResumeMode =>
|
||||
coerceRxResumeMode(
|
||||
input.savedMode ?? (input.hasV4 && !input.hasV5 ? "v4" : "v5"),
|
||||
);
|
||||
|
||||
export const getRxResumeBaseResumeSelection = (
|
||||
settings: RxResumeSettingsLike,
|
||||
mode: RxResumeMode,
|
||||
) => {
|
||||
const idsByMode = {
|
||||
v4:
|
||||
settings?.rxresumeBaseResumeIdV4 ??
|
||||
(mode === "v4" ? (settings?.rxresumeBaseResumeId ?? null) : null),
|
||||
v5:
|
||||
settings?.rxresumeBaseResumeIdV5 ??
|
||||
(mode === "v5" ? (settings?.rxresumeBaseResumeId ?? null) : null),
|
||||
} satisfies Record<RxResumeMode, string | null>;
|
||||
return { idsByMode, selectedId: idsByMode[mode] ?? null };
|
||||
};
|
||||
|
||||
export const getRxResumeCredentialDrafts = (input: {
|
||||
rxresumeEmail?: string | null;
|
||||
rxresumeUrl?: string | null;
|
||||
rxresumePassword?: string | null;
|
||||
rxresumeApiKey?: string | null;
|
||||
}) => ({
|
||||
email: input.rxresumeEmail?.trim() ?? "",
|
||||
baseUrl: input.rxresumeUrl?.trim() ?? "",
|
||||
password: input.rxresumePassword?.trim() ?? "",
|
||||
apiKey: input.rxresumeApiKey?.trim() ?? "",
|
||||
});
|
||||
|
||||
export type RxResumeCredentialDrafts = ReturnType<
|
||||
typeof getRxResumeCredentialDrafts
|
||||
>;
|
||||
export type RxResumeStoredCredentialAvailability = Pick<
|
||||
ReturnType<typeof getStoredRxResumeCredentialAvailability>,
|
||||
"email" | "password" | "apiKey"
|
||||
>;
|
||||
|
||||
export const getRxResumeCredentialPrecheckFailure = (input: {
|
||||
mode: RxResumeMode;
|
||||
stored: RxResumeStoredCredentialAvailability;
|
||||
draft: RxResumeCredentialDrafts;
|
||||
}) => {
|
||||
const hasV4 =
|
||||
(input.stored.email || Boolean(input.draft.email)) &&
|
||||
(input.stored.password || Boolean(input.draft.password));
|
||||
const hasV5 = input.stored.apiKey || Boolean(input.draft.apiKey);
|
||||
if (input.mode === "v5" && !hasV5) return "missing-v5-api-key" as const;
|
||||
if (input.mode === "v4" && !hasV4)
|
||||
return "missing-v4-email-password" as const;
|
||||
return null;
|
||||
};
|
||||
|
||||
export type RxResumeCredentialPrecheckFailure = ReturnType<
|
||||
typeof getRxResumeCredentialPrecheckFailure
|
||||
>;
|
||||
|
||||
export const getRxResumeMissingCredentialLabels = (input: {
|
||||
mode: RxResumeMode;
|
||||
stored: RxResumeStoredCredentialAvailability;
|
||||
draft: RxResumeCredentialDrafts;
|
||||
}) =>
|
||||
input.mode === "v5"
|
||||
? input.stored.apiKey || input.draft.apiKey
|
||||
? []
|
||||
: ["RxResume v5 API key"]
|
||||
: [
|
||||
...(input.stored.email || input.draft.email ? [] : ["RxResume email"]),
|
||||
...(input.stored.password || input.draft.password
|
||||
? []
|
||||
: ["RxResume password"]),
|
||||
];
|
||||
|
||||
export const toRxResumeValidationPayload = (
|
||||
draft: RxResumeCredentialDrafts,
|
||||
options?: {
|
||||
preserveBlankFields?: Array<keyof RxResumeCredentialDrafts>;
|
||||
},
|
||||
) => {
|
||||
const preserveBlankFields = new Set(options?.preserveBlankFields ?? []);
|
||||
return {
|
||||
email: preserveBlankFields.has("email")
|
||||
? draft.email
|
||||
: draft.email || undefined,
|
||||
baseUrl: preserveBlankFields.has("baseUrl")
|
||||
? draft.baseUrl
|
||||
: draft.baseUrl || undefined,
|
||||
password: preserveBlankFields.has("password")
|
||||
? draft.password
|
||||
: draft.password || undefined,
|
||||
apiKey: preserveBlankFields.has("apiKey")
|
||||
? draft.apiKey
|
||||
: draft.apiKey || undefined,
|
||||
};
|
||||
};
|
||||
|
||||
export const isRxResumeBlockingValidationFailure = (
|
||||
validation: ValidationResult,
|
||||
): boolean =>
|
||||
!validation.valid &&
|
||||
typeof validation.status === "number" &&
|
||||
validation.status >= 400 &&
|
||||
validation.status < 500;
|
||||
|
||||
export const isRxResumeAvailabilityValidationFailure = (
|
||||
validation: ValidationResult,
|
||||
): boolean =>
|
||||
!validation.valid &&
|
||||
(validation.status === 0 ||
|
||||
(typeof validation.status === "number" && validation.status >= 500));
|
||||
|
||||
export const buildRxResumeSettingsUpdate = (
|
||||
mode: RxResumeMode,
|
||||
draft: RxResumeCredentialDrafts,
|
||||
): Partial<UpdateSettingsInput> => {
|
||||
const update: Partial<UpdateSettingsInput> = {
|
||||
rxresumeMode: mode,
|
||||
rxresumeUrl: draft.baseUrl || null,
|
||||
};
|
||||
if (draft.email) update.rxresumeEmail = draft.email;
|
||||
if (draft.password) update.rxresumePassword = draft.password;
|
||||
if (draft.apiKey) update.rxresumeApiKey = draft.apiKey;
|
||||
return update;
|
||||
};
|
||||
|
||||
type ValidateAndMaybePersistRxResumeModeInput<TSettings> = {
|
||||
mode: RxResumeMode;
|
||||
stored: RxResumeStoredCredentialAvailability;
|
||||
draft: RxResumeCredentialDrafts;
|
||||
validate: (
|
||||
payload: { mode: RxResumeMode } & ReturnType<
|
||||
typeof toRxResumeValidationPayload
|
||||
>,
|
||||
) => Promise<ValidationResult>;
|
||||
persist?: (update: Partial<UpdateSettingsInput>) => Promise<TSettings>;
|
||||
persistOnSuccess?: boolean;
|
||||
skipPrecheck?: boolean;
|
||||
getPrecheckMessage?: (
|
||||
failure: Exclude<RxResumeCredentialPrecheckFailure, null>,
|
||||
) => string;
|
||||
getValidationErrorMessage?: (error: unknown, mode: RxResumeMode) => string;
|
||||
getPersistErrorMessage?: (error: unknown, mode: RxResumeMode) => string;
|
||||
};
|
||||
|
||||
export type ValidateAndMaybePersistRxResumeModeResult<TSettings> = {
|
||||
validation: ValidationResult;
|
||||
precheckFailure: RxResumeCredentialPrecheckFailure;
|
||||
updatedSettings: TSettings | null;
|
||||
};
|
||||
|
||||
export const validateAndMaybePersistRxResumeMode = async <TSettings>(
|
||||
input: ValidateAndMaybePersistRxResumeModeInput<TSettings>,
|
||||
): Promise<ValidateAndMaybePersistRxResumeModeResult<TSettings>> => {
|
||||
const {
|
||||
mode,
|
||||
stored,
|
||||
draft,
|
||||
validate,
|
||||
persist,
|
||||
persistOnSuccess = false,
|
||||
skipPrecheck = false,
|
||||
getPrecheckMessage = (failure) => RXRESUME_PRECHECK_MESSAGES[failure],
|
||||
getValidationErrorMessage = (error) =>
|
||||
error instanceof Error ? error.message : "RxResume validation failed",
|
||||
getPersistErrorMessage = (error) =>
|
||||
error instanceof Error
|
||||
? error.message
|
||||
: "Failed to save RxResume settings",
|
||||
} = input;
|
||||
|
||||
const precheckFailure = skipPrecheck
|
||||
? null
|
||||
: getRxResumeCredentialPrecheckFailure({
|
||||
mode,
|
||||
stored,
|
||||
draft,
|
||||
});
|
||||
if (precheckFailure !== null) {
|
||||
return {
|
||||
validation: {
|
||||
valid: false,
|
||||
message: getPrecheckMessage(precheckFailure),
|
||||
status: 400,
|
||||
},
|
||||
precheckFailure,
|
||||
updatedSettings: null,
|
||||
};
|
||||
}
|
||||
|
||||
let validation: ValidationResult;
|
||||
try {
|
||||
validation = await validate({
|
||||
mode,
|
||||
...toRxResumeValidationPayload(draft),
|
||||
});
|
||||
} catch (error) {
|
||||
return {
|
||||
validation: {
|
||||
valid: false,
|
||||
message: getValidationErrorMessage(error, mode),
|
||||
status: 0,
|
||||
},
|
||||
precheckFailure: null,
|
||||
updatedSettings: null,
|
||||
};
|
||||
}
|
||||
|
||||
if (!validation.valid || !persistOnSuccess || !persist) {
|
||||
return {
|
||||
validation: {
|
||||
valid: validation.valid,
|
||||
message: validation.valid ? null : (validation.message ?? null),
|
||||
status: validation.valid ? null : (validation.status ?? null),
|
||||
},
|
||||
precheckFailure: null,
|
||||
updatedSettings: null,
|
||||
};
|
||||
}
|
||||
|
||||
try {
|
||||
const updatedSettings = await persist(
|
||||
buildRxResumeSettingsUpdate(mode, draft),
|
||||
);
|
||||
return {
|
||||
validation: {
|
||||
valid: true,
|
||||
message: null,
|
||||
status: null,
|
||||
},
|
||||
precheckFailure: null,
|
||||
updatedSettings,
|
||||
};
|
||||
} catch (error) {
|
||||
return {
|
||||
validation: {
|
||||
valid: false,
|
||||
message: getPersistErrorMessage(error, mode),
|
||||
status: 0,
|
||||
},
|
||||
precheckFailure: null,
|
||||
updatedSettings: null,
|
||||
};
|
||||
}
|
||||
};
|
||||
@@ -39,6 +39,7 @@ const makeJob = (overrides: Partial<JobListItem>): JobListItem => ({
|
||||
closedAt: null,
|
||||
suitabilityScore: null,
|
||||
sponsorMatchScore: null,
|
||||
sponsorshipSignals: null,
|
||||
jobType: null,
|
||||
jobFunction: null,
|
||||
salaryMinAmount: null,
|
||||
|
||||
@@ -55,7 +55,6 @@ import {
|
||||
} from "@/lib/utils";
|
||||
import * as api from "../api";
|
||||
import { ConfirmDelete } from "../components/ConfirmDelete";
|
||||
import { GhostwriterDrawer } from "../components/ghostwriter/GhostwriterDrawer";
|
||||
import { JobDetailsEditDrawer } from "../components/JobDetailsEditDrawer";
|
||||
import { JobHeader } from "../components/JobHeader";
|
||||
import {
|
||||
@@ -587,7 +586,6 @@ export const JobPage: React.FC = () => {
|
||||
<CalendarClock className="h-4 w-4" />
|
||||
Application details
|
||||
</CardTitle>
|
||||
<GhostwriterDrawer job={job} />
|
||||
</div>
|
||||
</CardHeader>
|
||||
<CardContent className="space-y-4">
|
||||
|
||||
@@ -60,6 +60,7 @@ let mockAutomaticRunValues: AutomaticRunValues = {
|
||||
topN: 12,
|
||||
minSuitabilityScore: 55,
|
||||
searchTerms: ["backend"],
|
||||
activeKeywordSetId: null,
|
||||
runBudget: 150,
|
||||
country: "united kingdom",
|
||||
cityLocations: [],
|
||||
@@ -407,6 +408,7 @@ describe("OrchestratorPage", () => {
|
||||
topN: 12,
|
||||
minSuitabilityScore: 55,
|
||||
searchTerms: ["backend"],
|
||||
activeKeywordSetId: null,
|
||||
runBudget: 150,
|
||||
country: "united kingdom",
|
||||
cityLocations: [],
|
||||
@@ -759,17 +761,21 @@ describe("OrchestratorPage", () => {
|
||||
fireEvent.click(screen.getByTestId("run-automatic"));
|
||||
|
||||
await waitFor(() => {
|
||||
expect(api.updateSettings).toHaveBeenCalledWith({
|
||||
searchTerms: ["backend"],
|
||||
workplaceTypes: ["remote", "hybrid", "onsite"],
|
||||
jobspyResultsWanted: 150,
|
||||
gradcrackerMaxJobsPerTerm: 150,
|
||||
ukvisajobsMaxJobs: 150,
|
||||
adzunaMaxJobsPerTerm: 150,
|
||||
startupjobsMaxJobsPerTerm: 150,
|
||||
jobspyCountryIndeed: "united kingdom",
|
||||
searchCities: "United Kingdom",
|
||||
});
|
||||
expect(api.updateSettings).toHaveBeenCalledWith(
|
||||
expect.objectContaining({
|
||||
searchTerms: ["backend"],
|
||||
workplaceTypes: ["remote", "hybrid", "onsite"],
|
||||
jobspyResultsWanted: 150,
|
||||
gradcrackerMaxJobsPerTerm: 150,
|
||||
ukvisajobsMaxJobs: 150,
|
||||
adzunaMaxJobsPerTerm: 150,
|
||||
startupjobsMaxJobsPerTerm: 150,
|
||||
workingnomadsMaxJobsPerTerm: 150,
|
||||
testdevjobsMaxPages: 10,
|
||||
jobspyCountryIndeed: "united kingdom",
|
||||
searchCities: "United Kingdom",
|
||||
}),
|
||||
);
|
||||
});
|
||||
expect(api.runPipeline).toHaveBeenCalledWith({
|
||||
topN: 12,
|
||||
@@ -790,6 +796,7 @@ describe("OrchestratorPage", () => {
|
||||
topN: 12,
|
||||
minSuitabilityScore: 55,
|
||||
searchTerms: ["backend"],
|
||||
activeKeywordSetId: null,
|
||||
runBudget: 150,
|
||||
country: "united kingdom",
|
||||
cityLocations: ["London", "Manchester"],
|
||||
@@ -825,6 +832,7 @@ describe("OrchestratorPage", () => {
|
||||
topN: 12,
|
||||
minSuitabilityScore: 55,
|
||||
searchTerms: ["backend"],
|
||||
activeKeywordSetId: null,
|
||||
runBudget: 150,
|
||||
country: "united kingdom",
|
||||
cityLocations: ["Leeds", "Manchester"],
|
||||
@@ -860,6 +868,7 @@ describe("OrchestratorPage", () => {
|
||||
topN: 12,
|
||||
minSuitabilityScore: 55,
|
||||
searchTerms: ["backend"],
|
||||
activeKeywordSetId: null,
|
||||
runBudget: 150,
|
||||
country: "united kingdom",
|
||||
cityLocations: ["Leeds", "Manchester"],
|
||||
@@ -967,6 +976,7 @@ describe("OrchestratorPage", () => {
|
||||
topN: 12,
|
||||
minSuitabilityScore: 55,
|
||||
searchTerms: ["backend"],
|
||||
activeKeywordSetId: null,
|
||||
runBudget: 150,
|
||||
country: "united states",
|
||||
cityLocations: [],
|
||||
|
||||
@@ -1,6 +1,5 @@
|
||||
import { useSettings } from "@client/hooks/useSettings";
|
||||
import { buildDuplicateDismissHints } from "@client/lib/job-dedup";
|
||||
import { inferCountryKeyFromSearchGeography } from "@shared/search-cities";
|
||||
import type React from "react";
|
||||
import { useCallback, useEffect, useMemo, useState } from "react";
|
||||
import { useNavigate, useParams } from "react-router-dom";
|
||||
@@ -47,6 +46,10 @@ export const OrchestratorPage: React.FC = () => {
|
||||
setCountrySelection,
|
||||
sponsorFilter,
|
||||
setSponsorFilter,
|
||||
sponsorshipSignalsFilter,
|
||||
setSponsorshipSignalsFilter,
|
||||
hideSponsorBlockers,
|
||||
setHideSponsorBlockers,
|
||||
workplaceFilter,
|
||||
setWorkplaceFilter,
|
||||
salaryFilter,
|
||||
@@ -59,6 +62,8 @@ export const OrchestratorPage: React.FC = () => {
|
||||
setEmployerFilterTokens,
|
||||
applySettingsCompanySkipList,
|
||||
setApplySettingsCompanySkipList,
|
||||
hidePriorSkips,
|
||||
setHidePriorSkips,
|
||||
sort,
|
||||
setSort,
|
||||
resetFilters,
|
||||
@@ -69,7 +74,7 @@ export const OrchestratorPage: React.FC = () => {
|
||||
if (tab && validTabs.includes(tab as FilterTab)) {
|
||||
return tab as FilterTab;
|
||||
}
|
||||
return "ready";
|
||||
return "discovered";
|
||||
}, [tab]);
|
||||
|
||||
// Helper to change URL while preserving search params
|
||||
@@ -90,7 +95,7 @@ export const OrchestratorPage: React.FC = () => {
|
||||
// Effect to sync URL if it was invalid
|
||||
useEffect(() => {
|
||||
if (tab === "in_progress") {
|
||||
navigate("/applications/in-progress", { replace: true });
|
||||
navigate("/jobs/discovered", { replace: true });
|
||||
return;
|
||||
}
|
||||
const validTabs: FilterTab[] = ["ready", "discovered", "applied", "all"];
|
||||
@@ -162,15 +167,9 @@ export const OrchestratorPage: React.FC = () => {
|
||||
[settings],
|
||||
);
|
||||
|
||||
const searchGeographyCountryKey = useMemo(
|
||||
() =>
|
||||
inferCountryKeyFromSearchGeography(settings?.searchCities?.value ?? null),
|
||||
[settings?.searchCities?.value],
|
||||
);
|
||||
|
||||
const duplicateDismissHints = useMemo(
|
||||
() => buildDuplicateDismissHints(jobs),
|
||||
[jobs],
|
||||
() => (hidePriorSkips ? buildDuplicateDismissHints(jobs) : undefined),
|
||||
[hidePriorSkips, jobs],
|
||||
);
|
||||
|
||||
const jobListFilterExtras = useMemo(
|
||||
@@ -182,7 +181,7 @@ export const OrchestratorPage: React.FC = () => {
|
||||
settingsBlockedEmployerKeywords: applySettingsCompanySkipList
|
||||
? settingsSkipEmployerKeywords
|
||||
: [],
|
||||
searchGeographyCountryKey,
|
||||
searchGeographyCountryKey: null,
|
||||
duplicateDismissHints,
|
||||
}),
|
||||
[
|
||||
@@ -192,7 +191,6 @@ export const OrchestratorPage: React.FC = () => {
|
||||
employerExcludeFilter,
|
||||
applySettingsCompanySkipList,
|
||||
settingsSkipEmployerKeywords,
|
||||
searchGeographyCountryKey,
|
||||
duplicateDismissHints,
|
||||
],
|
||||
);
|
||||
@@ -205,6 +203,8 @@ export const OrchestratorPage: React.FC = () => {
|
||||
countriesFilter,
|
||||
countriesExcludeFilter,
|
||||
sponsorFilter,
|
||||
sponsorshipSignalsFilter,
|
||||
hideSponsorBlockers,
|
||||
workplaceFilter,
|
||||
salaryFilter,
|
||||
sort,
|
||||
@@ -387,6 +387,8 @@ export const OrchestratorPage: React.FC = () => {
|
||||
"countries",
|
||||
"countriesExclude",
|
||||
"sponsor",
|
||||
"sponsorSignal",
|
||||
"hideSponsorBlockers",
|
||||
"salaryMode",
|
||||
"salaryMin",
|
||||
"salaryMax",
|
||||
@@ -523,8 +525,14 @@ export const OrchestratorPage: React.FC = () => {
|
||||
onApplySettingsCompanySkipListChange={
|
||||
setApplySettingsCompanySkipList
|
||||
}
|
||||
hidePriorSkips={hidePriorSkips}
|
||||
onHidePriorSkipsChange={setHidePriorSkips}
|
||||
sponsorFilter={sponsorFilter}
|
||||
onSponsorFilterChange={setSponsorFilter}
|
||||
sponsorshipSignalsFilter={sponsorshipSignalsFilter}
|
||||
onSponsorshipSignalsFilterChange={setSponsorshipSignalsFilter}
|
||||
hideSponsorBlockers={hideSponsorBlockers}
|
||||
onHideSponsorBlockersChange={setHideSponsorBlockers}
|
||||
workplaceFilter={workplaceFilter}
|
||||
onWorkplaceFilterChange={setWorkplaceFilter}
|
||||
salaryFilter={salaryFilter}
|
||||
|
||||
@@ -2,9 +2,8 @@ import { createAppSettings } from "@shared/testing/factories.js";
|
||||
import { fireEvent, screen, waitFor } from "@testing-library/react";
|
||||
import { MemoryRouter } from "react-router-dom";
|
||||
import { toast } from "sonner";
|
||||
import { beforeEach, describe, expect, it, vi } from "vitest";
|
||||
import { afterAll, beforeEach, describe, expect, it, vi } from "vitest";
|
||||
import * as api from "../api";
|
||||
import { _resetTracerReadinessCache } from "../hooks/useTracerReadiness";
|
||||
import { renderWithQueryClient } from "../test/renderWithQueryClient";
|
||||
import { SettingsPage } from "./SettingsPage";
|
||||
|
||||
@@ -17,14 +16,8 @@ vi.mock("../api", () => ({
|
||||
getSettings: vi.fn(),
|
||||
getLlmModels: vi.fn().mockResolvedValue([]),
|
||||
updateSettings: vi.fn(),
|
||||
validateRxresume: vi.fn(),
|
||||
getRxResumeProjects: vi.fn(),
|
||||
clearDatabase: vi.fn(),
|
||||
deleteJobsByStatus: vi.fn(),
|
||||
getTracerReadiness: vi.fn(),
|
||||
getBackups: vi.fn().mockResolvedValue({ backups: [], nextScheduled: null }),
|
||||
createManualBackup: vi.fn(),
|
||||
deleteBackup: vi.fn(),
|
||||
}));
|
||||
|
||||
vi.mock("sonner", () => ({
|
||||
@@ -67,20 +60,6 @@ const openModelSection = async () => {
|
||||
fireEvent.click(modelTrigger);
|
||||
};
|
||||
|
||||
const openWritingStyleSection = async () => {
|
||||
const chatTrigger = await screen.findByRole("button", {
|
||||
name: /writing style & language/i,
|
||||
});
|
||||
fireEvent.click(chatTrigger);
|
||||
};
|
||||
|
||||
const openReactiveResumeSection = async () => {
|
||||
const trigger = await screen.findByRole("button", {
|
||||
name: /reactive resume/i,
|
||||
});
|
||||
fireEvent.click(trigger);
|
||||
};
|
||||
|
||||
describe("SettingsPage", () => {
|
||||
beforeEach(() => {
|
||||
vi.clearAllMocks();
|
||||
@@ -88,21 +67,6 @@ describe("SettingsPage", () => {
|
||||
configurable: true,
|
||||
value: vi.fn(),
|
||||
});
|
||||
_resetTracerReadinessCache();
|
||||
vi.mocked(api.getTracerReadiness).mockResolvedValue({
|
||||
status: "ready",
|
||||
canEnable: true,
|
||||
publicBaseUrl: "https://my-jobops.example.com",
|
||||
healthUrl: "https://my-jobops.example.com/health",
|
||||
checkedAt: Date.now(),
|
||||
lastSuccessAt: Date.now(),
|
||||
reason: null,
|
||||
});
|
||||
vi.mocked(api.validateRxresume).mockResolvedValue({
|
||||
valid: false,
|
||||
message: "Missing credentials",
|
||||
status: 400,
|
||||
});
|
||||
});
|
||||
|
||||
afterAll(() => {
|
||||
@@ -262,7 +226,7 @@ describe("SettingsPage", () => {
|
||||
);
|
||||
});
|
||||
|
||||
it("hides pipeline tuning sections that moved to run modal", async () => {
|
||||
it("hides pipeline tuning and unused settings sections", async () => {
|
||||
vi.mocked(api.getSettings).mockResolvedValue(baseSettings);
|
||||
renderPage();
|
||||
|
||||
@@ -279,243 +243,21 @@ describe("SettingsPage", () => {
|
||||
expect(
|
||||
screen.queryByRole("button", { name: /jobspy scraper/i }),
|
||||
).not.toBeInTheDocument();
|
||||
});
|
||||
|
||||
it("enables save button when display setting is changed", async () => {
|
||||
vi.mocked(api.getSettings).mockResolvedValue(baseSettings);
|
||||
renderPage();
|
||||
const saveButton = screen.getByRole("button", { name: /^save$/i });
|
||||
|
||||
const displayTrigger = await screen.findByRole("button", {
|
||||
name: /display settings/i,
|
||||
});
|
||||
fireEvent.click(displayTrigger);
|
||||
const sponsorCheckbox = screen.getByLabelText(
|
||||
/show visa sponsor information/i,
|
||||
);
|
||||
fireEvent.click(sponsorCheckbox);
|
||||
await waitFor(() => expect(saveButton).toBeEnabled());
|
||||
});
|
||||
|
||||
it("allows saving when both Reactive Resume v4 and v5 credentials are present", async () => {
|
||||
const settingsWithBothRxResumeAuth = createAppSettings({
|
||||
rxresumeEmail: "resume@example.com",
|
||||
rxresumePasswordHint: "pass",
|
||||
rxresumeApiKeyHint: "api_",
|
||||
});
|
||||
vi.mocked(api.getSettings).mockResolvedValue(settingsWithBothRxResumeAuth);
|
||||
vi.mocked(api.updateSettings).mockResolvedValue(
|
||||
settingsWithBothRxResumeAuth,
|
||||
);
|
||||
|
||||
renderPage();
|
||||
|
||||
const displayTrigger = await screen.findByRole("button", {
|
||||
name: /display settings/i,
|
||||
});
|
||||
fireEvent.click(displayTrigger);
|
||||
const sponsorCheckbox = screen.getByLabelText(
|
||||
/show visa sponsor information/i,
|
||||
);
|
||||
fireEvent.click(sponsorCheckbox);
|
||||
|
||||
const saveButton = screen.getByRole("button", { name: /^save$/i });
|
||||
await waitFor(() => expect(saveButton).toBeEnabled());
|
||||
fireEvent.click(saveButton);
|
||||
|
||||
await waitFor(() => expect(api.updateSettings).toHaveBeenCalled());
|
||||
expect(toast.error).not.toHaveBeenCalledWith(
|
||||
"Choose one Reactive Resume auth method",
|
||||
expect.anything(),
|
||||
);
|
||||
});
|
||||
|
||||
it("saves a shared RxResume URL from the Reactive Resume section", async () => {
|
||||
vi.mocked(api.getSettings).mockResolvedValue(baseSettings);
|
||||
vi.mocked(api.updateSettings).mockResolvedValue({
|
||||
...baseSettings,
|
||||
rxresumeUrl: "https://resume.example.com",
|
||||
});
|
||||
|
||||
renderPage();
|
||||
|
||||
const reactiveResumeTrigger = await screen.findByRole("button", {
|
||||
name: /reactive resume/i,
|
||||
});
|
||||
fireEvent.click(reactiveResumeTrigger);
|
||||
|
||||
const urlInput = screen.getByLabelText(/rxresume url/i);
|
||||
await waitFor(() => expect(urlInput).toBeEnabled());
|
||||
fireEvent.change(urlInput, {
|
||||
target: { value: "https://resume.example.com" },
|
||||
});
|
||||
|
||||
const saveButton = screen.getByRole("button", { name: /^save$/i });
|
||||
await waitFor(() => expect(saveButton).toBeEnabled());
|
||||
fireEvent.click(saveButton);
|
||||
|
||||
await waitFor(() => expect(api.updateSettings).toHaveBeenCalled());
|
||||
expect(api.updateSettings).toHaveBeenCalledWith(
|
||||
expect.objectContaining({
|
||||
rxresumeUrl: "https://resume.example.com",
|
||||
}),
|
||||
);
|
||||
});
|
||||
|
||||
it("blocks save and renders an inline alert when the v5 API key is invalid", async () => {
|
||||
vi.mocked(api.getSettings).mockResolvedValue(baseSettings);
|
||||
|
||||
renderPage();
|
||||
await openReactiveResumeSection();
|
||||
|
||||
await waitFor(() => expect(api.validateRxresume).toHaveBeenCalled());
|
||||
vi.mocked(api.validateRxresume).mockClear();
|
||||
vi.mocked(api.validateRxresume).mockResolvedValue({
|
||||
valid: false,
|
||||
message:
|
||||
"Reactive Resume v5 API key is invalid. Update the API key and try again.",
|
||||
status: 401,
|
||||
});
|
||||
|
||||
fireEvent.change(screen.getByLabelText(/v5 api key/i), {
|
||||
target: { value: "invalid-v5-key" },
|
||||
});
|
||||
|
||||
const saveButton = screen.getByRole("button", { name: /^save$/i });
|
||||
await waitFor(() => expect(saveButton).toBeEnabled());
|
||||
fireEvent.click(saveButton);
|
||||
|
||||
expect(
|
||||
await screen.findByText(/Reactive Resume v5 API key is invalid/i),
|
||||
).toBeInTheDocument();
|
||||
expect(api.updateSettings).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("allows saving on RxResume availability warnings and keeps the inline warning visible", async () => {
|
||||
vi.mocked(api.getSettings).mockResolvedValue(baseSettings);
|
||||
vi.mocked(api.updateSettings).mockResolvedValue({
|
||||
...baseSettings,
|
||||
rxresumeApiKeyHint: "rr-v",
|
||||
});
|
||||
|
||||
renderPage();
|
||||
await openReactiveResumeSection();
|
||||
|
||||
await waitFor(() => expect(api.validateRxresume).toHaveBeenCalled());
|
||||
vi.mocked(api.validateRxresume).mockClear();
|
||||
vi.mocked(api.validateRxresume).mockResolvedValue({
|
||||
valid: false,
|
||||
message:
|
||||
"JobOps could not verify Reactive Resume because the instance is unavailable right now.",
|
||||
status: 0,
|
||||
});
|
||||
|
||||
fireEvent.change(screen.getByLabelText(/v5 api key/i), {
|
||||
target: { value: "rr-v5-warning-key" },
|
||||
});
|
||||
|
||||
const saveButton = screen.getByRole("button", { name: /^save$/i });
|
||||
await waitFor(() => expect(saveButton).toBeEnabled());
|
||||
fireEvent.click(saveButton);
|
||||
|
||||
await waitFor(() => expect(api.updateSettings).toHaveBeenCalled());
|
||||
expect(
|
||||
await screen.findByText(/instance is unavailable right now/i),
|
||||
).toBeInTheDocument();
|
||||
expect(toast.success).toHaveBeenCalledWith("Settings saved");
|
||||
expect(toast.info).toHaveBeenCalledWith(
|
||||
"Settings saved, but JobOps could not verify Reactive Resume because the instance is unavailable.",
|
||||
);
|
||||
});
|
||||
|
||||
it("does not run RxResume validation for unrelated settings saves", async () => {
|
||||
vi.mocked(api.getSettings).mockResolvedValue(baseSettings);
|
||||
vi.mocked(api.updateSettings).mockResolvedValue({
|
||||
...baseSettings,
|
||||
model: {
|
||||
value: "new-model",
|
||||
default: baseSettings.model.default,
|
||||
override: "new-model",
|
||||
},
|
||||
});
|
||||
|
||||
renderPage();
|
||||
await openModelSection();
|
||||
await waitFor(() => expect(api.validateRxresume).toHaveBeenCalled());
|
||||
vi.mocked(api.validateRxresume).mockClear();
|
||||
|
||||
fireEvent.change(screen.getByLabelText(/default model/i), {
|
||||
target: { value: "new-model" },
|
||||
});
|
||||
|
||||
const saveButton = screen.getByRole("button", { name: /^save$/i });
|
||||
await waitFor(() => expect(saveButton).toBeEnabled());
|
||||
fireEvent.click(saveButton);
|
||||
|
||||
await waitFor(() => expect(api.updateSettings).toHaveBeenCalled());
|
||||
expect(api.validateRxresume).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("clears the previous RxResume warning when the key or URL changes", async () => {
|
||||
vi.mocked(api.getSettings).mockResolvedValue(baseSettings);
|
||||
vi.mocked(api.validateRxresume).mockResolvedValue({
|
||||
valid: false,
|
||||
message:
|
||||
"JobOps could not verify Reactive Resume because the instance is unavailable right now.",
|
||||
status: 0,
|
||||
});
|
||||
|
||||
renderPage();
|
||||
await openReactiveResumeSection();
|
||||
|
||||
expect(
|
||||
await screen.findByText(/instance is unavailable right now/i),
|
||||
).toBeInTheDocument();
|
||||
|
||||
fireEvent.change(screen.getByLabelText(/rxresume url/i), {
|
||||
target: { value: "https://resume.example.com" },
|
||||
});
|
||||
|
||||
await waitFor(() =>
|
||||
expect(
|
||||
screen.queryByText(/instance is unavailable right now/i),
|
||||
).not.toBeInTheDocument(),
|
||||
);
|
||||
});
|
||||
|
||||
it("saves the writing language mode through the settings page", async () => {
|
||||
vi.mocked(api.getSettings).mockResolvedValue(baseSettings);
|
||||
vi.mocked(api.updateSettings).mockResolvedValue(
|
||||
createAppSettings({
|
||||
chatStyleLanguageMode: {
|
||||
value: "match-resume",
|
||||
default: "manual",
|
||||
override: "match-resume",
|
||||
},
|
||||
}),
|
||||
);
|
||||
|
||||
renderPage();
|
||||
await openWritingStyleSection();
|
||||
|
||||
fireEvent.click(screen.getByRole("combobox", { name: /output language/i }));
|
||||
fireEvent.click(await screen.findByText("Match current resume language"));
|
||||
|
||||
expect(
|
||||
screen.queryByRole("combobox", { name: /specific language/i }),
|
||||
screen.queryByRole("button", { name: /display settings/i }),
|
||||
).not.toBeInTheDocument();
|
||||
expect(
|
||||
screen.queryByRole("button", { name: /writing style & language/i }),
|
||||
).not.toBeInTheDocument();
|
||||
expect(
|
||||
screen.queryByRole("button", { name: /backup/i }),
|
||||
).not.toBeInTheDocument();
|
||||
expect(
|
||||
screen.queryByRole("button", { name: /tracer/i }),
|
||||
).not.toBeInTheDocument();
|
||||
expect(
|
||||
screen.queryByRole("button", { name: /resume projects/i }),
|
||||
).not.toBeInTheDocument();
|
||||
|
||||
const saveButton = screen.getByRole("button", { name: /^save$/i });
|
||||
await waitFor(() => expect(saveButton).toBeEnabled());
|
||||
fireEvent.click(saveButton);
|
||||
|
||||
await waitFor(() => expect(api.updateSettings).toHaveBeenCalled());
|
||||
expect(api.updateSettings).toHaveBeenCalledWith(
|
||||
expect.objectContaining({
|
||||
chatStyleLanguageMode: "match-resume",
|
||||
chatStyleManualLanguage: null,
|
||||
}),
|
||||
);
|
||||
});
|
||||
|
||||
it("enables save button when basic auth toggle is changed", async () => {
|
||||
|
||||
@@ -1,29 +1,13 @@
|
||||
import * as api from "@client/api";
|
||||
import { PageHeader } from "@client/components/layout";
|
||||
import { useUpdateSettingsMutation } from "@client/hooks/queries/useSettingsMutation";
|
||||
import { useRxResumeConfigState } from "@client/hooks/useRxResumeConfigState";
|
||||
import { useTracerReadiness } from "@client/hooks/useTracerReadiness";
|
||||
import {
|
||||
coerceRxResumeMode,
|
||||
getRxResumeCredentialDrafts,
|
||||
getRxResumeCredentialPrecheckFailure,
|
||||
isRxResumeAvailabilityValidationFailure,
|
||||
isRxResumeBlockingValidationFailure,
|
||||
RXRESUME_MODES,
|
||||
RXRESUME_PRECHECK_MESSAGES,
|
||||
toRxResumeValidationPayload,
|
||||
validateAndMaybePersistRxResumeMode,
|
||||
} from "@client/lib/rxresume-config";
|
||||
import { BackupSettingsSection } from "@client/pages/settings/components/BackupSettingsSection";
|
||||
import { ChatSettingsSection } from "@client/pages/settings/components/ChatSettingsSection";
|
||||
import { DangerZoneSection } from "@client/pages/settings/components/DangerZoneSection";
|
||||
import { DisplaySettingsSection } from "@client/pages/settings/components/DisplaySettingsSection";
|
||||
import { EnvironmentSettingsSection } from "@client/pages/settings/components/EnvironmentSettingsSection";
|
||||
import { JobSearchProfileSection } from "@client/pages/settings/components/JobSearchProfileSection";
|
||||
import { JobSourcesSettingsSection } from "@client/pages/settings/components/JobSourcesSettingsSection";
|
||||
import { KeywordSetsSettingsSection } from "@client/pages/settings/components/KeywordSetsSettingsSection";
|
||||
import { ModelSettingsSection } from "@client/pages/settings/components/ModelSettingsSection";
|
||||
import { ReactiveResumeSection } from "@client/pages/settings/components/ReactiveResumeSection";
|
||||
import { ScoringSettingsSection } from "@client/pages/settings/components/ScoringSettingsSection";
|
||||
import { TracerLinksSettingsSection } from "@client/pages/settings/components/TracerLinksSettingsSection";
|
||||
import { WebhooksSection } from "@client/pages/settings/components/WebhooksSection";
|
||||
import {
|
||||
type LlmProviderId,
|
||||
@@ -37,24 +21,12 @@ import {
|
||||
type UpdateSettingsInput,
|
||||
updateSettingsSchema,
|
||||
} from "@shared/settings-schema.js";
|
||||
import type {
|
||||
AppSettings,
|
||||
JobStatus,
|
||||
ResumeProjectCatalogItem,
|
||||
ResumeProjectsSettings,
|
||||
RxResumeMode,
|
||||
ValidationResult,
|
||||
} from "@shared/types.js";
|
||||
import { useQuery, useQueryClient } from "@tanstack/react-query";
|
||||
import type { AppSettings, JobStatus } from "@shared/types.js";
|
||||
import { useQuery } from "@tanstack/react-query";
|
||||
import { Settings } from "lucide-react";
|
||||
import type React from "react";
|
||||
import { useCallback, useEffect, useState } from "react";
|
||||
import {
|
||||
FormProvider,
|
||||
type Resolver,
|
||||
useForm,
|
||||
useWatch,
|
||||
} from "react-hook-form";
|
||||
import { useEffect, useState } from "react";
|
||||
import { FormProvider, type Resolver, useForm } from "react-hook-form";
|
||||
import { toast } from "sonner";
|
||||
import { useQueryErrorToast } from "@/client/hooks/useQueryErrorToast";
|
||||
import { queryKeys } from "@/client/lib/queryKeys";
|
||||
@@ -105,37 +77,20 @@ const DEFAULT_FORM_VALUES: UpdateSettingsInput = {
|
||||
blockedCompanyKeywords: [],
|
||||
blockedCountries: [],
|
||||
scoringInstructions: "",
|
||||
ashbyCompanies: [],
|
||||
greenhouseCompanies: [],
|
||||
leverCompanies: [],
|
||||
smartrecruitersCompanies: [],
|
||||
teamtailorCompanies: [],
|
||||
huntflowTenants: [],
|
||||
factorialTenants: [],
|
||||
workdayTenants: [],
|
||||
careersPageUrls: [],
|
||||
icimsTenants: [],
|
||||
elutaRssLocations: [],
|
||||
};
|
||||
|
||||
type LlmProviderValue = LlmProviderId | null;
|
||||
type RxResumeValidationBadgeState = {
|
||||
checked: boolean;
|
||||
valid: boolean;
|
||||
message: string | null;
|
||||
status: number | null;
|
||||
};
|
||||
const EMPTY_RXRESUME_VALIDATION_BADGE_STATE: RxResumeValidationBadgeState = {
|
||||
checked: false,
|
||||
valid: false,
|
||||
message: null,
|
||||
status: null,
|
||||
};
|
||||
|
||||
const getRxResumeValidationFieldsForMode = (
|
||||
mode: RxResumeMode,
|
||||
): Array<keyof UpdateSettingsInput> =>
|
||||
mode === "v5"
|
||||
? ["rxresumeApiKey", "rxresumeUrl"]
|
||||
: ["rxresumeEmail", "rxresumePassword", "rxresumeUrl"];
|
||||
|
||||
const toRxResumeValidationBadgeState = (
|
||||
validation: ValidationResult,
|
||||
): RxResumeValidationBadgeState => ({
|
||||
checked: true,
|
||||
valid: validation.valid,
|
||||
message: validation.valid ? null : (validation.message ?? null),
|
||||
status: validation.valid ? null : (validation.status ?? null),
|
||||
});
|
||||
|
||||
const normalizeLlmProviderValue = (
|
||||
value: string | null | undefined,
|
||||
@@ -233,6 +188,17 @@ const mapSettingsToForm = (data: AppSettings): UpdateSettingsInput => ({
|
||||
blockedCompanyKeywords: data.blockedCompanyKeywords.override ?? [],
|
||||
blockedCountries: data.blockedCountries.override ?? [],
|
||||
scoringInstructions: data.scoringInstructions.override ?? "",
|
||||
ashbyCompanies: data.ashbyCompanies.value,
|
||||
greenhouseCompanies: data.greenhouseCompanies.value,
|
||||
leverCompanies: data.leverCompanies.value,
|
||||
smartrecruitersCompanies: data.smartrecruitersCompanies.value,
|
||||
teamtailorCompanies: data.teamtailorCompanies.value,
|
||||
huntflowTenants: data.huntflowTenants.value,
|
||||
factorialTenants: data.factorialTenants.value,
|
||||
workdayTenants: data.workdayTenants.value,
|
||||
careersPageUrls: data.careersPageUrls.value,
|
||||
icimsTenants: data.icimsTenants.value,
|
||||
elutaRssLocations: data.elutaRssLocations.value,
|
||||
});
|
||||
|
||||
const normalizeString = (value: string | null | undefined) => {
|
||||
@@ -254,43 +220,7 @@ const stringArraysEqual = (left: string[], right: string[]): boolean => {
|
||||
const nullIfSame = <T,>(value: T | null | undefined, defaultValue: T) =>
|
||||
value === defaultValue ? null : (value ?? null);
|
||||
|
||||
const normalizeResumeProjectsForCatalog = (
|
||||
catalog: ResumeProjectCatalogItem[],
|
||||
current: ResumeProjectsSettings | null,
|
||||
): ResumeProjectsSettings | null => {
|
||||
const allowed = new Set(catalog.map((project) => project.id));
|
||||
|
||||
const base = current ?? {
|
||||
maxProjects: 0,
|
||||
lockedProjectIds: catalog
|
||||
.filter((project) => project.isVisibleInBase)
|
||||
.map((project) => project.id),
|
||||
aiSelectableProjectIds: [],
|
||||
};
|
||||
|
||||
const lockedProjectIds = base.lockedProjectIds.filter((id) =>
|
||||
allowed.has(id),
|
||||
);
|
||||
const lockedSet = new Set(lockedProjectIds);
|
||||
const aiSelectableProjectIds = (
|
||||
current ? base.aiSelectableProjectIds : catalog.map((project) => project.id)
|
||||
)
|
||||
.filter((id) => allowed.has(id))
|
||||
.filter((id) => !lockedSet.has(id));
|
||||
const maxProjectsRaw = Number.isFinite(base.maxProjects)
|
||||
? base.maxProjects
|
||||
: 0;
|
||||
const maxProjectsInt = Math.max(0, Math.floor(maxProjectsRaw));
|
||||
const maxProjects = Math.min(
|
||||
catalog.length,
|
||||
Math.max(lockedProjectIds.length, maxProjectsInt, 3),
|
||||
);
|
||||
return { maxProjects, lockedProjectIds, aiSelectableProjectIds };
|
||||
};
|
||||
|
||||
const getDerivedSettings = (settings: AppSettings | null) => {
|
||||
const profileProjects = settings?.profileProjects ?? [];
|
||||
|
||||
return {
|
||||
model: {
|
||||
effective: settings?.model?.value ?? "",
|
||||
@@ -320,41 +250,13 @@ const getDerivedSettings = (settings: AppSettings | null) => {
|
||||
default: settings?.renderMarkdownInJobDescriptions?.default ?? true,
|
||||
},
|
||||
},
|
||||
chat: {
|
||||
tone: {
|
||||
effective: settings?.chatStyleTone?.value ?? "professional",
|
||||
default: settings?.chatStyleTone?.default ?? "professional",
|
||||
},
|
||||
formality: {
|
||||
effective: settings?.chatStyleFormality?.value ?? "medium",
|
||||
default: settings?.chatStyleFormality?.default ?? "medium",
|
||||
},
|
||||
constraints: {
|
||||
effective: settings?.chatStyleConstraints?.value ?? "",
|
||||
default: settings?.chatStyleConstraints?.default ?? "",
|
||||
},
|
||||
doNotUse: {
|
||||
effective: settings?.chatStyleDoNotUse?.value ?? "",
|
||||
default: settings?.chatStyleDoNotUse?.default ?? "",
|
||||
},
|
||||
languageMode: {
|
||||
effective: settings?.chatStyleLanguageMode?.value ?? "manual",
|
||||
default: settings?.chatStyleLanguageMode?.default ?? "manual",
|
||||
},
|
||||
manualLanguage: {
|
||||
effective: settings?.chatStyleManualLanguage?.value ?? "english",
|
||||
default: settings?.chatStyleManualLanguage?.default ?? "english",
|
||||
},
|
||||
},
|
||||
envSettings: {
|
||||
readable: {
|
||||
rxresumeEmail: settings?.rxresumeEmail ?? "",
|
||||
ukvisajobsEmail: settings?.ukvisajobsEmail ?? "",
|
||||
adzunaAppId: settings?.adzunaAppId ?? "",
|
||||
basicAuthUser: settings?.basicAuthUser ?? "",
|
||||
},
|
||||
private: {
|
||||
rxresumePasswordHint: settings?.rxresumePasswordHint ?? null,
|
||||
ukvisajobsPasswordHint: settings?.ukvisajobsPasswordHint ?? null,
|
||||
adzunaAppKeyHint: settings?.adzunaAppKeyHint ?? null,
|
||||
basicAuthPasswordHint: settings?.basicAuthPasswordHint ?? null,
|
||||
@@ -363,10 +265,6 @@ const getDerivedSettings = (settings: AppSettings | null) => {
|
||||
basicAuthActive: settings?.basicAuthActive ?? false,
|
||||
},
|
||||
defaultResumeProjects: settings?.resumeProjects?.default ?? null,
|
||||
|
||||
profileProjects,
|
||||
maxProjectsTotal: profileProjects.length,
|
||||
|
||||
backup: {
|
||||
backupEnabled: {
|
||||
effective: settings?.backupEnabled?.value ?? false,
|
||||
@@ -411,41 +309,61 @@ const getDerivedSettings = (settings: AppSettings | null) => {
|
||||
default: settings?.scoringInstructions?.default ?? "",
|
||||
},
|
||||
},
|
||||
jobSources: {
|
||||
ashbyCompanies: {
|
||||
effective: settings?.ashbyCompanies?.value ?? [],
|
||||
default: settings?.ashbyCompanies?.default ?? [],
|
||||
},
|
||||
greenhouseCompanies: {
|
||||
effective: settings?.greenhouseCompanies?.value ?? [],
|
||||
default: settings?.greenhouseCompanies?.default ?? [],
|
||||
},
|
||||
leverCompanies: {
|
||||
effective: settings?.leverCompanies?.value ?? [],
|
||||
default: settings?.leverCompanies?.default ?? [],
|
||||
},
|
||||
smartrecruitersCompanies: {
|
||||
effective: settings?.smartrecruitersCompanies?.value ?? [],
|
||||
default: settings?.smartrecruitersCompanies?.default ?? [],
|
||||
},
|
||||
teamtailorCompanies: {
|
||||
effective: settings?.teamtailorCompanies?.value ?? [],
|
||||
default: settings?.teamtailorCompanies?.default ?? [],
|
||||
},
|
||||
huntflowTenants: {
|
||||
effective: settings?.huntflowTenants?.value ?? [],
|
||||
default: settings?.huntflowTenants?.default ?? [],
|
||||
},
|
||||
factorialTenants: {
|
||||
effective: settings?.factorialTenants?.value ?? [],
|
||||
default: settings?.factorialTenants?.default ?? [],
|
||||
},
|
||||
workdayTenants: {
|
||||
effective: settings?.workdayTenants?.value ?? [],
|
||||
default: settings?.workdayTenants?.default ?? [],
|
||||
},
|
||||
careersPageUrls: {
|
||||
effective: settings?.careersPageUrls?.value ?? [],
|
||||
default: settings?.careersPageUrls?.default ?? [],
|
||||
},
|
||||
icimsTenants: {
|
||||
effective: settings?.icimsTenants?.value ?? [],
|
||||
default: settings?.icimsTenants?.default ?? [],
|
||||
},
|
||||
elutaRssLocations: {
|
||||
effective: settings?.elutaRssLocations?.value ?? [],
|
||||
default: settings?.elutaRssLocations?.default ?? [],
|
||||
},
|
||||
},
|
||||
};
|
||||
};
|
||||
|
||||
export const SettingsPage: React.FC = () => {
|
||||
const queryClient = useQueryClient();
|
||||
const [settings, setSettings] = useState<AppSettings | null>(null);
|
||||
const [isSaving, setIsSaving] = useState(false);
|
||||
const [rxresumeValidationStatuses, setRxresumeValidationStatuses] = useState<{
|
||||
v4: RxResumeValidationBadgeState;
|
||||
v5: RxResumeValidationBadgeState;
|
||||
}>({
|
||||
v4: EMPTY_RXRESUME_VALIDATION_BADGE_STATE,
|
||||
v5: EMPTY_RXRESUME_VALIDATION_BADGE_STATE,
|
||||
});
|
||||
const [statusesToClear, setStatusesToClear] = useState<JobStatus[]>([
|
||||
"discovered",
|
||||
]);
|
||||
const [rxResumeBaseResumeIdDraft, setRxResumeBaseResumeIdDraft] = useState<
|
||||
string | null
|
||||
>(null);
|
||||
const [rxResumeProjectsOverride, setRxResumeProjectsOverride] = useState<
|
||||
ResumeProjectCatalogItem[] | null
|
||||
>(null);
|
||||
const [isFetchingRxResumeProjects, setIsFetchingRxResumeProjects] =
|
||||
useState(false);
|
||||
|
||||
// Backup state
|
||||
const [isCreatingBackup, setIsCreatingBackup] = useState(false);
|
||||
const [isDeletingBackup, setIsDeletingBackup] = useState(false);
|
||||
const {
|
||||
readiness: tracerReadiness,
|
||||
isLoading: isTracerReadinessLoading,
|
||||
isChecking: isTracerReadinessChecking,
|
||||
refreshReadiness,
|
||||
} = useTracerReadiness();
|
||||
|
||||
const methods = useForm<UpdateSettingsInput>({
|
||||
resolver: zodResolver(
|
||||
@@ -456,49 +374,17 @@ export const SettingsPage: React.FC = () => {
|
||||
});
|
||||
|
||||
const {
|
||||
clearErrors,
|
||||
handleSubmit,
|
||||
reset,
|
||||
setError,
|
||||
setValue,
|
||||
getValues,
|
||||
control,
|
||||
formState: { isDirty, errors, isValid, dirtyFields },
|
||||
} = methods;
|
||||
const {
|
||||
storedRxResume,
|
||||
getBaseResumeIdForMode,
|
||||
setBaseResumeIdForMode,
|
||||
syncBaseResumeIdsForMode,
|
||||
} = useRxResumeConfigState(settings);
|
||||
|
||||
const settingsQuery = useQuery({
|
||||
queryKey: queryKeys.settings.current(),
|
||||
queryFn: api.getSettings,
|
||||
});
|
||||
const backupsQuery = useQuery({
|
||||
queryKey: queryKeys.backups.list(),
|
||||
queryFn: api.getBackups,
|
||||
});
|
||||
const updateSettingsMutation = useUpdateSettingsMutation();
|
||||
const isLoading = settingsQuery.isLoading;
|
||||
const backups = backupsQuery.data?.backups ?? [];
|
||||
const nextScheduled = backupsQuery.data?.nextScheduled ?? null;
|
||||
const isLoadingBackups = backupsQuery.isLoading;
|
||||
useQueryErrorToast(backupsQuery.error, "Failed to load backups");
|
||||
|
||||
const rxresumeMode = (settings?.rxresumeMode?.value ?? "v5") as RxResumeMode;
|
||||
const selectedRxresumeMode = (useWatch({
|
||||
control,
|
||||
name: "rxresumeMode",
|
||||
}) ?? rxresumeMode) as RxResumeMode;
|
||||
const resumeProjectsValue = useWatch({
|
||||
control,
|
||||
name: "resumeProjects",
|
||||
});
|
||||
const hasRxResumeAccess = Boolean(
|
||||
rxresumeValidationStatuses[selectedRxresumeMode].valid,
|
||||
);
|
||||
|
||||
useEffect(() => {
|
||||
if (!settingsQuery.data) return;
|
||||
@@ -508,251 +394,20 @@ export const SettingsPage: React.FC = () => {
|
||||
|
||||
useQueryErrorToast(settingsQuery.error, "Failed to load settings");
|
||||
|
||||
useEffect(() => {
|
||||
if (!settings) return;
|
||||
const effectiveMode = coerceRxResumeMode(settings.rxresumeMode?.value);
|
||||
const storedId = syncBaseResumeIdsForMode(effectiveMode);
|
||||
setRxResumeBaseResumeIdDraft(storedId);
|
||||
setValue("rxresumeBaseResumeId", storedId, { shouldDirty: false });
|
||||
setRxResumeProjectsOverride(null);
|
||||
}, [settings, setValue, syncBaseResumeIdsForMode]);
|
||||
|
||||
useEffect(() => {
|
||||
let isMounted = true;
|
||||
const controller = new AbortController();
|
||||
|
||||
if (!rxResumeBaseResumeIdDraft) {
|
||||
setRxResumeProjectsOverride(null);
|
||||
return () => {
|
||||
isMounted = false;
|
||||
controller.abort();
|
||||
};
|
||||
}
|
||||
|
||||
if (!hasRxResumeAccess)
|
||||
return () => {
|
||||
isMounted = false;
|
||||
controller.abort();
|
||||
};
|
||||
|
||||
setIsFetchingRxResumeProjects(true);
|
||||
api
|
||||
.getRxResumeProjects(
|
||||
rxResumeBaseResumeIdDraft,
|
||||
controller.signal,
|
||||
selectedRxresumeMode,
|
||||
)
|
||||
.then((projects) => {
|
||||
if (!isMounted) return;
|
||||
setRxResumeProjectsOverride(projects);
|
||||
const normalized = normalizeResumeProjectsForCatalog(
|
||||
projects,
|
||||
getValues("resumeProjects") ?? null,
|
||||
);
|
||||
if (normalized) {
|
||||
setValue("resumeProjects", normalized, { shouldDirty: true });
|
||||
}
|
||||
})
|
||||
.catch((error) => {
|
||||
if (!isMounted || error.name === "AbortError") return;
|
||||
const message =
|
||||
error instanceof Error
|
||||
? error.message
|
||||
: "Failed to load RxResume projects";
|
||||
toast.error(message);
|
||||
setRxResumeProjectsOverride(null);
|
||||
})
|
||||
.finally(() => {
|
||||
if (!isMounted) return;
|
||||
setIsFetchingRxResumeProjects(false);
|
||||
});
|
||||
|
||||
return () => {
|
||||
isMounted = false;
|
||||
controller.abort();
|
||||
};
|
||||
}, [
|
||||
rxResumeBaseResumeIdDraft,
|
||||
hasRxResumeAccess,
|
||||
selectedRxresumeMode,
|
||||
getValues,
|
||||
setValue,
|
||||
]);
|
||||
|
||||
const derived = getDerivedSettings(settings);
|
||||
const {
|
||||
model,
|
||||
pipelineWebhook,
|
||||
jobCompleteWebhook,
|
||||
display,
|
||||
chat,
|
||||
envSettings,
|
||||
defaultResumeProjects,
|
||||
profileProjects,
|
||||
backup,
|
||||
scoring,
|
||||
jobSearchProfile,
|
||||
jobSources,
|
||||
} = derived;
|
||||
|
||||
const handleCreateBackup = async () => {
|
||||
setIsCreatingBackup(true);
|
||||
try {
|
||||
await api.createManualBackup();
|
||||
toast.success("Backup created successfully");
|
||||
await queryClient.invalidateQueries({ queryKey: queryKeys.backups.all });
|
||||
} catch (error) {
|
||||
const message =
|
||||
error instanceof Error ? error.message : "Failed to create backup";
|
||||
toast.error(message);
|
||||
} finally {
|
||||
setIsCreatingBackup(false);
|
||||
}
|
||||
};
|
||||
|
||||
const handleDeleteBackup = async (filename: string) => {
|
||||
const confirmed = window.confirm(
|
||||
`Delete backup "${filename}"? This action cannot be undone.`,
|
||||
);
|
||||
if (!confirmed) {
|
||||
return;
|
||||
}
|
||||
setIsDeletingBackup(true);
|
||||
try {
|
||||
await api.deleteBackup(filename);
|
||||
toast.success("Backup deleted successfully");
|
||||
await queryClient.invalidateQueries({ queryKey: queryKeys.backups.all });
|
||||
} catch (error) {
|
||||
const message =
|
||||
error instanceof Error ? error.message : "Failed to delete backup";
|
||||
toast.error(message);
|
||||
} finally {
|
||||
setIsDeletingBackup(false);
|
||||
}
|
||||
};
|
||||
|
||||
const handleVerifyTracerReadiness = useCallback(async () => {
|
||||
try {
|
||||
const readiness = await refreshReadiness(true);
|
||||
if (!readiness) {
|
||||
toast.error("Tracer links are unavailable. Verify your public URL.");
|
||||
} else if (readiness.canEnable) {
|
||||
toast.success("Tracer links are ready");
|
||||
} else {
|
||||
toast.error(
|
||||
readiness.reason ??
|
||||
"Tracer links are unavailable. Verify your public URL.",
|
||||
);
|
||||
}
|
||||
} catch (error) {
|
||||
const message =
|
||||
error instanceof Error
|
||||
? error.message
|
||||
: "Failed to verify tracer-link readiness";
|
||||
toast.error(message);
|
||||
}
|
||||
}, [refreshReadiness]);
|
||||
|
||||
const setRxResumeValidationStatus = useCallback(
|
||||
(mode: RxResumeMode, validation: ValidationResult) => {
|
||||
setRxresumeValidationStatuses((current) => ({
|
||||
...current,
|
||||
[mode]: toRxResumeValidationBadgeState(validation),
|
||||
}));
|
||||
},
|
||||
[],
|
||||
);
|
||||
|
||||
const clearRxResumeValidationFeedback = useCallback(
|
||||
(mode: RxResumeMode) => {
|
||||
setRxresumeValidationStatuses((current) => ({
|
||||
...current,
|
||||
[mode]: EMPTY_RXRESUME_VALIDATION_BADGE_STATE,
|
||||
}));
|
||||
clearErrors(getRxResumeValidationFieldsForMode(mode));
|
||||
},
|
||||
[clearErrors],
|
||||
);
|
||||
|
||||
const validateRxresumeMode = useCallback(
|
||||
async (
|
||||
mode: RxResumeMode,
|
||||
options?: { silent?: boolean; persistOnSuccess?: boolean },
|
||||
) => {
|
||||
const { silent = false, persistOnSuccess = true } = options ?? {};
|
||||
const notify = !silent;
|
||||
const values = getValues();
|
||||
const draftCredentials = getRxResumeCredentialDrafts(values);
|
||||
const result = await validateAndMaybePersistRxResumeMode({
|
||||
mode,
|
||||
stored: storedRxResume,
|
||||
draft: draftCredentials,
|
||||
validate: api.validateRxresume,
|
||||
persist: api.updateSettings,
|
||||
persistOnSuccess,
|
||||
skipPrecheck: silent,
|
||||
getPrecheckMessage: (failure) => RXRESUME_PRECHECK_MESSAGES[failure],
|
||||
getValidationErrorMessage: (error) =>
|
||||
error instanceof Error ? error.message : "RxResume validation failed",
|
||||
getPersistErrorMessage: (error) =>
|
||||
error instanceof Error ? error.message : "RxResume validation failed",
|
||||
});
|
||||
|
||||
setRxResumeValidationStatus(mode, result.validation);
|
||||
|
||||
if (result.updatedSettings) {
|
||||
setSettings(result.updatedSettings);
|
||||
queryClient.setQueryData(
|
||||
queryKeys.settings.current(),
|
||||
result.updatedSettings,
|
||||
);
|
||||
if (notify) {
|
||||
toast.success(`Reactive Resume ${mode} validation passed`);
|
||||
}
|
||||
return;
|
||||
}
|
||||
|
||||
if (!notify || result.validation.valid) {
|
||||
return;
|
||||
}
|
||||
|
||||
if (result.precheckFailure) {
|
||||
toast.info(
|
||||
result.validation.message ??
|
||||
RXRESUME_PRECHECK_MESSAGES[result.precheckFailure],
|
||||
);
|
||||
return;
|
||||
}
|
||||
|
||||
toast.error(
|
||||
result.validation.message ||
|
||||
`Reactive Resume ${mode} validation failed`,
|
||||
);
|
||||
},
|
||||
[getValues, queryClient, setRxResumeValidationStatus, storedRxResume],
|
||||
);
|
||||
|
||||
useEffect(() => {
|
||||
if (!settings) return;
|
||||
|
||||
const modesToCheck = RXRESUME_MODES.filter(
|
||||
(mode) => !rxresumeValidationStatuses[mode].checked,
|
||||
);
|
||||
if (modesToCheck.length === 0) return;
|
||||
|
||||
void Promise.all(
|
||||
modesToCheck.map((mode) =>
|
||||
validateRxresumeMode(mode, { silent: true, persistOnSuccess: false }),
|
||||
),
|
||||
);
|
||||
}, [rxresumeValidationStatuses, settings, validateRxresumeMode]);
|
||||
|
||||
const effectiveProfileProjects =
|
||||
rxResumeProjectsOverride ??
|
||||
(selectedRxresumeMode === rxresumeMode ? profileProjects : []);
|
||||
const effectiveMaxProjectsTotal = effectiveProfileProjects.length;
|
||||
|
||||
const lockedCount = resumeProjectsValue?.lockedProjectIds.length ?? 0;
|
||||
|
||||
const canSave = isDirty && isValid;
|
||||
|
||||
const onSave = async (data: UpdateSettingsInput) => {
|
||||
@@ -781,14 +436,6 @@ export const SettingsPage: React.FC = () => {
|
||||
|
||||
const envPayload: Partial<UpdateSettingsInput> = {};
|
||||
|
||||
if (dirtyFields.rxresumeEmail || dirtyFields.rxresumePassword) {
|
||||
envPayload.rxresumeEmail = normalizeString(data.rxresumeEmail);
|
||||
}
|
||||
|
||||
if (dirtyFields.rxresumeUrl) {
|
||||
envPayload.rxresumeUrl = normalizeString(data.rxresumeUrl);
|
||||
}
|
||||
|
||||
if (dirtyFields.localResumeProfilePath) {
|
||||
envPayload.localResumeProfilePath = normalizeString(
|
||||
data.localResumeProfilePath,
|
||||
@@ -834,16 +481,6 @@ export const SettingsPage: React.FC = () => {
|
||||
if (value !== undefined) envPayload.llmApiKey = value;
|
||||
}
|
||||
|
||||
if (dirtyFields.rxresumePassword) {
|
||||
const value = normalizePrivateInput(data.rxresumePassword);
|
||||
if (value !== undefined) envPayload.rxresumePassword = value;
|
||||
}
|
||||
|
||||
if (dirtyFields.rxresumeApiKey) {
|
||||
const value = normalizePrivateInput(data.rxresumeApiKey);
|
||||
if (value !== undefined) envPayload.rxresumeApiKey = value;
|
||||
}
|
||||
|
||||
if (dirtyFields.ukvisajobsPassword) {
|
||||
const value = normalizePrivateInput(data.ukvisajobsPassword);
|
||||
if (value !== undefined) envPayload.ukvisajobsPassword = value;
|
||||
@@ -890,12 +527,6 @@ export const SettingsPage: React.FC = () => {
|
||||
pipelineWebhookUrl: normalizeString(data.pipelineWebhookUrl),
|
||||
jobCompleteWebhookUrl: normalizeString(data.jobCompleteWebhookUrl),
|
||||
resumeProjects: resumeProjectsOverride,
|
||||
...(dirtyFields.rxresumeMode
|
||||
? { rxresumeMode: data.rxresumeMode ?? "v5" }
|
||||
: {}),
|
||||
...(dirtyFields.rxresumeBaseResumeId
|
||||
? { rxresumeBaseResumeId: normalizeString(data.rxresumeBaseResumeId) }
|
||||
: {}),
|
||||
showSponsorInfo: nullIfSame(
|
||||
data.showSponsorInfo,
|
||||
display.showSponsorInfo.default,
|
||||
@@ -951,84 +582,107 @@ export const SettingsPage: React.FC = () => {
|
||||
normalizeString(data.scoringInstructions),
|
||||
scoring.scoringInstructions.default,
|
||||
),
|
||||
ashbyCompanies: (() => {
|
||||
const normalized = normalizeStringArray(data.ashbyCompanies);
|
||||
return stringArraysEqual(
|
||||
normalized,
|
||||
jobSources.ashbyCompanies.default,
|
||||
)
|
||||
? null
|
||||
: normalized;
|
||||
})(),
|
||||
greenhouseCompanies: (() => {
|
||||
const normalized = normalizeStringArray(data.greenhouseCompanies);
|
||||
return stringArraysEqual(
|
||||
normalized,
|
||||
jobSources.greenhouseCompanies.default,
|
||||
)
|
||||
? null
|
||||
: normalized;
|
||||
})(),
|
||||
leverCompanies: (() => {
|
||||
const normalized = normalizeStringArray(data.leverCompanies);
|
||||
return stringArraysEqual(
|
||||
normalized,
|
||||
jobSources.leverCompanies.default,
|
||||
)
|
||||
? null
|
||||
: normalized;
|
||||
})(),
|
||||
smartrecruitersCompanies: (() => {
|
||||
const normalized = normalizeStringArray(
|
||||
data.smartrecruitersCompanies,
|
||||
);
|
||||
return stringArraysEqual(
|
||||
normalized,
|
||||
jobSources.smartrecruitersCompanies.default,
|
||||
)
|
||||
? null
|
||||
: normalized;
|
||||
})(),
|
||||
teamtailorCompanies: (() => {
|
||||
const normalized = normalizeStringArray(data.teamtailorCompanies);
|
||||
return stringArraysEqual(
|
||||
normalized,
|
||||
jobSources.teamtailorCompanies.default,
|
||||
)
|
||||
? null
|
||||
: normalized;
|
||||
})(),
|
||||
huntflowTenants: (() => {
|
||||
const normalized = normalizeStringArray(data.huntflowTenants);
|
||||
return stringArraysEqual(
|
||||
normalized,
|
||||
jobSources.huntflowTenants.default,
|
||||
)
|
||||
? null
|
||||
: normalized;
|
||||
})(),
|
||||
factorialTenants: (() => {
|
||||
const normalized = normalizeStringArray(data.factorialTenants);
|
||||
return stringArraysEqual(
|
||||
normalized,
|
||||
jobSources.factorialTenants.default,
|
||||
)
|
||||
? null
|
||||
: normalized;
|
||||
})(),
|
||||
workdayTenants: (() => {
|
||||
const normalized = normalizeStringArray(data.workdayTenants);
|
||||
return stringArraysEqual(
|
||||
normalized,
|
||||
jobSources.workdayTenants.default,
|
||||
)
|
||||
? null
|
||||
: normalized;
|
||||
})(),
|
||||
careersPageUrls: (() => {
|
||||
const normalized = normalizeStringArray(data.careersPageUrls);
|
||||
return stringArraysEqual(
|
||||
normalized,
|
||||
jobSources.careersPageUrls.default,
|
||||
)
|
||||
? null
|
||||
: normalized;
|
||||
})(),
|
||||
icimsTenants: (() => {
|
||||
const normalized = normalizeStringArray(data.icimsTenants);
|
||||
return stringArraysEqual(normalized, jobSources.icimsTenants.default)
|
||||
? null
|
||||
: normalized;
|
||||
})(),
|
||||
elutaRssLocations: (() => {
|
||||
const normalized = normalizeStringArray(data.elutaRssLocations);
|
||||
return stringArraysEqual(
|
||||
normalized,
|
||||
jobSources.elutaRssLocations.default,
|
||||
)
|
||||
? null
|
||||
: normalized;
|
||||
})(),
|
||||
...envPayload,
|
||||
};
|
||||
|
||||
const shouldValidateRxResumeBeforeSave = Boolean(
|
||||
dirtyFields.rxresumeMode ||
|
||||
dirtyFields.rxresumeUrl ||
|
||||
dirtyFields.rxresumeApiKey ||
|
||||
dirtyFields.rxresumeEmail ||
|
||||
dirtyFields.rxresumePassword,
|
||||
);
|
||||
const rxResumeValidationMode = (data.rxresumeMode ??
|
||||
rxresumeMode) as RxResumeMode;
|
||||
let rxResumeSaveWarningMessage: string | null = null;
|
||||
|
||||
if (shouldValidateRxResumeBeforeSave) {
|
||||
const validationDraft = getRxResumeCredentialDrafts(data);
|
||||
const precheckFailure = getRxResumeCredentialPrecheckFailure({
|
||||
mode: rxResumeValidationMode,
|
||||
stored: storedRxResume,
|
||||
draft: validationDraft,
|
||||
});
|
||||
|
||||
if (!precheckFailure) {
|
||||
const preserveBlankFields = [
|
||||
...(dirtyFields.rxresumeEmail ? (["email"] as const) : []),
|
||||
...(dirtyFields.rxresumePassword ? (["password"] as const) : []),
|
||||
...(dirtyFields.rxresumeApiKey ? (["apiKey"] as const) : []),
|
||||
...(dirtyFields.rxresumeUrl ? (["baseUrl"] as const) : []),
|
||||
];
|
||||
const validation = await api.validateRxresume({
|
||||
mode: rxResumeValidationMode,
|
||||
...toRxResumeValidationPayload(validationDraft, {
|
||||
preserveBlankFields: preserveBlankFields as Array<
|
||||
keyof ReturnType<typeof getRxResumeCredentialDrafts>
|
||||
>,
|
||||
}),
|
||||
});
|
||||
|
||||
setRxResumeValidationStatus(rxResumeValidationMode, validation);
|
||||
|
||||
if (isRxResumeBlockingValidationFailure(validation)) {
|
||||
clearErrors(
|
||||
getRxResumeValidationFieldsForMode(rxResumeValidationMode),
|
||||
);
|
||||
if (rxResumeValidationMode === "v5") {
|
||||
setError("rxresumeApiKey", {
|
||||
type: "manual",
|
||||
message:
|
||||
validation.message ??
|
||||
"Reactive Resume v5 API key is invalid.",
|
||||
});
|
||||
} else {
|
||||
setError("rxresumeEmail", {
|
||||
type: "manual",
|
||||
message:
|
||||
validation.message ??
|
||||
"Reactive Resume v4 email/password is invalid.",
|
||||
});
|
||||
setError("rxresumePassword", {
|
||||
type: "manual",
|
||||
message:
|
||||
validation.message ??
|
||||
"Reactive Resume v4 email/password is invalid.",
|
||||
});
|
||||
}
|
||||
return;
|
||||
}
|
||||
|
||||
clearErrors(
|
||||
getRxResumeValidationFieldsForMode(rxResumeValidationMode),
|
||||
);
|
||||
if (isRxResumeAvailabilityValidationFailure(validation)) {
|
||||
rxResumeSaveWarningMessage =
|
||||
"Settings saved, but JobOps could not verify Reactive Resume because the instance is unavailable.";
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
const updated = await updateSettingsMutation.mutateAsync(payload);
|
||||
|
||||
if (
|
||||
@@ -1048,9 +702,6 @@ export const SettingsPage: React.FC = () => {
|
||||
setSettings(updated);
|
||||
reset(mapSettingsToForm(updated));
|
||||
toast.success("Settings saved");
|
||||
if (rxResumeSaveWarningMessage) {
|
||||
toast.info(rxResumeSaveWarningMessage);
|
||||
}
|
||||
} catch (error) {
|
||||
const message =
|
||||
error instanceof Error ? error.message : "Failed to save settings";
|
||||
@@ -1137,6 +788,57 @@ export const SettingsPage: React.FC = () => {
|
||||
}
|
||||
};
|
||||
|
||||
const handleClearDiscoveredMatchingArchived = async () => {
|
||||
try {
|
||||
setIsSaving(true);
|
||||
const result = await api.deleteDiscoveredMatchingArchived();
|
||||
|
||||
if (result.count > 0) {
|
||||
toast.success("Rediscovered matches cleared", {
|
||||
description: `Deleted ${result.count} discovered job${result.count === 1 ? "" : "s"} matching skipped or applied roles.`,
|
||||
});
|
||||
} else {
|
||||
toast.info("No matches found", {
|
||||
description:
|
||||
"No discovered jobs matched skipped or applied archive roles.",
|
||||
});
|
||||
}
|
||||
} catch (error) {
|
||||
const message =
|
||||
error instanceof Error
|
||||
? error.message
|
||||
: "Failed to clear rediscovered archive matches";
|
||||
toast.error(message);
|
||||
} finally {
|
||||
setIsSaving(false);
|
||||
}
|
||||
};
|
||||
|
||||
const handleExpireDeadListings = async () => {
|
||||
try {
|
||||
setIsSaving(true);
|
||||
const result = await api.expireDeadListings();
|
||||
|
||||
if (result.count > 0) {
|
||||
toast.success("Expired listings marked", {
|
||||
description: `Marked ${result.count} discovered job${result.count === 1 ? "" : "s"} as expired.`,
|
||||
});
|
||||
} else {
|
||||
toast.info("No expired listings found", {
|
||||
description: "All probed discovered job links still look live.",
|
||||
});
|
||||
}
|
||||
} catch (error) {
|
||||
const message =
|
||||
error instanceof Error
|
||||
? error.message
|
||||
: "Failed to mark expired listings";
|
||||
toast.error(message);
|
||||
} finally {
|
||||
setIsSaving(false);
|
||||
}
|
||||
};
|
||||
|
||||
const toggleStatusToClear = (status: JobStatus) => {
|
||||
setStatusesToClear((prev) =>
|
||||
prev.includes(status)
|
||||
@@ -1177,52 +879,8 @@ export const SettingsPage: React.FC = () => {
|
||||
isLoading={isLoading}
|
||||
isSaving={isSaving}
|
||||
/>
|
||||
<WebhooksSection
|
||||
pipelineWebhook={pipelineWebhook}
|
||||
jobCompleteWebhook={jobCompleteWebhook}
|
||||
webhookSecretHint={envSettings.private.webhookSecretHint}
|
||||
isLoading={isLoading}
|
||||
isSaving={isSaving}
|
||||
/>
|
||||
<ReactiveResumeSection
|
||||
rxResumeBaseResumeIdDraft={rxResumeBaseResumeIdDraft}
|
||||
onRxresumeModeChange={(mode) => {
|
||||
const nextId = getBaseResumeIdForMode(mode);
|
||||
setRxResumeBaseResumeIdDraft(nextId);
|
||||
setValue("rxresumeBaseResumeId", nextId, { shouldDirty: true });
|
||||
setRxResumeProjectsOverride(null);
|
||||
}}
|
||||
setRxResumeBaseResumeIdDraft={(value) => {
|
||||
const mode = (getValues("rxresumeMode") ??
|
||||
rxresumeMode) as RxResumeMode;
|
||||
setBaseResumeIdForMode(mode, value);
|
||||
setRxResumeBaseResumeIdDraft(value);
|
||||
setValue("rxresumeBaseResumeId", value, { shouldDirty: true });
|
||||
}}
|
||||
hasRxResumeAccess={hasRxResumeAccess}
|
||||
rxresumeMode={rxresumeMode}
|
||||
onCredentialFieldEdit={clearRxResumeValidationFeedback}
|
||||
validationStatuses={rxresumeValidationStatuses}
|
||||
profileProjects={effectiveProfileProjects}
|
||||
lockedCount={lockedCount}
|
||||
maxProjectsTotal={effectiveMaxProjectsTotal}
|
||||
isProjectsLoading={isFetchingRxResumeProjects}
|
||||
isLoading={isLoading}
|
||||
isSaving={isSaving}
|
||||
/>
|
||||
<TracerLinksSettingsSection
|
||||
readiness={tracerReadiness}
|
||||
isLoading={isLoading || isTracerReadinessLoading}
|
||||
isChecking={isTracerReadinessChecking}
|
||||
onVerifyNow={handleVerifyTracerReadiness}
|
||||
/>
|
||||
<DisplaySettingsSection
|
||||
values={display}
|
||||
isLoading={isLoading}
|
||||
isSaving={isSaving}
|
||||
/>
|
||||
<ChatSettingsSection
|
||||
values={chat}
|
||||
<ScoringSettingsSection
|
||||
values={scoring}
|
||||
isLoading={isLoading}
|
||||
isSaving={isSaving}
|
||||
/>
|
||||
@@ -1232,8 +890,19 @@ export const SettingsPage: React.FC = () => {
|
||||
isLoading={isLoading}
|
||||
isSaving={isSaving}
|
||||
/>
|
||||
<ScoringSettingsSection
|
||||
values={scoring}
|
||||
<KeywordSetsSettingsSection
|
||||
isLoading={isLoading}
|
||||
isSaving={isSaving}
|
||||
/>
|
||||
<JobSourcesSettingsSection
|
||||
values={jobSources}
|
||||
isLoading={isLoading}
|
||||
isSaving={isSaving}
|
||||
/>
|
||||
<WebhooksSection
|
||||
pipelineWebhook={pipelineWebhook}
|
||||
jobCompleteWebhook={jobCompleteWebhook}
|
||||
webhookSecretHint={envSettings.private.webhookSecretHint}
|
||||
isLoading={isLoading}
|
||||
isSaving={isSaving}
|
||||
/>
|
||||
@@ -1242,23 +911,16 @@ export const SettingsPage: React.FC = () => {
|
||||
isLoading={isLoading}
|
||||
isSaving={isSaving}
|
||||
/>
|
||||
<BackupSettingsSection
|
||||
values={backup}
|
||||
backups={backups}
|
||||
nextScheduled={nextScheduled}
|
||||
isLoading={isLoading || isLoadingBackups}
|
||||
isSaving={isSaving}
|
||||
onCreateBackup={handleCreateBackup}
|
||||
onDeleteBackup={handleDeleteBackup}
|
||||
isCreatingBackup={isCreatingBackup}
|
||||
isDeletingBackup={isDeletingBackup}
|
||||
/>
|
||||
<DangerZoneSection
|
||||
statusesToClear={statusesToClear}
|
||||
toggleStatusToClear={toggleStatusToClear}
|
||||
handleClearByStatuses={handleClearByStatuses}
|
||||
handleClearDatabase={handleClearDatabase}
|
||||
handleClearByScore={handleClearByScore}
|
||||
handleClearDiscoveredMatchingArchived={
|
||||
handleClearDiscoveredMatchingArchived
|
||||
}
|
||||
handleExpireDeadListings={handleExpireDeadListings}
|
||||
isLoading={isLoading}
|
||||
isSaving={isSaving}
|
||||
/>
|
||||
|
||||
@@ -114,6 +114,7 @@ beforeEach(() => {
|
||||
status: "applied",
|
||||
suitabilityScore: null,
|
||||
sponsorMatchScore: null,
|
||||
sponsorshipSignals: null,
|
||||
jobType: null,
|
||||
jobFunction: null,
|
||||
salaryMinAmount: null,
|
||||
@@ -137,6 +138,7 @@ beforeEach(() => {
|
||||
status: "applied",
|
||||
suitabilityScore: null,
|
||||
sponsorMatchScore: null,
|
||||
sponsorshipSignals: null,
|
||||
jobType: null,
|
||||
jobFunction: null,
|
||||
salaryMinAmount: null,
|
||||
|
||||
@@ -25,6 +25,10 @@ vi.mock("@/lib/user-location", () => ({
|
||||
getDetectedCountryKey: getDetectedCountryKeyMock,
|
||||
}));
|
||||
|
||||
vi.mock("./KeywordSetPicker", () => ({
|
||||
KeywordSetPicker: () => null,
|
||||
}));
|
||||
|
||||
describe("AutomaticRunTab", () => {
|
||||
beforeEach(() => {
|
||||
getDetectedCountryKeyMock.mockReset();
|
||||
|
||||
@@ -5,7 +5,7 @@ import {
|
||||
normalizeCountryKey,
|
||||
SUPPORTED_COUNTRY_KEYS,
|
||||
} from "@shared/location-support.js";
|
||||
import type { AppSettings, JobSource } from "@shared/types";
|
||||
import type { AppSettings, JobSource, KeywordSet } from "@shared/types";
|
||||
import { Loader2, Sparkles } from "lucide-react";
|
||||
import { useCallback, useEffect, useMemo, useState } from "react";
|
||||
import { useForm } from "react-hook-form";
|
||||
@@ -45,6 +45,7 @@ import {
|
||||
WORKPLACE_TYPE_OPTIONS,
|
||||
type WorkplaceType,
|
||||
} from "./automatic-run";
|
||||
import { KeywordSetPicker } from "./KeywordSetPicker";
|
||||
import { TokenizedInput } from "./TokenizedInput";
|
||||
|
||||
interface AutomaticRunTabProps {
|
||||
@@ -62,7 +63,8 @@ const DEFAULT_VALUES: AutomaticRunValues = {
|
||||
topN: 10,
|
||||
minSuitabilityScore: 50,
|
||||
searchTerms: ["web developer"],
|
||||
runBudget: 200,
|
||||
activeKeywordSetId: null,
|
||||
runBudget: 500,
|
||||
country: "united kingdom",
|
||||
cityLocations: [],
|
||||
workplaceTypes: ["remote", "hybrid", "onsite"],
|
||||
@@ -163,6 +165,9 @@ export const AutomaticRunTab: React.FC<AutomaticRunTabProps> = ({
|
||||
}) => {
|
||||
const [isSaving, setIsSaving] = useState(false);
|
||||
const [advancedOpen, setAdvancedOpen] = useState(false);
|
||||
const [activeKeywordSetId, setActiveKeywordSetId] = useState<string | null>(
|
||||
null,
|
||||
);
|
||||
const { watch, reset, setValue } = useForm<AutomaticRunFormValues>({
|
||||
defaultValues: {
|
||||
topN: String(DEFAULT_VALUES.topN),
|
||||
@@ -187,6 +192,25 @@ export const AutomaticRunTab: React.FC<AutomaticRunTabProps> = ({
|
||||
const searchTerms = watch("searchTerms");
|
||||
const searchTermDraft = watch("searchTermDraft");
|
||||
|
||||
const handleActiveKeywordSetChange = useCallback(
|
||||
(set: KeywordSet) => {
|
||||
setActiveKeywordSetId(set.id);
|
||||
const terms = set.terms.map((t) => t.trim()).filter(Boolean);
|
||||
if (terms.length > 0) {
|
||||
setValue("searchTerms", terms, { shouldDirty: false });
|
||||
return;
|
||||
}
|
||||
const profileRoles =
|
||||
settings?.jobSearchProfile?.value?.targetRoles
|
||||
?.map((t) => t.trim())
|
||||
.filter((t): t is string => Boolean(t)) ?? [];
|
||||
if (profileRoles.length > 0) {
|
||||
setValue("searchTerms", profileRoles, { shouldDirty: false });
|
||||
}
|
||||
},
|
||||
[setValue, settings?.jobSearchProfile?.value?.targetRoles],
|
||||
);
|
||||
|
||||
useEffect(() => {
|
||||
if (!open) return;
|
||||
const memory = loadAutomaticRunMemory();
|
||||
@@ -274,6 +298,7 @@ export const AutomaticRunTab: React.FC<AutomaticRunTabProps> = ({
|
||||
cityLocations,
|
||||
workplaceTypes: normalizeWorkplaceTypes(workplaceTypes),
|
||||
searchTerms,
|
||||
activeKeywordSetId,
|
||||
};
|
||||
}, [
|
||||
topNInput,
|
||||
@@ -283,6 +308,7 @@ export const AutomaticRunTab: React.FC<AutomaticRunTabProps> = ({
|
||||
cityLocations,
|
||||
workplaceTypes,
|
||||
searchTerms,
|
||||
activeKeywordSetId,
|
||||
]);
|
||||
|
||||
const workplaceTypeSelectionInvalid = workplaceTypes.length === 0;
|
||||
@@ -320,13 +346,20 @@ export const AutomaticRunTab: React.FC<AutomaticRunTabProps> = ({
|
||||
const filtered = pipelineSources.filter((source) =>
|
||||
isSourceAvailableForRun(source),
|
||||
);
|
||||
const missing = compatibleEnabledSources.filter(
|
||||
(source) => !filtered.includes(source),
|
||||
);
|
||||
if (missing.length > 0) {
|
||||
onSetPipelineSources([...filtered, ...missing]);
|
||||
return;
|
||||
}
|
||||
if (filtered.length === pipelineSources.length) return;
|
||||
if (filtered.length > 0) {
|
||||
onSetPipelineSources(filtered);
|
||||
return;
|
||||
}
|
||||
if (compatibleEnabledSources.length > 0) {
|
||||
onSetPipelineSources([compatibleEnabledSources[0]]);
|
||||
onSetPipelineSources([...compatibleEnabledSources]);
|
||||
}
|
||||
}, [
|
||||
compatibleEnabledSources,
|
||||
@@ -335,14 +368,19 @@ export const AutomaticRunTab: React.FC<AutomaticRunTabProps> = ({
|
||||
pipelineSources,
|
||||
]);
|
||||
|
||||
const mergedDiscoveryTerms = useMemo(
|
||||
() =>
|
||||
mergeDiscoverySearchTerms(
|
||||
values.searchTerms,
|
||||
settings?.jobSearchProfile?.value?.targetRoles,
|
||||
),
|
||||
[values.searchTerms, settings?.jobSearchProfile?.value?.targetRoles],
|
||||
);
|
||||
const mergedDiscoveryTerms = useMemo(() => {
|
||||
if (values.activeKeywordSetId) {
|
||||
return values.searchTerms.map((t) => t.trim()).filter(Boolean);
|
||||
}
|
||||
return mergeDiscoverySearchTerms(
|
||||
values.searchTerms,
|
||||
settings?.jobSearchProfile?.value?.targetRoles,
|
||||
);
|
||||
}, [
|
||||
values.searchTerms,
|
||||
values.activeKeywordSetId,
|
||||
settings?.jobSearchProfile?.value?.targetRoles,
|
||||
]);
|
||||
|
||||
const estimate = useMemo(
|
||||
() =>
|
||||
@@ -600,7 +638,12 @@ export const AutomaticRunTab: React.FC<AutomaticRunTabProps> = ({
|
||||
<CardHeader className="pb-3">
|
||||
<CardTitle>Search terms</CardTitle>
|
||||
</CardHeader>
|
||||
<CardContent>
|
||||
<CardContent className="space-y-4">
|
||||
<KeywordSetPicker
|
||||
key={settings?.activeProfileId ?? "default"}
|
||||
disabled={isSaving || isPipelineRunning}
|
||||
onActiveSetChange={handleActiveKeywordSetChange}
|
||||
/>
|
||||
<TokenizedInput
|
||||
id="search-terms-input"
|
||||
values={searchTerms}
|
||||
@@ -619,10 +662,37 @@ export const AutomaticRunTab: React.FC<AutomaticRunTabProps> = ({
|
||||
|
||||
<Card>
|
||||
<CardHeader className="pb-3">
|
||||
<CardTitle>
|
||||
Sources ({compatiblePipelineSources.length}/
|
||||
{compatibleEnabledSources.length})
|
||||
</CardTitle>
|
||||
<div className="flex flex-wrap items-center justify-between gap-2">
|
||||
<CardTitle>
|
||||
Sources ({compatiblePipelineSources.length}/
|
||||
{compatibleEnabledSources.length})
|
||||
</CardTitle>
|
||||
<div className="flex gap-2">
|
||||
<Button
|
||||
type="button"
|
||||
size="sm"
|
||||
variant="outline"
|
||||
disabled={isSaving || isPipelineRunning}
|
||||
onClick={() =>
|
||||
onSetPipelineSources([...compatibleEnabledSources])
|
||||
}
|
||||
>
|
||||
Select all
|
||||
</Button>
|
||||
<Button
|
||||
type="button"
|
||||
size="sm"
|
||||
variant="outline"
|
||||
disabled={isSaving || isPipelineRunning}
|
||||
onClick={() => {
|
||||
if (compatibleEnabledSources.length === 0) return;
|
||||
onSetPipelineSources([compatibleEnabledSources[0]]);
|
||||
}}
|
||||
>
|
||||
Clear
|
||||
</Button>
|
||||
</div>
|
||||
</div>
|
||||
</CardHeader>
|
||||
<CardContent className="flex flex-wrap gap-2">
|
||||
<TooltipProvider>
|
||||
|
||||
@@ -37,7 +37,7 @@ describe("JobCommandBar", () => {
|
||||
|
||||
expect(
|
||||
screen.getByPlaceholderText(
|
||||
"Search jobs by job title or company name...",
|
||||
"Search jobs by title, company, location, or sponsorship signals...",
|
||||
),
|
||||
).toBeInTheDocument();
|
||||
});
|
||||
@@ -52,7 +52,7 @@ describe("JobCommandBar", () => {
|
||||
|
||||
openWithKeyboard();
|
||||
const input = screen.getByPlaceholderText(
|
||||
"Search jobs by job title or company name...",
|
||||
"Search jobs by title, company, location, or sponsorship signals...",
|
||||
);
|
||||
fireEvent.change(input, { target: { value: "@disc" } });
|
||||
fireEvent.keyDown(input, { key: "Tab" });
|
||||
@@ -70,7 +70,7 @@ describe("JobCommandBar", () => {
|
||||
|
||||
openWithKeyboard();
|
||||
const input = screen.getByPlaceholderText(
|
||||
"Search jobs by job title or company name...",
|
||||
"Search jobs by title, company, location, or sponsorship signals...",
|
||||
);
|
||||
fireEvent.change(input, { target: { value: "@ready" } });
|
||||
fireEvent.keyDown(input, { key: "Enter" });
|
||||
@@ -91,7 +91,7 @@ describe("JobCommandBar", () => {
|
||||
expect(dialog.className).not.toContain("border-sky-500/50");
|
||||
|
||||
const input = screen.getByPlaceholderText(
|
||||
"Search jobs by job title or company name...",
|
||||
"Search jobs by title, company, location, or sponsorship signals...",
|
||||
);
|
||||
fireEvent.change(input, { target: { value: "@disc" } });
|
||||
fireEvent.keyDown(input, { key: "Tab" });
|
||||
@@ -118,7 +118,7 @@ describe("JobCommandBar", () => {
|
||||
|
||||
openWithKeyboard();
|
||||
const input = screen.getByPlaceholderText(
|
||||
"Search jobs by job title or company name...",
|
||||
"Search jobs by title, company, location, or sponsorship signals...",
|
||||
);
|
||||
fireEvent.change(input, { target: { value: "@ready" } });
|
||||
|
||||
@@ -141,7 +141,7 @@ describe("JobCommandBar", () => {
|
||||
|
||||
openWithKeyboard();
|
||||
const input = screen.getByPlaceholderText(
|
||||
"Search jobs by job title or company name...",
|
||||
"Search jobs by title, company, location, or sponsorship signals...",
|
||||
);
|
||||
fireEvent.change(input, { target: { value: "@" } });
|
||||
|
||||
@@ -163,7 +163,7 @@ describe("JobCommandBar", () => {
|
||||
|
||||
openWithKeyboard();
|
||||
const input = screen.getByPlaceholderText(
|
||||
"Search jobs by job title or company name...",
|
||||
"Search jobs by title, company, location, or sponsorship signals...",
|
||||
);
|
||||
fireEvent.change(input, { target: { value: "@prog" } });
|
||||
fireEvent.keyDown(input, { key: "Tab" });
|
||||
@@ -192,7 +192,7 @@ describe("JobCommandBar", () => {
|
||||
openWithKeyboard();
|
||||
fireEvent.change(
|
||||
screen.getByPlaceholderText(
|
||||
"Search jobs by job title or company name...",
|
||||
"Search jobs by title, company, location, or sponsorship signals...",
|
||||
),
|
||||
{
|
||||
target: { value: "Globex" },
|
||||
@@ -220,7 +220,7 @@ describe("JobCommandBar", () => {
|
||||
|
||||
openWithKeyboard();
|
||||
const input = screen.getByPlaceholderText(
|
||||
"Search jobs by job title or company name...",
|
||||
"Search jobs by title, company, location, or sponsorship signals...",
|
||||
);
|
||||
fireEvent.change(input, { target: { value: "@disc" } });
|
||||
fireEvent.keyDown(input, { key: "Tab" });
|
||||
@@ -257,7 +257,7 @@ describe("JobCommandBar", () => {
|
||||
|
||||
openWithKeyboard();
|
||||
const input = screen.getByPlaceholderText(
|
||||
"Search jobs by job title or company name...",
|
||||
"Search jobs by title, company, location, or sponsorship signals...",
|
||||
);
|
||||
fireEvent.change(input, { target: { value: "@disc" } });
|
||||
fireEvent.keyDown(input, { key: "Tab" });
|
||||
@@ -280,7 +280,7 @@ describe("JobCommandBar", () => {
|
||||
|
||||
openWithKeyboard();
|
||||
const input = screen.getByPlaceholderText(
|
||||
"Search jobs by job title or company name...",
|
||||
"Search jobs by title, company, location, or sponsorship signals...",
|
||||
);
|
||||
fireEvent.change(input, { target: { value: "@ready" } });
|
||||
fireEvent.keyDown(input, { key: "Tab" });
|
||||
@@ -302,7 +302,7 @@ describe("JobCommandBar", () => {
|
||||
|
||||
openWithKeyboard();
|
||||
const input = screen.getByPlaceholderText(
|
||||
"Search jobs by job title or company name...",
|
||||
"Search jobs by title, company, location, or sponsorship signals...",
|
||||
);
|
||||
fireEvent.change(input, { target: { value: "@ready" } });
|
||||
fireEvent.keyDown(input, { key: "Tab" });
|
||||
@@ -323,7 +323,7 @@ describe("JobCommandBar", () => {
|
||||
|
||||
openWithKeyboard();
|
||||
const input = screen.getByPlaceholderText(
|
||||
"Search jobs by job title or company name...",
|
||||
"Search jobs by title, company, location, or sponsorship signals...",
|
||||
);
|
||||
fireEvent.change(input, { target: { value: "@ready" } });
|
||||
fireEvent.keyDown(input, { key: "Tab" });
|
||||
@@ -342,7 +342,7 @@ describe("JobCommandBar", () => {
|
||||
|
||||
openWithKeyboard();
|
||||
const input = screen.getByPlaceholderText(
|
||||
"Search jobs by job title or company name...",
|
||||
"Search jobs by title, company, location, or sponsorship signals...",
|
||||
);
|
||||
fireEvent.change(input, { target: { value: "@ready" } });
|
||||
fireEvent.keyDown(input, { key: "Tab" });
|
||||
@@ -353,7 +353,7 @@ describe("JobCommandBar", () => {
|
||||
expect(screen.queryByText("@ready")).not.toBeInTheDocument();
|
||||
expect(
|
||||
screen.getByPlaceholderText(
|
||||
"Search jobs by job title or company name...",
|
||||
"Search jobs by title, company, location, or sponsorship signals...",
|
||||
),
|
||||
).toBeInTheDocument();
|
||||
});
|
||||
@@ -368,7 +368,7 @@ describe("JobCommandBar", () => {
|
||||
|
||||
openWithKeyboard();
|
||||
const input = screen.getByPlaceholderText(
|
||||
"Search jobs by job title or company name...",
|
||||
"Search jobs by title, company, location, or sponsorship signals...",
|
||||
);
|
||||
fireEvent.change(input, { target: { value: "@ready" } });
|
||||
fireEvent.keyDown(input, { key: "Tab" });
|
||||
@@ -390,7 +390,7 @@ describe("JobCommandBar", () => {
|
||||
|
||||
openWithKeyboard();
|
||||
const input = screen.getByPlaceholderText(
|
||||
"Search jobs by job title or company name...",
|
||||
"Search jobs by title, company, location, or sponsorship signals...",
|
||||
);
|
||||
fireEvent.change(input, { target: { value: "@all" } });
|
||||
fireEvent.keyDown(input, { key: "Tab" });
|
||||
@@ -423,7 +423,7 @@ describe("JobCommandBar", () => {
|
||||
openWithKeyboard();
|
||||
fireEvent.change(
|
||||
screen.getByPlaceholderText(
|
||||
"Search jobs by job title or company name...",
|
||||
"Search jobs by title, company, location, or sponsorship signals...",
|
||||
),
|
||||
{
|
||||
target: { value: "Globex" },
|
||||
@@ -472,7 +472,7 @@ describe("JobCommandBar", () => {
|
||||
|
||||
openWithKeyboard();
|
||||
const input = screen.getByPlaceholderText(
|
||||
"Search jobs by job title or company name...",
|
||||
"Search jobs by title, company, location, or sponsorship signals...",
|
||||
);
|
||||
const lockTokens = ["@ready", "@disc", "@applied", "@skip", "@exp"];
|
||||
|
||||
|
||||
@@ -169,10 +169,11 @@ export const JobCommandBar: React.FC<JobCommandBarProps> = ({
|
||||
>
|
||||
<DialogTitle className="sr-only">Job Search</DialogTitle>
|
||||
<DialogDescription className="sr-only">
|
||||
Search jobs across all states by job title or company name.
|
||||
Search jobs across all states by title, company, location, or
|
||||
sponsorship signals such as TN or visa sponsorship.
|
||||
</DialogDescription>
|
||||
<CommandInput
|
||||
placeholder="Search jobs by job title or company name..."
|
||||
placeholder="Search jobs by title, company, location, or sponsorship signals..."
|
||||
value={query}
|
||||
onValueChange={setQuery}
|
||||
onKeyDown={handleInputKeyDown}
|
||||
|
||||
@@ -1,3 +1,4 @@
|
||||
import { sponsorshipSignalSearchHaystack } from "@shared/sponsorship-signals";
|
||||
import type { JobListItem, JobStatus } from "@shared/types.js";
|
||||
import type { FilterTab } from "./constants";
|
||||
|
||||
@@ -144,12 +145,23 @@ export const computeJobMatchScore = (
|
||||
job.location ?? "",
|
||||
normalizedQuery,
|
||||
);
|
||||
const sponsorshipScore = computeFieldMatchScore(
|
||||
sponsorshipSignalSearchHaystack(job.sponsorshipSignals),
|
||||
normalizedQuery,
|
||||
);
|
||||
|
||||
// Prefer title/company matches over location when scores tie.
|
||||
// Only apply bias when a field actually matched.
|
||||
const titleRankedScore = titleScore > 0 ? titleScore + 8 : 0;
|
||||
const employerRankedScore = employerScore > 0 ? employerScore + 12 : 0;
|
||||
return Math.max(titleRankedScore, employerRankedScore, locationScore);
|
||||
const sponsorshipRankedScore =
|
||||
sponsorshipScore > 0 ? sponsorshipScore + 4 : 0;
|
||||
return Math.max(
|
||||
titleRankedScore,
|
||||
employerRankedScore,
|
||||
locationScore,
|
||||
sponsorshipRankedScore,
|
||||
);
|
||||
};
|
||||
|
||||
export const groupJobsForCommandBar = (
|
||||
|
||||
@@ -1,3 +1,4 @@
|
||||
import { SponsorshipSignalsPills } from "@client/components/SponsorshipSignalsPills";
|
||||
import type { JobListItem } from "@shared/types.js";
|
||||
import { cn } from "@/lib/utils";
|
||||
import { defaultStatusToken, statusTokens } from "./constants";
|
||||
@@ -77,6 +78,11 @@ export const JobRowContent = ({
|
||||
Found {formatDiscoveredAt(job.discoveredAt)}
|
||||
</div>
|
||||
)}
|
||||
<SponsorshipSignalsPills
|
||||
sponsorshipSignals={job.sponsorshipSignals}
|
||||
size="xs"
|
||||
className="mt-1"
|
||||
/>
|
||||
</div>
|
||||
|
||||
{hasScore && (
|
||||
|
||||
@@ -0,0 +1,253 @@
|
||||
import * as api from "@client/api";
|
||||
import type { KeywordSet } from "@shared/types";
|
||||
import { Loader2, Plus, Trash2 } from "lucide-react";
|
||||
import type React from "react";
|
||||
import { useCallback, useEffect, useRef, useState } from "react";
|
||||
import { toast } from "sonner";
|
||||
import { Button } from "@/components/ui/button";
|
||||
import { Input } from "@/components/ui/input";
|
||||
import {
|
||||
Select,
|
||||
SelectContent,
|
||||
SelectItem,
|
||||
SelectTrigger,
|
||||
SelectValue,
|
||||
} from "@/components/ui/select";
|
||||
import { cn } from "@/lib/utils";
|
||||
|
||||
type KeywordSetPickerProps = {
|
||||
disabled?: boolean;
|
||||
onActiveSetChange: (set: KeywordSet) => void;
|
||||
};
|
||||
|
||||
export const KeywordSetPicker: React.FC<KeywordSetPickerProps> = ({
|
||||
disabled = false,
|
||||
onActiveSetChange,
|
||||
}) => {
|
||||
const [sets, setSets] = useState<KeywordSet[]>([]);
|
||||
const [loading, setLoading] = useState(false);
|
||||
const [creating, setCreating] = useState(false);
|
||||
const [newSetName, setNewSetName] = useState("");
|
||||
const [showCreate, setShowCreate] = useState(false);
|
||||
const onActiveSetChangeRef = useRef(onActiveSetChange);
|
||||
const lastNotifiedIdRef = useRef<string | null>(null);
|
||||
|
||||
onActiveSetChangeRef.current = onActiveSetChange;
|
||||
|
||||
const notifyActiveSet = useCallback((set: KeywordSet) => {
|
||||
if (lastNotifiedIdRef.current === set.id) return;
|
||||
lastNotifiedIdRef.current = set.id;
|
||||
onActiveSetChangeRef.current(set);
|
||||
}, []);
|
||||
|
||||
const loadSets = useCallback(async () => {
|
||||
setLoading(true);
|
||||
try {
|
||||
const list = await api.listKeywordSets();
|
||||
setSets(list);
|
||||
const active = list.find((s) => s.isActive) ?? list[0];
|
||||
if (active) notifyActiveSet(active);
|
||||
} catch {
|
||||
toast.error("Failed to load keyword sets");
|
||||
} finally {
|
||||
setLoading(false);
|
||||
}
|
||||
}, [notifyActiveSet]);
|
||||
|
||||
useEffect(() => {
|
||||
void loadSets();
|
||||
}, [loadSets]);
|
||||
|
||||
const handleSelect = useCallback(
|
||||
async (setId: string) => {
|
||||
if (disabled || loading) return;
|
||||
const current = sets.find((s) => s.id === setId);
|
||||
if (current?.isActive) return;
|
||||
setLoading(true);
|
||||
try {
|
||||
const activated = await api.activateKeywordSet(setId);
|
||||
setSets((prev) =>
|
||||
prev.map((s) => ({ ...s, isActive: s.id === activated.id })),
|
||||
);
|
||||
notifyActiveSet(activated);
|
||||
} catch {
|
||||
toast.error("Failed to switch keyword set");
|
||||
} finally {
|
||||
setLoading(false);
|
||||
}
|
||||
},
|
||||
[disabled, loading, notifyActiveSet, sets],
|
||||
);
|
||||
|
||||
const handleCreate = useCallback(async () => {
|
||||
const name = newSetName.trim();
|
||||
if (!name) {
|
||||
toast.error("Enter a name for the keyword set");
|
||||
return;
|
||||
}
|
||||
setCreating(true);
|
||||
try {
|
||||
const created = await api.createKeywordSet({ name, terms: [] });
|
||||
const activated = await api.activateKeywordSet(created.id);
|
||||
setSets((prev) => [
|
||||
...prev.map((s) => ({ ...s, isActive: false })),
|
||||
{ ...activated, isActive: true },
|
||||
]);
|
||||
notifyActiveSet(activated);
|
||||
setNewSetName("");
|
||||
setShowCreate(false);
|
||||
toast.success(`Keyword set "${activated.name}" created`);
|
||||
} catch (error) {
|
||||
const message =
|
||||
error instanceof Error ? error.message : "Failed to create keyword set";
|
||||
toast.error(message);
|
||||
} finally {
|
||||
setCreating(false);
|
||||
}
|
||||
}, [newSetName, notifyActiveSet]);
|
||||
|
||||
const handleDelete = useCallback(
|
||||
async (setId: string) => {
|
||||
if (disabled || loading) return;
|
||||
setLoading(true);
|
||||
try {
|
||||
await api.deleteKeywordSet(setId);
|
||||
lastNotifiedIdRef.current = null;
|
||||
const list = await api.listKeywordSets();
|
||||
setSets(list);
|
||||
const active = list.find((s) => s.isActive) ?? list[0];
|
||||
if (active) notifyActiveSet(active);
|
||||
toast.success("Keyword set deleted");
|
||||
} catch (error) {
|
||||
const message =
|
||||
error instanceof Error
|
||||
? error.message
|
||||
: "Failed to delete keyword set";
|
||||
toast.error(message);
|
||||
} finally {
|
||||
setLoading(false);
|
||||
}
|
||||
},
|
||||
[disabled, loading, notifyActiveSet],
|
||||
);
|
||||
|
||||
const activeSet = sets.find((s) => s.isActive) ?? sets[0];
|
||||
return (
|
||||
<div className="space-y-3">
|
||||
<div className="flex flex-wrap items-center gap-2">
|
||||
<span className="text-sm font-medium shrink-0">Active set</span>
|
||||
{activeSet ? (
|
||||
<Select
|
||||
value={activeSet.id}
|
||||
onValueChange={handleSelect}
|
||||
disabled={disabled || loading}
|
||||
>
|
||||
<SelectTrigger
|
||||
className="w-[min(100%,240px)]"
|
||||
aria-label="Keyword set"
|
||||
>
|
||||
<SelectValue placeholder="Keyword set" />
|
||||
</SelectTrigger>
|
||||
<SelectContent>
|
||||
{sets.map((set) => (
|
||||
<SelectItem key={set.id} value={set.id}>
|
||||
{set.name}
|
||||
{set.isActive ? " (active)" : ""}
|
||||
</SelectItem>
|
||||
))}
|
||||
</SelectContent>
|
||||
</Select>
|
||||
) : (
|
||||
<span className="text-sm text-muted-foreground">
|
||||
{loading ? "Loading sets…" : "No keyword sets"}
|
||||
</span>
|
||||
)}
|
||||
{loading ? (
|
||||
<Loader2 className="h-4 w-4 animate-spin text-muted-foreground" />
|
||||
) : null}
|
||||
<Button
|
||||
type="button"
|
||||
variant="outline"
|
||||
size="sm"
|
||||
disabled={disabled || creating}
|
||||
onClick={() => setShowCreate((v) => !v)}
|
||||
>
|
||||
<Plus className="h-4 w-4 mr-1" />
|
||||
New set
|
||||
</Button>
|
||||
{activeSet && sets.length > 1 ? (
|
||||
<Button
|
||||
type="button"
|
||||
variant="ghost"
|
||||
size="sm"
|
||||
disabled={disabled || loading}
|
||||
onClick={() => void handleDelete(activeSet.id)}
|
||||
title={`Delete "${activeSet.name}"`}
|
||||
>
|
||||
<Trash2 className="h-4 w-4" />
|
||||
</Button>
|
||||
) : null}
|
||||
</div>
|
||||
{showCreate ? (
|
||||
<div className="flex flex-wrap items-center gap-2">
|
||||
<Input
|
||||
value={newSetName}
|
||||
onChange={(e) => setNewSetName(e.target.value)}
|
||||
placeholder="e.g. SDET, Caseware, Lead"
|
||||
className="max-w-xs"
|
||||
disabled={disabled || creating}
|
||||
onKeyDown={(e) => {
|
||||
if (e.key === "Enter") {
|
||||
e.preventDefault();
|
||||
void handleCreate();
|
||||
}
|
||||
}}
|
||||
/>
|
||||
<Button
|
||||
type="button"
|
||||
size="sm"
|
||||
disabled={disabled || creating}
|
||||
onClick={() => void handleCreate()}
|
||||
>
|
||||
{creating ? <Loader2 className="h-4 w-4 animate-spin" /> : "Create"}
|
||||
</Button>
|
||||
</div>
|
||||
) : null}
|
||||
{sets.length > 0 ? (
|
||||
<div className="flex flex-wrap gap-1.5 pt-1">
|
||||
{sets.map((set) => {
|
||||
const isActive = set.id === activeSet?.id;
|
||||
return (
|
||||
<button
|
||||
key={set.id}
|
||||
type="button"
|
||||
disabled={disabled || loading || isActive}
|
||||
onClick={() => void handleSelect(set.id)}
|
||||
className={cn(
|
||||
"rounded-md border px-2.5 py-1 text-xs font-medium transition-colors",
|
||||
isActive
|
||||
? "border-primary bg-primary/10 text-foreground"
|
||||
: "border-border bg-muted/40 text-muted-foreground hover:bg-muted",
|
||||
)}
|
||||
>
|
||||
{set.name}
|
||||
<span className="ml-1 text-muted-foreground">
|
||||
({set.terms.length})
|
||||
</span>
|
||||
</button>
|
||||
);
|
||||
})}
|
||||
</div>
|
||||
) : !loading ? (
|
||||
<p className="text-xs text-amber-600 dark:text-amber-400">
|
||||
No keyword sets loaded. Use "New set" or check that the API
|
||||
is running.
|
||||
</p>
|
||||
) : null}
|
||||
<p className="text-xs text-muted-foreground">
|
||||
Switch between named lists for different searches. Terms below belong to
|
||||
the active set.
|
||||
</p>
|
||||
</div>
|
||||
);
|
||||
};
|
||||
@@ -53,9 +53,15 @@ const renderFilters = (overrides?: Partial<FiltersProps>) => {
|
||||
onEmployerIncludeChange: vi.fn(),
|
||||
onEmployerExcludeChange: vi.fn(),
|
||||
applySettingsCompanySkipList: true,
|
||||
hidePriorSkips: true,
|
||||
onHidePriorSkipsChange: vi.fn(),
|
||||
onApplySettingsCompanySkipListChange: vi.fn(),
|
||||
sponsorFilter: "all" as SponsorFilter,
|
||||
onSponsorFilterChange: vi.fn(),
|
||||
sponsorshipSignalsFilter: [],
|
||||
onSponsorshipSignalsFilterChange: vi.fn(),
|
||||
hideSponsorBlockers: false,
|
||||
onHideSponsorBlockersChange: vi.fn(),
|
||||
workplaceFilter: "all" as WorkplaceFilter,
|
||||
onWorkplaceFilterChange: vi.fn(),
|
||||
salaryFilter: {
|
||||
|
||||
@@ -1,6 +1,13 @@
|
||||
import { KbdHint } from "@client/components/KbdHint";
|
||||
import { getDisplayKey, SHORTCUTS } from "@client/lib/shortcut-map";
|
||||
import { formatCountryLabel } from "@shared/location-support";
|
||||
import {
|
||||
SPONSORSHIP_GREEN_FLAG_IDS,
|
||||
SPONSORSHIP_RED_FLAG_IDS,
|
||||
SPONSORSHIP_SIGNAL_META,
|
||||
SPONSORSHIP_YELLOW_FLAG_IDS,
|
||||
type SponsorshipSignalId,
|
||||
} from "@shared/sponsorship-signals";
|
||||
import type { JobSource } from "@shared/types.js";
|
||||
import { Filter, Search } from "lucide-react";
|
||||
import type React from "react";
|
||||
@@ -62,8 +69,14 @@ interface OrchestratorFiltersProps {
|
||||
onEmployerExcludeChange: (values: string[]) => void;
|
||||
applySettingsCompanySkipList: boolean;
|
||||
onApplySettingsCompanySkipListChange: (value: boolean) => void;
|
||||
hidePriorSkips: boolean;
|
||||
onHidePriorSkipsChange: (value: boolean) => void;
|
||||
sponsorFilter: SponsorFilter;
|
||||
onSponsorFilterChange: (value: SponsorFilter) => void;
|
||||
sponsorshipSignalsFilter: SponsorshipSignalId[];
|
||||
onSponsorshipSignalsFilterChange: (values: SponsorshipSignalId[]) => void;
|
||||
hideSponsorBlockers: boolean;
|
||||
onHideSponsorBlockersChange: (value: boolean) => void;
|
||||
workplaceFilter: WorkplaceFilter;
|
||||
onWorkplaceFilterChange: (value: WorkplaceFilter) => void;
|
||||
salaryFilter: SalaryFilter;
|
||||
@@ -172,8 +185,14 @@ export const OrchestratorFilters: React.FC<OrchestratorFiltersProps> = ({
|
||||
onEmployerExcludeChange,
|
||||
applySettingsCompanySkipList,
|
||||
onApplySettingsCompanySkipListChange,
|
||||
hidePriorSkips,
|
||||
onHidePriorSkipsChange,
|
||||
sponsorFilter,
|
||||
onSponsorFilterChange,
|
||||
sponsorshipSignalsFilter,
|
||||
onSponsorshipSignalsFilterChange,
|
||||
hideSponsorBlockers,
|
||||
onHideSponsorBlockersChange,
|
||||
workplaceFilter,
|
||||
onWorkplaceFilterChange,
|
||||
salaryFilter,
|
||||
@@ -238,7 +257,10 @@ export const OrchestratorFilters: React.FC<OrchestratorFiltersProps> = ({
|
||||
employerIncludeFilter.length > 0 || employerExcludeFilter.length > 0,
|
||||
) +
|
||||
Number(!applySettingsCompanySkipList) +
|
||||
Number(hidePriorSkips) +
|
||||
Number(sponsorFilter !== "all") +
|
||||
Number(sponsorshipSignalsFilter.length > 0) +
|
||||
Number(hideSponsorBlockers) +
|
||||
Number(workplaceFilter !== "all") +
|
||||
Number(
|
||||
(typeof salaryFilter.min === "number" && salaryFilter.min > 0) ||
|
||||
@@ -254,7 +276,10 @@ export const OrchestratorFilters: React.FC<OrchestratorFiltersProps> = ({
|
||||
employerIncludeFilter.length,
|
||||
employerExcludeFilter.length,
|
||||
applySettingsCompanySkipList,
|
||||
hidePriorSkips,
|
||||
sponsorFilter,
|
||||
sponsorshipSignalsFilter.length,
|
||||
hideSponsorBlockers,
|
||||
workplaceFilter,
|
||||
salaryFilter.min,
|
||||
salaryFilter.max,
|
||||
@@ -523,6 +548,29 @@ export const OrchestratorFilters: React.FC<OrchestratorFiltersProps> = ({
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div className="flex items-start gap-3 rounded-md border border-border/60 bg-muted/20 px-3 py-3">
|
||||
<Checkbox
|
||||
id="hide-prior-skips"
|
||||
checked={hidePriorSkips}
|
||||
onCheckedChange={(checked) => {
|
||||
onHidePriorSkipsChange(checked === true);
|
||||
}}
|
||||
/>
|
||||
<div className="space-y-1">
|
||||
<label
|
||||
htmlFor="hide-prior-skips"
|
||||
className="text-sm font-medium leading-none cursor-pointer"
|
||||
>
|
||||
Hide roles you already skipped
|
||||
</label>
|
||||
<p className="text-xs text-muted-foreground">
|
||||
Hides Discovered and Ready rows that match a skipped
|
||||
or applied job with the same company and title from
|
||||
the last 90 days.
|
||||
</p>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<TokenizedInput
|
||||
id="employer-include-filter"
|
||||
values={employerIncludeFilter}
|
||||
@@ -553,22 +601,160 @@ export const OrchestratorFilters: React.FC<OrchestratorFiltersProps> = ({
|
||||
<CardHeader className="pb-3">
|
||||
<CardTitle>Sponsor status</CardTitle>
|
||||
</CardHeader>
|
||||
<CardContent className="flex flex-wrap gap-2">
|
||||
{sponsorOptions.map((option) => (
|
||||
<Button
|
||||
key={option.value}
|
||||
type="button"
|
||||
size="sm"
|
||||
variant={
|
||||
sponsorFilter === option.value
|
||||
? "default"
|
||||
: "outline"
|
||||
}
|
||||
onClick={() => onSponsorFilterChange(option.value)}
|
||||
>
|
||||
{option.label}
|
||||
</Button>
|
||||
))}
|
||||
<CardContent className="space-y-4">
|
||||
<div className="flex flex-wrap gap-2">
|
||||
{sponsorOptions.map((option) => (
|
||||
<Button
|
||||
key={option.value}
|
||||
type="button"
|
||||
size="sm"
|
||||
variant={
|
||||
sponsorFilter === option.value
|
||||
? "default"
|
||||
: "outline"
|
||||
}
|
||||
onClick={() => onSponsorFilterChange(option.value)}
|
||||
>
|
||||
{option.label}
|
||||
</Button>
|
||||
))}
|
||||
</div>
|
||||
<div className="space-y-3">
|
||||
<div className="flex items-start gap-3 rounded-md border border-border/60 bg-muted/20 px-3 py-3">
|
||||
<Checkbox
|
||||
id="hide-sponsor-blockers"
|
||||
checked={hideSponsorBlockers}
|
||||
onCheckedChange={(checked) => {
|
||||
onHideSponsorBlockersChange(checked === true);
|
||||
}}
|
||||
/>
|
||||
<div className="space-y-1">
|
||||
<label
|
||||
htmlFor="hide-sponsor-blockers"
|
||||
className="text-sm font-medium leading-none cursor-pointer"
|
||||
>
|
||||
Hide red-flag roles
|
||||
</label>
|
||||
<p className="text-xs text-muted-foreground">
|
||||
Hides citizenship/GC/EAD-only, clearance, and
|
||||
unrestricted-work-auth posts. Yellow no-sponsor
|
||||
listings stay visible so you can ask about TN.
|
||||
</p>
|
||||
</div>
|
||||
</div>
|
||||
<div className="space-y-2">
|
||||
<p className="text-xs font-medium text-emerald-200/90">
|
||||
Green flags (TN-friendly)
|
||||
</p>
|
||||
<p className="text-xs text-muted-foreground">
|
||||
Show jobs matching any selected signal. Search{" "}
|
||||
<span className="font-mono">TN</span>,{" "}
|
||||
<span className="font-mono">sponsor</span>, or{" "}
|
||||
<span className="font-mono">
|
||||
authorized to work
|
||||
</span>{" "}
|
||||
in the job search bar too.
|
||||
</p>
|
||||
<div className="flex flex-wrap gap-2">
|
||||
{SPONSORSHIP_GREEN_FLAG_IDS.map((signalId) => {
|
||||
const selected =
|
||||
sponsorshipSignalsFilter.includes(signalId);
|
||||
const meta = SPONSORSHIP_SIGNAL_META[signalId];
|
||||
return (
|
||||
<Button
|
||||
key={signalId}
|
||||
type="button"
|
||||
size="sm"
|
||||
variant={selected ? "default" : "outline"}
|
||||
onClick={() => {
|
||||
const next = selected
|
||||
? sponsorshipSignalsFilter.filter(
|
||||
(value) => value !== signalId,
|
||||
)
|
||||
: [...sponsorshipSignalsFilter, signalId];
|
||||
onSponsorshipSignalsFilterChange(next);
|
||||
}}
|
||||
>
|
||||
{meta.label}
|
||||
</Button>
|
||||
);
|
||||
})}
|
||||
</div>
|
||||
</div>
|
||||
<div className="space-y-2">
|
||||
<p className="text-xs font-medium text-amber-200/90">
|
||||
Yellow flags (verify TN)
|
||||
</p>
|
||||
<p className="text-xs text-muted-foreground">
|
||||
Says no sponsorship — ask if TN support letter is
|
||||
available.
|
||||
</p>
|
||||
<div className="flex flex-wrap gap-2">
|
||||
{SPONSORSHIP_YELLOW_FLAG_IDS.map((signalId) => {
|
||||
const selected =
|
||||
sponsorshipSignalsFilter.includes(signalId);
|
||||
const meta = SPONSORSHIP_SIGNAL_META[signalId];
|
||||
return (
|
||||
<Button
|
||||
key={signalId}
|
||||
type="button"
|
||||
size="sm"
|
||||
variant={selected ? "secondary" : "outline"}
|
||||
className={
|
||||
selected
|
||||
? "border-amber-500/40 bg-amber-500/20 text-amber-100"
|
||||
: undefined
|
||||
}
|
||||
onClick={() => {
|
||||
const next = selected
|
||||
? sponsorshipSignalsFilter.filter(
|
||||
(value) => value !== signalId,
|
||||
)
|
||||
: [...sponsorshipSignalsFilter, signalId];
|
||||
onSponsorshipSignalsFilterChange(next);
|
||||
}}
|
||||
>
|
||||
{meta.label}
|
||||
</Button>
|
||||
);
|
||||
})}
|
||||
</div>
|
||||
</div>
|
||||
<div className="space-y-2">
|
||||
<p className="text-xs font-medium text-rose-200/90">
|
||||
Red flags (hard skip)
|
||||
</p>
|
||||
<p className="text-xs text-muted-foreground">
|
||||
Citizenship/GC/EAD only, clearance, or unrestricted
|
||||
work authorization — usually not TN-viable.
|
||||
</p>
|
||||
<div className="flex flex-wrap gap-2">
|
||||
{SPONSORSHIP_RED_FLAG_IDS.map((signalId) => {
|
||||
const selected =
|
||||
sponsorshipSignalsFilter.includes(signalId);
|
||||
const meta = SPONSORSHIP_SIGNAL_META[signalId];
|
||||
return (
|
||||
<Button
|
||||
key={signalId}
|
||||
type="button"
|
||||
size="sm"
|
||||
variant={selected ? "destructive" : "outline"}
|
||||
onClick={() => {
|
||||
const next = selected
|
||||
? sponsorshipSignalsFilter.filter(
|
||||
(value) => value !== signalId,
|
||||
)
|
||||
: [...sponsorshipSignalsFilter, signalId];
|
||||
onSponsorshipSignalsFilterChange(next);
|
||||
}}
|
||||
>
|
||||
{meta.label}
|
||||
</Button>
|
||||
);
|
||||
})}
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</CardContent>
|
||||
</Card>
|
||||
|
||||
|
||||
@@ -5,7 +5,7 @@ import type { SearchProfile } from "@shared/types";
|
||||
import { useMutation, useQuery, useQueryClient } from "@tanstack/react-query";
|
||||
import { Loader2 } from "lucide-react";
|
||||
import type React from "react";
|
||||
import { useMemo } from "react";
|
||||
import { useEffect, useMemo, useState } from "react";
|
||||
import { toast } from "sonner";
|
||||
import { Button } from "@/components/ui/button";
|
||||
import {
|
||||
@@ -27,17 +27,38 @@ export const ProfileQuickSwitch: React.FC = () => {
|
||||
const { data: profiles = [], isLoading } = useQuery<SearchProfile[]>({
|
||||
queryKey: queryKeys.searchProfiles.list(),
|
||||
queryFn: api.listProfiles,
|
||||
select: (rows) => {
|
||||
const seen = new Set<string>();
|
||||
return rows.filter((profile) => {
|
||||
if (seen.has(profile.id)) return false;
|
||||
seen.add(profile.id);
|
||||
return true;
|
||||
});
|
||||
},
|
||||
});
|
||||
|
||||
const [selectedId, setSelectedId] = useState<string | null>(null);
|
||||
|
||||
const activeId = useMemo(() => {
|
||||
if (selectedId && profiles.some((profile) => profile.id === selectedId)) {
|
||||
return selectedId;
|
||||
}
|
||||
const sid = settings?.activeProfileId?.trim() ?? "";
|
||||
if (sid && profiles.some((p) => p.id === sid)) return sid;
|
||||
if (sid && profiles.some((profile) => profile.id === sid)) return sid;
|
||||
return profiles[0]?.id ?? "";
|
||||
}, [selectedId, settings?.activeProfileId, profiles]);
|
||||
|
||||
useEffect(() => {
|
||||
const sid = settings?.activeProfileId?.trim() ?? "";
|
||||
if (sid && profiles.some((profile) => profile.id === sid)) {
|
||||
setSelectedId(sid);
|
||||
}
|
||||
}, [settings?.activeProfileId, profiles]);
|
||||
|
||||
const activateMutation = useMutation({
|
||||
mutationFn: (id: string) => api.activateProfile(id),
|
||||
onSuccess: async () => {
|
||||
onSuccess: async (_data, profileId) => {
|
||||
setSelectedId(profileId);
|
||||
await Promise.all([
|
||||
queryClient.invalidateQueries({ queryKey: queryKeys.jobs.all }),
|
||||
queryClient.invalidateQueries({ queryKey: queryKeys.profile.all }),
|
||||
@@ -46,7 +67,10 @@ export const ProfileQuickSwitch: React.FC = () => {
|
||||
}),
|
||||
]);
|
||||
await refreshSettings();
|
||||
toast.success("Search profile updated");
|
||||
const profileName =
|
||||
profiles.find((profile) => profile.id === profileId)?.name ??
|
||||
"Search profile";
|
||||
toast.success(`${profileName} active`);
|
||||
},
|
||||
onError: (err: unknown) => {
|
||||
const message =
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
import { describe, expect, it } from "vitest";
|
||||
import {
|
||||
AUTOMATIC_PRESETS,
|
||||
buildMaxCoverageDiscoveryLimits,
|
||||
calculateAutomaticEstimate,
|
||||
deriveExtractorLimits,
|
||||
mergeDiscoverySearchTerms,
|
||||
@@ -26,6 +27,7 @@ describe("automatic-run utilities", () => {
|
||||
topN: 10,
|
||||
minSuitabilityScore: 50,
|
||||
searchTerms: ["backend", "platform"],
|
||||
activeKeywordSetId: null,
|
||||
runBudget: 100,
|
||||
country: "united kingdom",
|
||||
cityLocations: [],
|
||||
@@ -71,6 +73,7 @@ describe("automatic-run utilities", () => {
|
||||
topN: 10,
|
||||
minSuitabilityScore: 50,
|
||||
searchTerms: [],
|
||||
activeKeywordSetId: null,
|
||||
runBudget: 750,
|
||||
country: "united kingdom",
|
||||
cityLocations: [],
|
||||
@@ -112,6 +115,7 @@ describe("automatic-run utilities", () => {
|
||||
topN: 10,
|
||||
minSuitabilityScore: 50,
|
||||
searchTerms: ["backend", "platform"],
|
||||
activeKeywordSetId: null,
|
||||
runBudget: 120,
|
||||
country: "united kingdom",
|
||||
cityLocations: [],
|
||||
@@ -130,6 +134,7 @@ describe("automatic-run utilities", () => {
|
||||
topN: 10,
|
||||
minSuitabilityScore: 50,
|
||||
searchTerms: ["backend", "platform"],
|
||||
activeKeywordSetId: null,
|
||||
runBudget: 120,
|
||||
country: "united kingdom",
|
||||
cityLocations: [],
|
||||
@@ -148,6 +153,7 @@ describe("automatic-run utilities", () => {
|
||||
topN: 10,
|
||||
minSuitabilityScore: 50,
|
||||
searchTerms: ["backend", "platform"],
|
||||
activeKeywordSetId: null,
|
||||
runBudget: 120,
|
||||
country: "united kingdom",
|
||||
cityLocations: [],
|
||||
@@ -159,4 +165,16 @@ describe("automatic-run utilities", () => {
|
||||
expect(estimate.discovered.cap).toBeGreaterThan(0);
|
||||
expect(estimate.discovered.cap).toBeLessThanOrEqual(120);
|
||||
});
|
||||
|
||||
it("raises discovery caps for max coverage runs", () => {
|
||||
const limits = buildMaxCoverageDiscoveryLimits({
|
||||
budget: 500,
|
||||
searchTerms: ["qa engineer"],
|
||||
sources: ["workingnomads", "testdevjobs", "indeed"],
|
||||
});
|
||||
|
||||
expect(limits.workingnomadsMaxJobsPerTerm).toBeGreaterThanOrEqual(100);
|
||||
expect(limits.testdevjobsMaxPages).toBe(10);
|
||||
expect(limits.jobspyResultsWanted).toBeGreaterThanOrEqual(100);
|
||||
});
|
||||
});
|
||||
|
||||
@@ -2,6 +2,7 @@ import {
|
||||
parseSearchCitiesSetting,
|
||||
serializeSearchCitiesSetting,
|
||||
} from "@shared/search-cities.js";
|
||||
import type { UpdateSettingsInput } from "@shared/settings-schema.js";
|
||||
import type { JobSource } from "@shared/types";
|
||||
|
||||
/**
|
||||
@@ -39,6 +40,7 @@ export interface AutomaticRunValues {
|
||||
topN: number;
|
||||
minSuitabilityScore: number;
|
||||
searchTerms: string[];
|
||||
activeKeywordSetId: string | null;
|
||||
runBudget: number;
|
||||
country: string;
|
||||
cityLocations: string[];
|
||||
@@ -80,7 +82,7 @@ export const AUTOMATIC_PRESETS: Record<
|
||||
detailed: {
|
||||
topN: 20,
|
||||
minSuitabilityScore: 35,
|
||||
runBudget: 750,
|
||||
runBudget: 1000,
|
||||
},
|
||||
};
|
||||
|
||||
@@ -163,6 +165,51 @@ export function deriveExtractorLimits(args: {
|
||||
};
|
||||
}
|
||||
|
||||
/** Raise per-source caps before a pipeline run for maximum discovery coverage. */
|
||||
export function buildMaxCoverageDiscoveryLimits(args: {
|
||||
budget: number;
|
||||
searchTerms: string[];
|
||||
sources: JobSource[];
|
||||
}): Partial<UpdateSettingsInput> {
|
||||
const limits = deriveExtractorLimits(args);
|
||||
const perTermCap = Math.max(100, limits.jobspyResultsWanted);
|
||||
const boardCap = Math.max(150, perTermCap);
|
||||
|
||||
return {
|
||||
jobspyResultsWanted: perTermCap,
|
||||
gradcrackerMaxJobsPerTerm: boardCap,
|
||||
ukvisajobsMaxJobs: boardCap,
|
||||
adzunaMaxJobsPerTerm: boardCap,
|
||||
startupjobsMaxJobsPerTerm: boardCap,
|
||||
usajobsMaxJobsPerTerm: boardCap,
|
||||
jobicyMaxJobsPerTerm: boardCap,
|
||||
themuseMaxJobsPerTerm: boardCap,
|
||||
joobleMaxJobsPerTerm: boardCap,
|
||||
careerjetMaxJobsPerTerm: boardCap,
|
||||
reedMaxJobsPerTerm: boardCap,
|
||||
remoteokMaxJobsPerTerm: boardCap,
|
||||
remotiveMaxJobsPerTerm: boardCap,
|
||||
arbeitnowMaxJobsPerTerm: boardCap,
|
||||
himalayasMaxJobsPerTerm: boardCap,
|
||||
weworkremotelyMaxJobsPerTerm: boardCap,
|
||||
workingnomadsMaxJobsPerTerm: boardCap,
|
||||
fourdayweekMaxJobsPerTerm: boardCap,
|
||||
testdevjobsMaxJobsPerTerm: boardCap,
|
||||
testdevjobsMaxPages: 10,
|
||||
builtinMaxJobsPerTerm: boardCap,
|
||||
builtinMaxPagesPerTerm: 10,
|
||||
wellfoundMaxJobsPerTerm: Math.min(100, boardCap),
|
||||
googleJobsMaxJobsPerTerm: Math.min(30, boardCap),
|
||||
qajobsboardMaxJobsPerTerm: boardCap,
|
||||
arcMaxJobsPerPath: boardCap,
|
||||
smartrecruitersMaxJobsPerCompany: boardCap,
|
||||
elutaMaxJobsPerTerm: boardCap,
|
||||
bctenetMaxJobsPerTerm: 400,
|
||||
icimsMaxJobsPerTenant: 250,
|
||||
icimsMaxPagesPerSearch: 10,
|
||||
};
|
||||
}
|
||||
|
||||
export function parseSearchTermsInput(input: string): string[] {
|
||||
return input
|
||||
.split(/[\n,]/g)
|
||||
|
||||
@@ -3,13 +3,26 @@ import {
|
||||
EXTRACTOR_SOURCE_METADATA,
|
||||
PIPELINE_EXTRACTOR_SOURCE_IDS,
|
||||
} from "@shared/extractors";
|
||||
import {
|
||||
SPONSORSHIP_SIGNAL_IDS,
|
||||
type SponsorshipSignalId,
|
||||
} from "@shared/sponsorship-signals";
|
||||
import type { JobSource, JobStatus } from "@shared/types";
|
||||
|
||||
export type { SponsorshipSignalId };
|
||||
export const sponsorshipSignalFilterOptions = [
|
||||
...SPONSORSHIP_SIGNAL_IDS,
|
||||
] as const;
|
||||
|
||||
/** Defaults aligned with the live cron set on jobs.levkin.ca. */
|
||||
export const DEFAULT_PIPELINE_SOURCES: JobSource[] = [
|
||||
"gradcracker",
|
||||
"indeed",
|
||||
"linkedin",
|
||||
"ukvisajobs",
|
||||
"indeed",
|
||||
"glassdoor",
|
||||
"qajobsboard",
|
||||
"arcdev",
|
||||
"eluta",
|
||||
"bctenet",
|
||||
];
|
||||
export const PIPELINE_SOURCES_STORAGE_KEY = "jobops.pipeline.sources";
|
||||
|
||||
|
||||
@@ -1,3 +1,4 @@
|
||||
import { serializeSponsorshipSignals } from "@shared/sponsorship-signals";
|
||||
import { createJob } from "@shared/testing/factories";
|
||||
import type { Job } from "@shared/types";
|
||||
import { renderHook } from "@testing-library/react";
|
||||
@@ -36,6 +37,8 @@ describe("useFilteredJobs", () => {
|
||||
[],
|
||||
[],
|
||||
"all",
|
||||
[],
|
||||
false,
|
||||
"all",
|
||||
{ mode: "at_least", min: null, max: null },
|
||||
{
|
||||
@@ -67,6 +70,8 @@ describe("useFilteredJobs", () => {
|
||||
[],
|
||||
[],
|
||||
"all",
|
||||
[],
|
||||
false,
|
||||
"all",
|
||||
{ mode: "at_least", min: null, max: null },
|
||||
{
|
||||
@@ -99,6 +104,8 @@ describe("useFilteredJobs", () => {
|
||||
[],
|
||||
[],
|
||||
"confirmed",
|
||||
[],
|
||||
false,
|
||||
"all",
|
||||
{ mode: "at_least", min: null, max: null },
|
||||
{
|
||||
@@ -111,6 +118,90 @@ describe("useFilteredJobs", () => {
|
||||
expect(result.current.map((job) => job.id)).toEqual(["confirmed"]);
|
||||
});
|
||||
|
||||
it("filters by sponsorship text signals", () => {
|
||||
const jobs: Job[] = [
|
||||
{
|
||||
...baseJob,
|
||||
id: "tn",
|
||||
sponsorshipSignals: serializeSponsorshipSignals(["tn_eligible"]),
|
||||
},
|
||||
{
|
||||
...baseJob,
|
||||
id: "sponsor",
|
||||
sponsorshipSignals: serializeSponsorshipSignals(["will_sponsor"]),
|
||||
},
|
||||
{ ...baseJob, id: "none", sponsorshipSignals: null },
|
||||
];
|
||||
|
||||
const { result } = renderHook(() =>
|
||||
useFilteredJobs(
|
||||
jobs,
|
||||
"all",
|
||||
[],
|
||||
[],
|
||||
[],
|
||||
[],
|
||||
"all",
|
||||
["tn_eligible", "will_sponsor"],
|
||||
false,
|
||||
"all",
|
||||
{ mode: "at_least", min: null, max: null },
|
||||
{ key: "score", direction: "desc" },
|
||||
),
|
||||
);
|
||||
|
||||
expect(result.current.map((job) => job.id).sort()).toEqual([
|
||||
"sponsor",
|
||||
"tn",
|
||||
]);
|
||||
});
|
||||
|
||||
it("hides red-flag jobs but keeps yellow no-sponsor listings when enabled", () => {
|
||||
const jobs: Job[] = [
|
||||
{
|
||||
...baseJob,
|
||||
id: "red",
|
||||
sponsorshipSignals: serializeSponsorshipSignals([
|
||||
"citizenship_required",
|
||||
]),
|
||||
},
|
||||
{
|
||||
...baseJob,
|
||||
id: "yellow",
|
||||
sponsorshipSignals: serializeSponsorshipSignals([
|
||||
"no_sponsorship_stated",
|
||||
]),
|
||||
},
|
||||
{
|
||||
...baseJob,
|
||||
id: "tn",
|
||||
sponsorshipSignals: serializeSponsorshipSignals(["tn_eligible"]),
|
||||
},
|
||||
];
|
||||
|
||||
const { result } = renderHook(() =>
|
||||
useFilteredJobs(
|
||||
jobs,
|
||||
"all",
|
||||
[],
|
||||
[],
|
||||
[],
|
||||
[],
|
||||
"all",
|
||||
[],
|
||||
true,
|
||||
"all",
|
||||
{ mode: "at_least", min: null, max: null },
|
||||
{ key: "score", direction: "desc" },
|
||||
),
|
||||
);
|
||||
|
||||
expect(result.current.map((job) => job.id).sort()).toEqual([
|
||||
"tn",
|
||||
"yellow",
|
||||
]);
|
||||
});
|
||||
|
||||
it("filters by salary range using structured and text salary fields", () => {
|
||||
const jobs: Job[] = [
|
||||
{ ...baseJob, id: "structured", salaryMinAmount: 70000 },
|
||||
@@ -128,6 +219,8 @@ describe("useFilteredJobs", () => {
|
||||
[],
|
||||
[],
|
||||
"all",
|
||||
[],
|
||||
false,
|
||||
"all",
|
||||
{ mode: "between", min: 60000, max: 80000 },
|
||||
{
|
||||
@@ -160,6 +253,8 @@ describe("useFilteredJobs", () => {
|
||||
[],
|
||||
[],
|
||||
"all",
|
||||
[],
|
||||
false,
|
||||
"all",
|
||||
{ mode: "at_least", min: null, max: null },
|
||||
{
|
||||
@@ -193,6 +288,8 @@ describe("useFilteredJobs", () => {
|
||||
[],
|
||||
[],
|
||||
"all",
|
||||
[],
|
||||
false,
|
||||
"remote",
|
||||
{ mode: "at_least", min: null, max: null },
|
||||
{ key: "score", direction: "desc" },
|
||||
@@ -209,6 +306,8 @@ describe("useFilteredJobs", () => {
|
||||
[],
|
||||
[],
|
||||
"all",
|
||||
[],
|
||||
false,
|
||||
"not_remote",
|
||||
{ mode: "at_least", min: null, max: null },
|
||||
{ key: "score", direction: "desc" },
|
||||
@@ -225,6 +324,8 @@ describe("useFilteredJobs", () => {
|
||||
[],
|
||||
[],
|
||||
"all",
|
||||
[],
|
||||
false,
|
||||
"unknown",
|
||||
{ mode: "at_least", min: null, max: null },
|
||||
{ key: "score", direction: "desc" },
|
||||
@@ -249,6 +350,8 @@ describe("useFilteredJobs", () => {
|
||||
[],
|
||||
[],
|
||||
"all",
|
||||
[],
|
||||
false,
|
||||
"all",
|
||||
{ mode: "at_least", min: null, max: null },
|
||||
{ key: "score", direction: "desc" },
|
||||
@@ -288,6 +391,8 @@ describe("useFilteredJobs", () => {
|
||||
["united kingdom"],
|
||||
[],
|
||||
"all",
|
||||
[],
|
||||
false,
|
||||
"all",
|
||||
{ mode: "at_least", min: null, max: null },
|
||||
{ key: "score", direction: "desc" },
|
||||
@@ -312,6 +417,8 @@ describe("useFilteredJobs", () => {
|
||||
[],
|
||||
[],
|
||||
"all",
|
||||
[],
|
||||
false,
|
||||
"all",
|
||||
{ mode: "at_least", min: null, max: null },
|
||||
{ key: "score", direction: "desc" },
|
||||
@@ -359,6 +466,8 @@ describe("useFilteredJobs", () => {
|
||||
[],
|
||||
["united kingdom"],
|
||||
"all",
|
||||
[],
|
||||
false,
|
||||
"all",
|
||||
{ mode: "at_least", min: null, max: null },
|
||||
{ key: "score", direction: "desc" },
|
||||
@@ -381,6 +490,8 @@ describe("useFilteredJobs", () => {
|
||||
[],
|
||||
[],
|
||||
"all",
|
||||
[],
|
||||
false,
|
||||
"all",
|
||||
{ mode: "at_least", min: null, max: null },
|
||||
{ key: "score", direction: "desc" },
|
||||
@@ -412,6 +523,8 @@ describe("useFilteredJobs", () => {
|
||||
[],
|
||||
[],
|
||||
"all",
|
||||
[],
|
||||
false,
|
||||
"all",
|
||||
{ mode: "at_least", min: null, max: null },
|
||||
{ key: "score", direction: "desc" },
|
||||
@@ -443,6 +556,8 @@ describe("useFilteredJobs", () => {
|
||||
[],
|
||||
[],
|
||||
"all",
|
||||
[],
|
||||
false,
|
||||
"all",
|
||||
{ mode: "at_least", min: null, max: null },
|
||||
{ key: "score", direction: "desc" },
|
||||
@@ -466,6 +581,8 @@ describe("useFilteredJobs", () => {
|
||||
[],
|
||||
[],
|
||||
"all",
|
||||
[],
|
||||
false,
|
||||
"all",
|
||||
{ mode: "at_least", min: null, max: null },
|
||||
{ key: "score", direction: "desc" },
|
||||
@@ -497,6 +614,8 @@ describe("useFilteredJobs", () => {
|
||||
[],
|
||||
[],
|
||||
"all",
|
||||
[],
|
||||
false,
|
||||
"all",
|
||||
{ mode: "at_least", min: null, max: null },
|
||||
{ key: "score", direction: "desc" },
|
||||
@@ -520,6 +639,8 @@ describe("useFilteredJobs", () => {
|
||||
[],
|
||||
[],
|
||||
"all",
|
||||
[],
|
||||
false,
|
||||
"all",
|
||||
{ mode: "at_least", min: null, max: null },
|
||||
{ key: "score", direction: "desc" },
|
||||
|
||||
@@ -1,6 +1,11 @@
|
||||
import type { DuplicateDismissReason } from "@client/lib/job-dedup";
|
||||
import { jobMatchesAllowedCountry } from "@shared/blocked-countries";
|
||||
import { textMatchesKeyword } from "@shared/keyword-match";
|
||||
import type { SponsorshipSignalId } from "@shared/sponsorship-signals";
|
||||
import {
|
||||
jobHasAnySponsorshipSignal,
|
||||
jobHasRedSponsorshipFlag,
|
||||
} from "@shared/sponsorship-signals";
|
||||
import type { JobListItem, JobSource } from "@shared/types";
|
||||
import { useMemo } from "react";
|
||||
import type {
|
||||
@@ -57,6 +62,8 @@ export const useFilteredJobs = (
|
||||
countriesFilter: string[],
|
||||
countriesExcludeFilter: string[],
|
||||
sponsorFilter: SponsorFilter,
|
||||
sponsorshipSignalsFilter: SponsorshipSignalId[],
|
||||
hideSponsorBlockers: boolean,
|
||||
workplaceFilter: WorkplaceFilter,
|
||||
salaryFilter: SalaryFilter,
|
||||
sort: JobSort,
|
||||
@@ -154,6 +161,21 @@ export const useFilteredJobs = (
|
||||
);
|
||||
}
|
||||
|
||||
if (sponsorshipSignalsFilter.length > 0) {
|
||||
filtered = filtered.filter((job) =>
|
||||
jobHasAnySponsorshipSignal(
|
||||
job.sponsorshipSignals,
|
||||
sponsorshipSignalsFilter,
|
||||
),
|
||||
);
|
||||
}
|
||||
|
||||
if (hideSponsorBlockers) {
|
||||
filtered = filtered.filter(
|
||||
(job) => !jobHasRedSponsorshipFlag(job.sponsorshipSignals),
|
||||
);
|
||||
}
|
||||
|
||||
if (workplaceFilter !== "all") {
|
||||
filtered = filtered.filter((job) => {
|
||||
if (workplaceFilter === "remote") return job.isRemote === true;
|
||||
@@ -252,6 +274,8 @@ export const useFilteredJobs = (
|
||||
countriesFilter,
|
||||
countriesExcludeFilter,
|
||||
sponsorFilter,
|
||||
sponsorshipSignalsFilter,
|
||||
hideSponsorBlockers,
|
||||
workplaceFilter,
|
||||
salaryFilter,
|
||||
sort,
|
||||
|
||||
@@ -635,6 +635,7 @@ describe("useOrchestratorData", () => {
|
||||
status: "discovered",
|
||||
suitabilityScore: null,
|
||||
sponsorMatchScore: null,
|
||||
sponsorshipSignals: null,
|
||||
jobType: null,
|
||||
jobFunction: null,
|
||||
salaryMinAmount: null,
|
||||
|
||||
@@ -3,6 +3,7 @@ import {
|
||||
normalizeCountryKey,
|
||||
SUPPORTED_COUNTRY_KEYS,
|
||||
} from "@shared/location-support";
|
||||
import type { SponsorshipSignalId } from "@shared/sponsorship-signals";
|
||||
import type { JobSource } from "@shared/types";
|
||||
import { useCallback, useEffect, useMemo } from "react";
|
||||
import { useSearchParams } from "react-router-dom";
|
||||
@@ -13,7 +14,7 @@ import type {
|
||||
SponsorFilter,
|
||||
WorkplaceFilter,
|
||||
} from "./constants";
|
||||
import { DEFAULT_SORT } from "./constants";
|
||||
import { DEFAULT_SORT, sponsorshipSignalFilterOptions } from "./constants";
|
||||
|
||||
const allowedSponsorFilters: SponsorFilter[] = [
|
||||
"all",
|
||||
@@ -43,6 +44,10 @@ const allowedWorkplaceFilters: WorkplaceFilter[] = [
|
||||
"unknown",
|
||||
];
|
||||
|
||||
const allowedSponsorshipSignals = new Set<string>(
|
||||
sponsorshipSignalFilterOptions,
|
||||
);
|
||||
|
||||
const allowedJobSources = new Set<string>(EXTRACTOR_SOURCE_IDS);
|
||||
const allowedCountryKeys = new Set(SUPPORTED_COUNTRY_KEYS);
|
||||
|
||||
@@ -214,6 +219,58 @@ export const useOrchestratorFilters = () => {
|
||||
: "all";
|
||||
}, [searchParams]);
|
||||
|
||||
const sponsorshipSignalsFilter = useMemo((): SponsorshipSignalId[] => {
|
||||
const raw = searchParams.getAll("sponsorSignal");
|
||||
const out: SponsorshipSignalId[] = [];
|
||||
const seen = new Set<string>();
|
||||
for (const value of raw) {
|
||||
const trimmed = value.trim();
|
||||
if (!trimmed || !allowedSponsorshipSignals.has(trimmed)) continue;
|
||||
if (seen.has(trimmed)) continue;
|
||||
seen.add(trimmed);
|
||||
out.push(trimmed as SponsorshipSignalId);
|
||||
}
|
||||
return out;
|
||||
}, [searchParams]);
|
||||
|
||||
const hideSponsorBlockers = useMemo(
|
||||
() => searchParams.get("hideSponsorBlockers") === "1",
|
||||
[searchParams],
|
||||
);
|
||||
|
||||
const setHideSponsorBlockers = useCallback(
|
||||
(value: boolean) => {
|
||||
setSearchParams(
|
||||
(prev) => {
|
||||
if (value) prev.set("hideSponsorBlockers", "1");
|
||||
else prev.delete("hideSponsorBlockers");
|
||||
return prev;
|
||||
},
|
||||
{ replace: true },
|
||||
);
|
||||
},
|
||||
[setSearchParams],
|
||||
);
|
||||
|
||||
const setSponsorshipSignalsFilter = useCallback(
|
||||
(values: SponsorshipSignalId[]) => {
|
||||
setSearchParams(
|
||||
(prev) => {
|
||||
prev.delete("sponsorSignal");
|
||||
const unique = [
|
||||
...new Set(
|
||||
values.filter((value) => allowedSponsorshipSignals.has(value)),
|
||||
),
|
||||
];
|
||||
for (const value of unique) prev.append("sponsorSignal", value);
|
||||
return prev;
|
||||
},
|
||||
{ replace: true },
|
||||
);
|
||||
},
|
||||
[setSearchParams],
|
||||
);
|
||||
|
||||
const setSponsorFilter = useCallback(
|
||||
(value: SponsorFilter) => {
|
||||
setSearchParams(
|
||||
@@ -378,6 +435,11 @@ export const useOrchestratorFilters = () => {
|
||||
[searchParams],
|
||||
);
|
||||
|
||||
const hidePriorSkips = useMemo(
|
||||
() => searchParams.get("hidePriorSkips") === "1",
|
||||
[searchParams],
|
||||
);
|
||||
|
||||
const setApplySettingsCompanySkipList = useCallback(
|
||||
(value: boolean) => {
|
||||
setSearchParams(
|
||||
@@ -392,6 +454,20 @@ export const useOrchestratorFilters = () => {
|
||||
[setSearchParams],
|
||||
);
|
||||
|
||||
const setHidePriorSkips = useCallback(
|
||||
(value: boolean) => {
|
||||
setSearchParams(
|
||||
(prev) => {
|
||||
if (value) prev.set("hidePriorSkips", "1");
|
||||
else prev.delete("hidePriorSkips");
|
||||
return prev;
|
||||
},
|
||||
{ replace: true },
|
||||
);
|
||||
},
|
||||
[setSearchParams],
|
||||
);
|
||||
|
||||
const sort = useMemo((): JobSort => {
|
||||
const sortValue = searchParams.get("sort");
|
||||
if (!sortValue) return DEFAULT_SORT;
|
||||
@@ -438,6 +514,8 @@ export const useOrchestratorFilters = () => {
|
||||
prev.delete("countries");
|
||||
prev.delete("countriesExclude");
|
||||
prev.delete("sponsor");
|
||||
prev.delete("sponsorSignal");
|
||||
prev.delete("hideSponsorBlockers");
|
||||
prev.delete("workplace");
|
||||
prev.delete("salaryMode");
|
||||
prev.delete("salaryMin");
|
||||
@@ -449,6 +527,7 @@ export const useOrchestratorFilters = () => {
|
||||
prev.delete("employer");
|
||||
prev.delete("employerExclude");
|
||||
prev.delete("skipList");
|
||||
prev.delete("hidePriorSkips");
|
||||
return prev;
|
||||
},
|
||||
{ replace: true },
|
||||
@@ -465,6 +544,10 @@ export const useOrchestratorFilters = () => {
|
||||
setCountrySelection,
|
||||
sponsorFilter,
|
||||
setSponsorFilter,
|
||||
sponsorshipSignalsFilter,
|
||||
setSponsorshipSignalsFilter,
|
||||
hideSponsorBlockers,
|
||||
setHideSponsorBlockers,
|
||||
workplaceFilter,
|
||||
setWorkplaceFilter,
|
||||
salaryFilter,
|
||||
@@ -477,6 +560,8 @@ export const useOrchestratorFilters = () => {
|
||||
setEmployerFilterTokens,
|
||||
applySettingsCompanySkipList,
|
||||
setApplySettingsCompanySkipList,
|
||||
hidePriorSkips,
|
||||
setHidePriorSkips,
|
||||
sort,
|
||||
setSort,
|
||||
resetFilters,
|
||||
|
||||
@@ -10,7 +10,7 @@ import { toast } from "sonner";
|
||||
import { trackProductEvent } from "@/lib/analytics";
|
||||
import type { AutomaticRunValues } from "./automatic-run";
|
||||
import {
|
||||
deriveExtractorLimits,
|
||||
buildMaxCoverageDiscoveryLimits,
|
||||
mergeDiscoverySearchTerms,
|
||||
serializeCityLocationsSetting,
|
||||
} from "./automatic-run";
|
||||
@@ -169,10 +169,12 @@ export function usePipelineControls(
|
||||
return;
|
||||
}
|
||||
|
||||
const mergedSearchTerms = mergeDiscoverySearchTerms(
|
||||
values.searchTerms,
|
||||
settings?.jobSearchProfile?.value?.targetRoles,
|
||||
);
|
||||
const mergedSearchTerms = values.activeKeywordSetId
|
||||
? values.searchTerms.map((t) => t.trim()).filter(Boolean)
|
||||
: mergeDiscoverySearchTerms(
|
||||
values.searchTerms,
|
||||
settings?.jobSearchProfile?.value?.targetRoles,
|
||||
);
|
||||
if (mergedSearchTerms.length === 0) {
|
||||
toast.error(
|
||||
"Add at least one search term, or set target roles on your job search profile.",
|
||||
@@ -180,11 +182,6 @@ export function usePipelineControls(
|
||||
return;
|
||||
}
|
||||
|
||||
const limits = deriveExtractorLimits({
|
||||
budget: values.runBudget,
|
||||
searchTerms: mergedSearchTerms,
|
||||
sources: compatibleSources,
|
||||
});
|
||||
const hasJobSpySite = compatibleSources.some(
|
||||
(source) =>
|
||||
source === "indeed" ||
|
||||
@@ -202,14 +199,19 @@ export function usePipelineControls(
|
||||
serializedCities
|
||||
? serializedCities
|
||||
: formatCountryLabel(values.country);
|
||||
if (values.activeKeywordSetId) {
|
||||
await api.updateKeywordSet(values.activeKeywordSetId, {
|
||||
terms: values.searchTerms,
|
||||
});
|
||||
}
|
||||
await api.updateSettings({
|
||||
searchTerms: values.searchTerms,
|
||||
workplaceTypes: values.workplaceTypes,
|
||||
jobspyResultsWanted: limits.jobspyResultsWanted,
|
||||
gradcrackerMaxJobsPerTerm: limits.gradcrackerMaxJobsPerTerm,
|
||||
ukvisajobsMaxJobs: limits.ukvisajobsMaxJobs,
|
||||
adzunaMaxJobsPerTerm: limits.adzunaMaxJobsPerTerm,
|
||||
startupjobsMaxJobsPerTerm: limits.startupjobsMaxJobsPerTerm,
|
||||
...buildMaxCoverageDiscoveryLimits({
|
||||
budget: values.runBudget,
|
||||
searchTerms: mergedSearchTerms,
|
||||
sources: compatibleSources,
|
||||
}),
|
||||
jobspyCountryIndeed: values.country,
|
||||
searchCities,
|
||||
});
|
||||
|
||||
@@ -67,7 +67,36 @@ describe("usePipelineSources", () => {
|
||||
expect(result.current.pipelineSources).toEqual(["gradcracker"]);
|
||||
});
|
||||
|
||||
it("falls back to the first enabled source", () => {
|
||||
it("merges newly enabled sources into stored selection", () => {
|
||||
ensureStorage().setItem(
|
||||
PIPELINE_SOURCES_STORAGE_KEY,
|
||||
JSON.stringify(["gradcracker"]),
|
||||
);
|
||||
|
||||
const enabledSources = [
|
||||
"gradcracker",
|
||||
"linkedin",
|
||||
"workingnomads",
|
||||
] as const;
|
||||
|
||||
const { result } = renderHook(() => usePipelineSources(enabledSources));
|
||||
|
||||
expect(result.current.pipelineSources).toEqual([
|
||||
"gradcracker",
|
||||
"linkedin",
|
||||
"workingnomads",
|
||||
]);
|
||||
});
|
||||
|
||||
it("selects all enabled sources when storage is empty", () => {
|
||||
const enabledSources = ["gradcracker", "linkedin"] as const;
|
||||
|
||||
const { result } = renderHook(() => usePipelineSources(enabledSources));
|
||||
|
||||
expect(result.current.pipelineSources).toEqual(["gradcracker", "linkedin"]);
|
||||
});
|
||||
|
||||
it("falls back to all enabled sources when stored sources are unavailable", () => {
|
||||
ensureStorage().setItem(
|
||||
PIPELINE_SOURCES_STORAGE_KEY,
|
||||
JSON.stringify(["ukvisajobs"]),
|
||||
@@ -77,7 +106,7 @@ describe("usePipelineSources", () => {
|
||||
|
||||
const { result } = renderHook(() => usePipelineSources(enabledSources));
|
||||
|
||||
expect(result.current.pipelineSources).toEqual(["gradcracker"]);
|
||||
expect(result.current.pipelineSources).toEqual(["gradcracker", "linkedin"]);
|
||||
});
|
||||
|
||||
it("ignores toggles for disabled sources", () => {
|
||||
|
||||
@@ -11,12 +11,14 @@ const resolveAllowedSources = (enabledSources?: readonly JobSource[]) =>
|
||||
? (enabledSources as JobSource[])
|
||||
: DEFAULT_PIPELINE_SOURCES;
|
||||
|
||||
const normalizeSources = (
|
||||
const mergeAllowedSources = (
|
||||
sources: JobSource[],
|
||||
allowedSources: JobSource[],
|
||||
) => {
|
||||
const filtered = sources.filter((value) => allowedSources.includes(value));
|
||||
return filtered.length > 0 ? filtered : allowedSources.slice(0, 1);
|
||||
): JobSource[] => {
|
||||
const kept = sources.filter((value) => allowedSources.includes(value));
|
||||
const missing = allowedSources.filter((value) => !kept.includes(value));
|
||||
const merged = [...kept, ...missing];
|
||||
return merged.length > 0 ? merged : [...allowedSources];
|
||||
};
|
||||
|
||||
const sourcesMatch = (left: JobSource[], right: JobSource[]) =>
|
||||
@@ -31,22 +33,22 @@ export const usePipelineSources = (enabledSources?: readonly JobSource[]) => {
|
||||
const [pipelineSources, setPipelineSources] = useState<JobSource[]>(() => {
|
||||
try {
|
||||
const raw = localStorage.getItem(PIPELINE_SOURCES_STORAGE_KEY);
|
||||
if (!raw) return normalizeSources(allowedSources, allowedSources);
|
||||
if (!raw) return mergeAllowedSources([], allowedSources);
|
||||
const parsed = JSON.parse(raw) as unknown;
|
||||
if (!Array.isArray(parsed))
|
||||
return normalizeSources(allowedSources, allowedSources);
|
||||
return mergeAllowedSources([], allowedSources);
|
||||
const next = parsed.filter((value): value is JobSource =>
|
||||
orderedSources.includes(value as JobSource),
|
||||
);
|
||||
return normalizeSources(next, allowedSources);
|
||||
return mergeAllowedSources(next, allowedSources);
|
||||
} catch {
|
||||
return normalizeSources(allowedSources, allowedSources);
|
||||
return mergeAllowedSources([], allowedSources);
|
||||
}
|
||||
});
|
||||
|
||||
useEffect(() => {
|
||||
setPipelineSources((current) => {
|
||||
const normalized = normalizeSources(current, allowedSources);
|
||||
const normalized = mergeAllowedSources(current, allowedSources);
|
||||
return sourcesMatch(current, normalized) ? current : normalized;
|
||||
});
|
||||
}, [allowedSources]);
|
||||
|
||||
@@ -27,6 +27,38 @@ describe("orchestrator utils", () => {
|
||||
expect(enabled).toContain("themuse");
|
||||
});
|
||||
|
||||
it("enables new public job boards without credentials", () => {
|
||||
const enabled = getEnabledSources(createAppSettings());
|
||||
for (const source of [
|
||||
"workingnomads",
|
||||
"testdevjobs",
|
||||
"wellfound",
|
||||
"builtin",
|
||||
"google-jobs",
|
||||
] as const) {
|
||||
expect(enabled).toContain(source);
|
||||
}
|
||||
});
|
||||
|
||||
it("enables ATS sources when company lists are configured", () => {
|
||||
const enabled = getEnabledSources(
|
||||
createAppSettings({
|
||||
teamtailorCompanies: {
|
||||
value: ["testgorilla"],
|
||||
default: [],
|
||||
override: null,
|
||||
},
|
||||
careersPageUrls: {
|
||||
value: ["https://sentry.io/careers/"],
|
||||
default: [],
|
||||
override: null,
|
||||
},
|
||||
}),
|
||||
);
|
||||
expect(enabled).toContain("teamtailor");
|
||||
expect(enabled).toContain("careerspages");
|
||||
});
|
||||
|
||||
it("counts processing jobs in ready and discovered tabs", () => {
|
||||
const jobs = [
|
||||
createJob({ id: "ready", status: "ready", closedAt: null }),
|
||||
|
||||
@@ -212,6 +212,12 @@ export const getEnabledSources = (
|
||||
const hasWorkdayTenants = (settings.workdayTenants?.value ?? []).length > 0;
|
||||
const hasSmartrecruitersCompanies =
|
||||
(settings.smartrecruitersCompanies?.value ?? []).length > 0;
|
||||
const hasTeamtailorCompanies =
|
||||
(settings.teamtailorCompanies?.value ?? []).length > 0;
|
||||
const hasHuntflowTenants = (settings.huntflowTenants?.value ?? []).length > 0;
|
||||
const hasFactorialTenants =
|
||||
(settings.factorialTenants?.value ?? []).length > 0;
|
||||
const hasCareersPageUrls = (settings.careersPageUrls?.value ?? []).length > 0;
|
||||
const hasElutaRssLocations =
|
||||
(settings.elutaRssLocations?.value ?? []).length > 0;
|
||||
const hasIcimsTenants = (settings.icimsTenants?.value ?? []).length > 0;
|
||||
@@ -281,6 +287,22 @@ export const getEnabledSources = (
|
||||
if (hasSmartrecruitersCompanies) enabled.push(source);
|
||||
continue;
|
||||
}
|
||||
if (source === "teamtailor") {
|
||||
if (hasTeamtailorCompanies) enabled.push(source);
|
||||
continue;
|
||||
}
|
||||
if (source === "huntflow") {
|
||||
if (hasHuntflowTenants) enabled.push(source);
|
||||
continue;
|
||||
}
|
||||
if (source === "factorial") {
|
||||
if (hasFactorialTenants) enabled.push(source);
|
||||
continue;
|
||||
}
|
||||
if (source === "careerspages") {
|
||||
if (hasCareersPageUrls) enabled.push(source);
|
||||
continue;
|
||||
}
|
||||
if (source === "icims") {
|
||||
if (hasIcimsTenants) enabled.push(source);
|
||||
continue;
|
||||
@@ -307,7 +329,12 @@ export const getEnabledSources = (
|
||||
source === "arbeitnow" ||
|
||||
source === "himalayas" ||
|
||||
source === "weworkremotely" ||
|
||||
source === "workingnomads" ||
|
||||
source === "fourdayweek" ||
|
||||
source === "testdevjobs" ||
|
||||
source === "wellfound" ||
|
||||
source === "builtin" ||
|
||||
source === "google-jobs" ||
|
||||
source === "qajobsboard" ||
|
||||
source === "arcdev"
|
||||
) {
|
||||
|
||||
@@ -1,137 +0,0 @@
|
||||
import * as api from "@client/api";
|
||||
import type { RxResumeMode } from "@shared/types.js";
|
||||
import { RefreshCw } from "lucide-react";
|
||||
import type React from "react";
|
||||
import { useCallback, useEffect, useState } from "react";
|
||||
import { Button } from "@/components/ui/button";
|
||||
import {
|
||||
Select,
|
||||
SelectContent,
|
||||
SelectItem,
|
||||
SelectTrigger,
|
||||
SelectValue,
|
||||
} from "@/components/ui/select";
|
||||
|
||||
type BaseResumeSelectionProps = {
|
||||
value: string | null;
|
||||
onValueChange: (value: string | null) => void;
|
||||
hasRxResumeAccess: boolean;
|
||||
rxresumeMode?: RxResumeMode;
|
||||
disabled?: boolean;
|
||||
isLoading?: boolean;
|
||||
};
|
||||
|
||||
export const BaseResumeSelection: React.FC<BaseResumeSelectionProps> = ({
|
||||
value,
|
||||
onValueChange,
|
||||
hasRxResumeAccess,
|
||||
rxresumeMode,
|
||||
disabled = false,
|
||||
isLoading = false,
|
||||
}) => {
|
||||
const [resumes, setResumes] = useState<{ id: string; name: string }[]>([]);
|
||||
const [isFetchingResumes, setIsFetchingResumes] = useState(false);
|
||||
const [fetchError, setFetchError] = useState<string | null>(null);
|
||||
|
||||
const fetchResumes = useCallback(async () => {
|
||||
if (!hasRxResumeAccess) {
|
||||
setResumes([]);
|
||||
setFetchError(null);
|
||||
return;
|
||||
}
|
||||
|
||||
setIsFetchingResumes(true);
|
||||
setFetchError(null);
|
||||
try {
|
||||
const data = await api.getRxResumes(rxresumeMode);
|
||||
setResumes(data);
|
||||
|
||||
// Preselect if only one option is available and no value is currently set
|
||||
if (data.length === 1 && !value) {
|
||||
onValueChange(data[0].id);
|
||||
}
|
||||
} catch (error) {
|
||||
setResumes([]);
|
||||
setFetchError(
|
||||
error instanceof Error ? error.message : "Failed to fetch resumes",
|
||||
);
|
||||
} finally {
|
||||
setIsFetchingResumes(false);
|
||||
}
|
||||
}, [hasRxResumeAccess, onValueChange, rxresumeMode, value]);
|
||||
|
||||
useEffect(() => {
|
||||
if (hasRxResumeAccess) {
|
||||
fetchResumes();
|
||||
}
|
||||
}, [hasRxResumeAccess, fetchResumes]);
|
||||
|
||||
useEffect(() => {
|
||||
if (!hasRxResumeAccess) {
|
||||
setResumes([]);
|
||||
setFetchError(null);
|
||||
}
|
||||
}, [hasRxResumeAccess]);
|
||||
|
||||
return (
|
||||
<div className="space-y-2">
|
||||
<div className="flex items-center justify-between">
|
||||
<div className="text-sm font-medium">Template Resume</div>
|
||||
<Button
|
||||
variant="ghost"
|
||||
size="sm"
|
||||
onClick={fetchResumes}
|
||||
disabled={isFetchingResumes || isLoading || disabled}
|
||||
className="h-8 px-2"
|
||||
>
|
||||
<RefreshCw
|
||||
className={`h-3 w-3 mr-1 ${isFetchingResumes ? "animate-spin" : ""}`}
|
||||
/>
|
||||
Refresh
|
||||
</Button>
|
||||
</div>
|
||||
|
||||
<Select
|
||||
value={value || ""}
|
||||
onValueChange={(val: string) => onValueChange(val || null)}
|
||||
disabled={disabled || isLoading || isFetchingResumes}
|
||||
>
|
||||
<SelectTrigger>
|
||||
<SelectValue
|
||||
placeholder={
|
||||
resumes.length > 0
|
||||
? "Select a template resume..."
|
||||
: "No resumes found"
|
||||
}
|
||||
/>
|
||||
</SelectTrigger>
|
||||
<SelectContent>
|
||||
{resumes.map((resume) => (
|
||||
<SelectItem key={resume.id} value={resume.id}>
|
||||
{resume.name}
|
||||
</SelectItem>
|
||||
))}
|
||||
</SelectContent>
|
||||
</Select>
|
||||
|
||||
{resumes.length === 0 && !isFetchingResumes && !fetchError && (
|
||||
<div className="text-xs text-amber-600 dark:text-amber-400 mt-2">
|
||||
No resumes found in your account. Please create a resume on the{" "}
|
||||
<a
|
||||
href="https://rxresu.me"
|
||||
target="_blank"
|
||||
rel="noreferrer"
|
||||
className="font-semibold underline underline-offset-2"
|
||||
>
|
||||
Reactive Resume website
|
||||
</a>{" "}
|
||||
first.
|
||||
</div>
|
||||
)}
|
||||
|
||||
{fetchError && (
|
||||
<div className="text-xs text-destructive mt-1">{fetchError}</div>
|
||||
)}
|
||||
</div>
|
||||
);
|
||||
};
|
||||
@@ -34,6 +34,8 @@ type DangerZoneSectionProps = {
|
||||
handleClearByStatuses: () => void;
|
||||
handleClearDatabase: () => void;
|
||||
handleClearByScore?: (threshold: number) => void;
|
||||
handleClearDiscoveredMatchingArchived?: () => void;
|
||||
handleExpireDeadListings?: () => void;
|
||||
isLoading: boolean;
|
||||
isSaving: boolean;
|
||||
};
|
||||
@@ -44,6 +46,8 @@ export const DangerZoneSection: React.FC<DangerZoneSectionProps> = ({
|
||||
handleClearByStatuses,
|
||||
handleClearDatabase,
|
||||
handleClearByScore,
|
||||
handleClearDiscoveredMatchingArchived,
|
||||
handleExpireDeadListings,
|
||||
isLoading,
|
||||
isSaving,
|
||||
}) => {
|
||||
@@ -154,6 +158,106 @@ export const DangerZoneSection: React.FC<DangerZoneSectionProps> = ({
|
||||
|
||||
<Separator />
|
||||
|
||||
{handleClearDiscoveredMatchingArchived && (
|
||||
<div className="flex flex-col gap-4 sm:flex-row sm:items-center sm:justify-between p-3 rounded-md">
|
||||
<div className="space-y-0.5">
|
||||
<div className="text-sm font-semibold text-destructive">
|
||||
Clear Rediscovered Archive Matches
|
||||
</div>
|
||||
<div className="text-xs text-muted-foreground">
|
||||
Delete Discovered jobs that match roles you already skipped or
|
||||
applied (same employer + title, including cross-source
|
||||
reposts).
|
||||
</div>
|
||||
</div>
|
||||
<AlertDialog>
|
||||
<AlertDialogTrigger asChild>
|
||||
<Button
|
||||
variant="destructive"
|
||||
size="sm"
|
||||
disabled={isLoading || isSaving}
|
||||
>
|
||||
<Trash2 className="mr-2 h-4 w-4" />
|
||||
Clear Matches
|
||||
</Button>
|
||||
</AlertDialogTrigger>
|
||||
<AlertDialogContent>
|
||||
<AlertDialogHeader>
|
||||
<AlertDialogTitle>
|
||||
Clear rediscovered archive matches?
|
||||
</AlertDialogTitle>
|
||||
<AlertDialogDescription>
|
||||
This permanently deletes Discovered jobs that match
|
||||
skipped or applied roles. Ready, applied, and skipped jobs
|
||||
are not deleted. This action cannot be undone.
|
||||
</AlertDialogDescription>
|
||||
</AlertDialogHeader>
|
||||
<AlertDialogFooter>
|
||||
<AlertDialogCancel>Cancel</AlertDialogCancel>
|
||||
<AlertDialogAction
|
||||
onClick={handleClearDiscoveredMatchingArchived}
|
||||
className="bg-destructive text-destructive-foreground hover:bg-destructive/90"
|
||||
>
|
||||
Clear matches
|
||||
</AlertDialogAction>
|
||||
</AlertDialogFooter>
|
||||
</AlertDialogContent>
|
||||
</AlertDialog>
|
||||
</div>
|
||||
)}
|
||||
|
||||
{handleExpireDeadListings && (
|
||||
<>
|
||||
<Separator />
|
||||
<div className="flex flex-col gap-4 sm:flex-row sm:items-center sm:justify-between p-3 rounded-md">
|
||||
<div className="space-y-0.5">
|
||||
<div className="text-sm font-semibold text-destructive">
|
||||
Mark Expired Listings
|
||||
</div>
|
||||
<div className="text-xs text-muted-foreground">
|
||||
Open each Discovered job URL. If the page is gone (404/410
|
||||
or “job expired”), mark it expired so it leaves the queue.
|
||||
</div>
|
||||
</div>
|
||||
<AlertDialog>
|
||||
<AlertDialogTrigger asChild>
|
||||
<Button
|
||||
variant="destructive"
|
||||
size="sm"
|
||||
disabled={isLoading || isSaving}
|
||||
>
|
||||
<Trash2 className="mr-2 h-4 w-4" />
|
||||
Mark Expired
|
||||
</Button>
|
||||
</AlertDialogTrigger>
|
||||
<AlertDialogContent>
|
||||
<AlertDialogHeader>
|
||||
<AlertDialogTitle>
|
||||
Mark expired listings?
|
||||
</AlertDialogTitle>
|
||||
<AlertDialogDescription>
|
||||
This probes Discovered job links and marks dead ones as
|
||||
expired. Network failures are ignored (jobs are kept).
|
||||
This can take a few minutes for large queues.
|
||||
</AlertDialogDescription>
|
||||
</AlertDialogHeader>
|
||||
<AlertDialogFooter>
|
||||
<AlertDialogCancel>Cancel</AlertDialogCancel>
|
||||
<AlertDialogAction
|
||||
onClick={handleExpireDeadListings}
|
||||
className="bg-destructive text-destructive-foreground hover:bg-destructive/90"
|
||||
>
|
||||
Mark expired
|
||||
</AlertDialogAction>
|
||||
</AlertDialogFooter>
|
||||
</AlertDialogContent>
|
||||
</AlertDialog>
|
||||
</div>
|
||||
</>
|
||||
)}
|
||||
|
||||
<Separator />
|
||||
|
||||
{/* Clear Jobs Below Score */}
|
||||
{handleClearByScore && (
|
||||
<div className="p-3 rounded-md space-y-4">
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user