feat: add JS Remotely (javascript.jobs) extractor
Adds a pipeline extractor for javascript.jobs (the board now promoted under the jsremotely.com alias, which 301s straight here). No public API/RSS, so it scrapes the /remote listing (server-side keyword+page filtering) and enriches each match from the job detail page's JobPosting schema.org block, which is emitted as near-JSON (unescaped newlines, an occasional missing comma) and is parsed with targeted regexes instead of JSON.parse. Also guards against the source's own baseSalary bug (0 placeholders, and at least one posting with minValue == maxValue) by keeping the listing-card salary text as a trustworthy fallback. Wires the new "javascriptjobs" source through the shared extractor catalog, settings registry/types/factories, the max-coverage automatic-run preset, demo seed data, and the smoke-test target.
This commit is contained in:
@@ -0,0 +1,31 @@
|
||||
# javascriptjobs-extractor
|
||||
|
||||
Pulls remote listings from [javascript.jobs](https://javascript.jobs) (the
|
||||
board formerly promoted under the `jsremotely.com` alias — that domain now
|
||||
301s straight here). JS/TS-focused: React, Vue, Node, Angular, TypeScript.
|
||||
|
||||
- No authentication, no public API/RSS — this is a server-rendered HTML
|
||||
scrape of `/remote`, which supports a `keyword` query param for server-side
|
||||
term filtering (`?keyword=react&page=2`), so pipeline search terms are
|
||||
iterated as separate paginated fetches rather than filtered client-side.
|
||||
- Listing cards give title/employer/job-type badge/salary text/logo. Detail
|
||||
pages (`/job/<slug>`) additionally embed a `JobPosting` (schema.org)
|
||||
`<script type="application/ld+json">` block with description, exact
|
||||
`datePosted`, `employmentType`, and structured `baseSalary`.
|
||||
- That embedded block is **not valid JSON** (unescaped newlines inside
|
||||
`description`, and at least one observed missing comma between sibling
|
||||
keys), so we extract fields with targeted regexes instead of
|
||||
`JSON.parse`. If a detail fetch fails or a field is missing, we fall back
|
||||
to the listing-card values.
|
||||
- `baseSalary` in that block is unreliable on the source side: postings
|
||||
with no salary entered still emit `minValue`/`maxValue: 0` (we treat 0 as
|
||||
"not specified"), and at least one observed posting had `minValue` equal
|
||||
to `maxValue` (both set to the range's upper bound) despite the
|
||||
human-readable listing-card text showing a real range. We keep the raw
|
||||
listing-card salary string in `salary` as a trustworthy fallback alongside
|
||||
the (possibly wrong) structured `salaryMinAmount`/`salaryMaxAmount`.
|
||||
- Caps via `javascriptjobsMaxJobsPerTerm` (default 50) and
|
||||
`javascriptjobsMaxPages` (default 3, pages per term).
|
||||
- Every matched job triggers one extra HTTP GET (detail page) for
|
||||
enrichment — keep the caps conservative given this is a low-volume niche
|
||||
board.
|
||||
Reference in New Issue
Block a user