ilia
40c7cdece3
Ship a Camoufox-backed Google Jobs source, restore Glassdoor results discarded by a python-jobspy GraphQL quirk, and fix Himalayas/Gradcracker zero-job failures. Also harden prior-skip dismiss matching and multi-profile basic-auth switching for CA/US runs.
46 lines
2.0 KiB
Markdown
46 lines
2.0 KiB
Markdown
---
|
|
id: himalayas
|
|
title: Himalayas Extractor
|
|
description: Public remote-jobs API pagination and client-side term filtering.
|
|
sidebar_position: 16
|
|
---
|
|
|
|
## What it is
|
|
|
|
Original site: [himalayas.app](https://himalayas.app)
|
|
|
|
The extractor lives in `extractors/himalayas/manifest.ts`. It paginates the public JSON API (`https://himalayas.app/jobs/api?limit=&offset=`), filters rows client-side by pipeline search terms (title + categories), and maps matches into `CreateJobInput`.
|
|
|
|
## Why it exists
|
|
|
|
Himalayas is a large remote-jobs index with a stable, unauthenticated API — useful for remote-first discovery without browser automation or credentials.
|
|
|
|
## How to use it
|
|
|
|
1. Enable **Himalayas** in pipeline sources.
|
|
2. Optionally set **Himalayas max jobs per term** (`himalayasMaxJobsPerTerm`, default `100`).
|
|
3. Pipeline `searchTerms` filter titles/categories client-side (substring match, case-insensitive).
|
|
4. Run the pipeline.
|
|
|
|
### Defaults and constraints
|
|
|
|
- The upstream API **silently caps `limit` at 20** per request. The extractor uses `PAGE_SIZE = 20` and paginates with `offset` (up to 10 pages / 200 rows scanned per run before the per-term cap).
|
|
- There is no server-side search — if a term is rare, early pages may not contain matches even though later pages do. Raise the page budget only by changing the extractor constants if you need deeper scans.
|
|
- No auth required.
|
|
|
|
## Common problems
|
|
|
|
### Zero jobs for a term I know exists
|
|
|
|
- The term may not appear in the first ~200 API rows (newest-first feed). Try a shorter token (`software`, `engineer`, `SDET`) or accept that Himalayas skews toward whatever is currently at the top of the feed.
|
|
- Confirm you are not hitting an old build that requested `limit=50` and stopped after one page (fixed: the API returns at most 20, which used to look like end-of-feed).
|
|
|
|
### Rate limits / HTTP errors
|
|
|
|
- Retry later; the public API is generally stable but can throttle burst traffic.
|
|
|
|
## Related pages
|
|
|
|
- [Extractors Overview](/docs/next/extractors/overview)
|
|
- [Add an Extractor](/docs/next/workflows/add-an-extractor)
|