ilia/Jobber
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: Google Jobs extractor and scraper reliability fixes #2

Merged
ilia merged 1 commits from feat/google-jobs-and-scraper-reliability into main 2026-07-09 14:17:09 -05:00
Conversation 0 Commits 1 Files Changed 47
+1989 -87
ilia commented 2026-07-09 14:14:05 -05:00
Owner
Copy Link
Copy Source

Summary

  • Add Camoufox-backed Google Jobs extractor (udm=8) with session persistence, CAPTCHA wait mode, and docs.
  • Fix Glassdoor always returning 0 jobs (python-jobspy discarded pages on unrelated GraphQL partial errors).
  • Fix Himalayas early-exit bug (API caps limit at 20; extractor assumed 50).
  • Fix Gradcracker region pages failing on optional employer logo timeouts.
  • Harden prior-skip dismiss matching (90-day window, employer/title fuzzy match) and multi-profile basic-auth active-profile switching for CA/US runs.

Test plan

  • biome ci . (warnings only on unrelated unused vars)
  • npm run check:types:shared + orchestrator / google-jobs / gradcracker / ukvisajobs typechecks
  • npm --workspace orchestrator run build:client
  • npm --workspace orchestrator run test:run (978 tests)
  • npm run check:docs
  • Live smoke: google-jobs, hiringcafe, wellfound, jobspy, himalayas, gradcracker, and HTTP extractors
  • Homelab Gitea Actions (docker-ci, secret-scan) on this PR
  • Deploy VM (jobber-delian / 10.0.10.178): pull + docker compose up -d --build after merge
## Summary - Add Camoufox-backed **Google Jobs** extractor (`udm=8`) with session persistence, CAPTCHA wait mode, and docs. - Fix **Glassdoor** always returning 0 jobs (python-jobspy discarded pages on unrelated GraphQL partial errors). - Fix **Himalayas** early-exit bug (API caps `limit` at 20; extractor assumed 50). - Fix **Gradcracker** region pages failing on optional employer logo timeouts. - Harden prior-skip dismiss matching (90-day window, employer/title fuzzy match) and multi-profile basic-auth active-profile switching for CA/US runs. ## Test plan - [x] `biome ci .` (warnings only on unrelated unused vars) - [x] `npm run check:types:shared` + orchestrator / google-jobs / gradcracker / ukvisajobs typechecks - [x] `npm --workspace orchestrator run build:client` - [x] `npm --workspace orchestrator run test:run` (978 tests) - [x] `npm run check:docs` - [x] Live smoke: google-jobs, hiringcafe, wellfound, jobspy, himalayas, gradcracker, and HTTP extractors - [ ] Homelab Gitea Actions (`docker-ci`, `secret-scan`) on this PR - [ ] Deploy VM (`jobber-delian` / `10.0.10.178`): pull + `docker compose up -d --build` after merge
ilia added 1 commit 2026-07-09 14:14:05 -05:00
feat: add Google Jobs extractor and fix scraper reliability gaps
CI / skip-ci-check (pull_request) Successful in 8s
Details
CI / docker-ci (pull_request) Successful in 10s
Details
CI / secret-scan (pull_request) Successful in 7s
Details
40c7cdece3 
Ship a Camoufox-backed Google Jobs source, restore Glassdoor results discarded by a python-jobspy GraphQL quirk, and fix Himalayas/Gradcracker zero-job failures. Also harden prior-skip dismiss matching and multi-profile basic-auth switching for CA/US runs.
ilia merged commit 529f0a7b02 into main 2026-07-09 14:17:09 -05:00
Sign in to join this conversation.
Reviewers
No Reviewers
Labels
No items
No labels
Milestone
No items
No Milestone
Projects
Clear projects
No projects
Assignees
Clear assignees
ilia (Ilia)
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: ilia/Jobber#2