Jobber/docs-site/docs/extractors/gradcracker.md

id, title, description, sidebar_position

id	title	description	sidebar_position
gradcracker	Gradcracker Extractor	How the Gradcracker crawler builds search URLs and extracts jobs.	2

A plain-English walkthrough of the Gradcracker extractor in extractors/gradcracker.

Original website: gradcracker.com

Big picture

The crawler builds search URLs, scrapes listing pages, then opens job details for descriptions and apply URLs.

1) Build search URLs

• Combines UK regions with role terms.
• Defaults include roles such as web-development and software-systems.
• GRADCRACKER_SEARCH_TERMS overrides defaults.

2) Crawl list pages

• Waits for job cards (article[wire:key]).
• Extracts title, employer, discipline, deadline, salary, location, degree, start date.
• Queues job detail pages.

Controls:

• GRADCRACKER_MAX_JOBS_PER_TERM
• JOBOPS_SKIP_APPLY_FOR_EXISTING=1
• JOBOPS_EXISTING_JOB_URLS / JOBOPS_EXISTING_JOB_URLS_FILE

3) Crawl detail pages

• Waits for .body-content
• Captures full description text
• Clicks apply button to resolve final application URL
• Handles popup and same-tab redirects

4) Progress reporting

Set JOBOPS_EMIT_PROGRESS=1 for structured progress lines consumable by orchestrator UI.

Notes

• Uses Playwright + Crawlee via Camoufox.
• Low concurrency and longer timeouts for stability.
• Employer logo (figure img alt text) and employer link are optional. Cards that omit them used to throw a Playwright getAttribute timeout and fail the whole region page (0 jobs saved even after cards were counted). Missing employer fields are now skipped per card.

Common problems

Region pages time out / run returns 0 jobs

• Gradcracker Livewire markup sometimes omits the employer logo on a card. Older builds aborted the whole page on that timeout. Current builds continue and leave employer blank.
• Anti-bot / Camoufox issues: ensure Camoufox is installed (npx camoufox-js fetch) and prefer headed runs when debugging.