---
id: google-jobs
title: Google Jobs Extractor
description: Browser-backed scraping of the Google for Jobs search widget.
sidebar_position: 8
---

## What it is

Original feature: [Google for Jobs](https://www.google.com/search?q=jobs&udm=8)

Google Jobs is a browser-backed extractor that opens Google's Jobs vertical (`udm=8`) in Firefox (Camoufox when available) for each pipeline search term, scrolls the results panel to load additional cards, then clicks into each card to resolve a real "Apply on &lt;site&gt;" link and description before mapping rows into the orchestrator `CreateJobInput` shape.

Implementation split:

1. `extractors/google-jobs/src/main.ts` builds the search URL, waits for the jobs list, scrolls to lazy-load more cards, extracts card metadata (title/employer/location/"via" source), clicks each card to read the detail pane, and writes dataset JSON.
2. `extractors/google-jobs/src/run.ts` spawns the browser subprocess, applies a hard timeout, reads the dataset, and maps rows to `CreateJobInput`.
3. `extractors/google-jobs/manifest.ts` wires pipeline `searchTerms` into the extractor.

### A note on the search URL and location

Google's Jobs vertical is reached with `udm=8` (its current internal vertical id). An older, commonly-documented deep link — `ibp=htl;jobs` — still works but gets silently redirected to `udm=8` anyway, and combining it with extra parameters (a `num=` result-count hint, or a `gl=` country override that doesn't match the browser's real/proxied IP) has been observed to make Google return "no matches" or "can't determine location" even though the tab itself loads correctly.

Because of that, this extractor does **not** send a `gl` (country) parameter — location is resolved by Google from the browser's real IP (or `GOOGLE_JOBS_PROXY`'s exit IP), which Camoufox's `geoip` option keeps internally consistent. To target a different country's jobs, use a residential proxy with an exit node in that country rather than trying to override the country via a parameter.

### Dealing with Google's CAPTCHA / bot detection

Google actively fingerprints and challenges automated browser traffic, especially from datacenter/VPS IPs (which most servers running JobOps' cron pipeline will have). The extractor has layered mitigations, from always-on to opt-in:

1. **Always on:** Camoufox launches with `humanize: true` and `geoip: true` (human-like cursor movement, IP-consistent locale/geolocation) to reduce — not eliminate — fingerprint-based detection.
2. **Always on:** browser cookies/session are persisted to `extractors/google-jobs/storage/state.json` and reused on the next run. A session that has cleared a CAPTCHA once (including via option 4 below) tends to see fewer challenges for a while afterward.
3. **Opt-in, most reliable:** set `GOOGLE_JOBS_PROXY` (e.g. a residential/mobile proxy URL) to avoid datacenter-IP blocks in the first place. This is the standard real-world fix; unproxied runs from cloud servers should expect to be blocked often.
4. **Opt-in, manual/local only:** set `GOOGLE_JOBS_HEADLESS=false` and run the extractor directly (not via the pipeline) to open a real, visible Firefox window. If Google shows a CAPTCHA, solve it yourself in that window — the script polls and automatically resumes once it clears (up to `GOOGLE_JOBS_UNBLOCK_TIMEOUT_MS`, default 5 minutes). The resulting session cookies are saved and reused by later automated/headless runs. **Do not** set `GOOGLE_JOBS_HEADLESS=false` in the server's always-on `.env` — a headless server has no display for a human to interact with, and the run will just wait until the timeout.

## Why it exists

Google for Jobs aggregates postings across the open web — including boards and employer career pages that JobOps does not otherwise scrape. Adding it as a proper source closes some of the gap between "jobs found by manually Googling a role" and "jobs discovered by the pipeline."

It is **not** a general web crawler and does not replace dedicated extractors: it only surfaces whatever Google's Jobs widget chooses to show for a query, subject to Google's own ranking, freshness, and regional availability.

## How to use it

1. Open **Run jobs** and choose **Automatic**.
2. **Google Jobs** is enabled by default in **Sources** (toggle it off if you do not want it for this run).
3. Set your existing automatic run knobs:
   - `searchTerms` become the `"<term> jobs"` query sent to Google per term.
   - `googleJobsMaxJobsPerTerm` (Settings) caps jobs scraped per run (default `30`, max `150`).
   - Location is derived from the browser's real/proxied IP, not from a country setting — see [the note above](#a-note-on-the-search-url-and-location).
4. Start the run and watch progress in the pipeline progress card.

Local run example (headless, no proxy):

```bash
GOOGLE_JOBS_SEARCH_TERMS='["automation engineer"]' \
GOOGLE_JOBS_MAX_JOBS='10' \
npm --workspace google-jobs-extractor run start
```

Local run example (headed, solve the CAPTCHA yourself if one appears):

```bash
GOOGLE_JOBS_HEADLESS=false \
GOOGLE_JOBS_SEARCH_TERMS='["automation engineer"]' \
npm --workspace google-jobs-extractor run start
```

Defaults and constraints:

- No credentials required; this is a direct HTML/DOM scrape of Google's public search results, not an official API (Google does not offer a free Google Jobs API).
- Google's markup and CSS class names change frequently and are obfuscated; the scraper favors known current card classes (e.g. `.EimVGf`) with multiple fallback selectors and a leaf-node text heuristic, but selectors can still rot and return zero results until updated.
- Every card carries Google's own `data-share-url` permalink for that specific listing, which the extractor uses as `jobUrl` — so a job stays unique and reviewable even if the "click into the card for a detail pane" step below fails for it.
- Because resolving a real application link requires clicking each card in a real browser, this extractor is slower per job than API-backed sources — keep `googleJobsMaxJobsPerTerm` modest for frequent runs.
- If no external "Apply on &lt;site&gt;" link is found for a card, the extractor falls back to Google's own permalink for that listing so it's still reviewable, but not directly one-click-applyable.
- After scraping, JobOps probes each apply URL (preferring the external link over Google's share permalink). Listings that return **404/410** or pages that say the job expired / is no longer available are **dropped before import**. Network failures fail-open (job is kept).
- Automated, frequent, or high-volume scraping of Google search results may be against Google's Terms of Service; treat this extractor as best-effort and keep run frequency/volume conservative.
- See [Dealing with Google's CAPTCHA / bot detection](#dealing-with-googles-captcha--bot-detection) above for proxy and interactive-solve options.

## Common problems

### Google Jobs returns 0 jobs for a term

The extractor logs one of two distinct messages to help tell these apart:

- `Google's Jobs tab reported no matches for "<term>"` — Google correctly landed on the Jobs tab but genuinely has no results for that query/locale. Try a broader, more common phrasing (e.g. `"automation engineer"` instead of a very narrow title).
- `Jobs list did not appear for "<term>"` — an unrecognized page state (consent wall, layout change, or rotted selectors). The extractor saves a screenshot and full HTML dump to `extractors/google-jobs/storage/debug-last.png` / `debug-last.html` for inspection; compare it against a fresh manual search to see what changed.

### Results look sparse or missing company/location

- The list-view heuristics rely on leaf DOM nodes near the job title; if Google restructures the card markup, company/location extraction can degrade even when titles still work. This is expected best-effort behavior for an unofficial scrape.

### Run is slow

- Each job requires a real click + detail-pane render, unlike single-request API extractors. Lower `googleJobsMaxJobsPerTerm` or reduce the number of search terms for faster runs.

### Blocked, CAPTCHA'd, or consistently empty results

- Google may rate-limit or challenge automated browser traffic, especially from datacenter/VPS IPs. In order of effort:
  1. Set `GOOGLE_JOBS_PROXY` to a residential/mobile proxy — the most reliable fix for unattended/server runs.
  2. Run locally with `GOOGLE_JOBS_HEADLESS=false` and solve the CAPTCHA yourself once; the saved session (`storage/state.json`) may reduce blocks on subsequent headless runs for a while.
  3. Reduce run frequency, lower the per-term cap, or disable the source for that run and rely on other extractors / Manual Import instead.

## Related pages

- [Extractors Overview](/docs/next/extractors/overview)
- [Manual Import](/docs/next/extractors/manual)
- [Pipeline Run](/docs/next/features/pipeline-run)
- [Settings](/docs/next/features/settings)
