feat(extractors): expand catalog, smoke coverage, and sourcing docs
CI / Linting (Biome) (push) Failing after 40s
CI / Tests (push) Successful in 5m54s
CI / Type Check (adzuna-extractor) (push) Successful in 1m8s
CI / Type Check (gradcracker-extractor) (push) Successful in 1m11s
CI / Type Check (hiringcafe-extractor) (push) Successful in 1m8s
CI / Type Check (orchestrator) (push) Successful in 1m23s
CI / Type Check (startupjobs-extractor) (push) Successful in 1m6s
CI / Type Check (ukvisajobs-extractor) (push) Successful in 1m7s
CI / Documentation (push) Successful in 1m54s
CI / Linting (Biome) (push) Failing after 40s
CI / Tests (push) Successful in 5m54s
CI / Type Check (adzuna-extractor) (push) Successful in 1m8s
CI / Type Check (gradcracker-extractor) (push) Successful in 1m11s
CI / Type Check (hiringcafe-extractor) (push) Successful in 1m8s
CI / Type Check (orchestrator) (push) Successful in 1m23s
CI / Type Check (startupjobs-extractor) (push) Successful in 1m6s
CI / Type Check (ukvisajobs-extractor) (push) Successful in 1m7s
CI / Documentation (push) Successful in 1m54s
Adds Arc.dev, BC T-Net, Eluta, iCIMS tenants, QAJobsBoard, and SmartRecruiters manifests with registry/settings/UI wiring; registers full extractor list in smoke-extractors and documents supplementary board access paths. Aligns Careerjet v4 with the url query parameter and fixes strict typing in QAJobsBoard. Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
@@ -41,7 +41,8 @@ That keeps runtime wiring dynamic while preserving compile-time safety in API an
|
||||
- append to `EXTRACTOR_SOURCE_IDS`
|
||||
- add an entry in `EXTRACTOR_SOURCE_METADATA`
|
||||
5. Ensure your extractor maps output to `CreateJobInput[]`.
|
||||
6. Run the full CI checks.
|
||||
6. Register it in `scripts/smoke-extractors.ts` (`ALL_TARGETS`): add one row per manifest so `npx tsx scripts/smoke-extractors.ts` exercises every shipped extractor (keyed sources `SKIP` until env vars exist).
|
||||
7. Run the full CI checks.
|
||||
|
||||
Example manifest:
|
||||
|
||||
@@ -77,6 +78,22 @@ Subprocess extractors are supported. Keep subprocess spawning inside `run(contex
|
||||
- Add the new source id to `shared/src/extractors/index.ts`.
|
||||
- Confirm metadata exists for that source id.
|
||||
|
||||
### Smoke connectivity
|
||||
|
||||
After wiring settings/env, run:
|
||||
|
||||
```bash
|
||||
npx tsx scripts/smoke-extractors.ts myextractor
|
||||
```
|
||||
|
||||
Or the full suite (may take several minutes — JobSpy invokes Python, Hiring Cafe / startup.jobs may need browser deps):
|
||||
|
||||
```bash
|
||||
npx tsx scripts/smoke-extractors.ts
|
||||
```
|
||||
|
||||
Keep `ALL_TARGETS` in that script aligned with manifests under each `extractors/<name>/` package (`manifest.ts` or `src/manifest.ts`).
|
||||
|
||||
### Source appears in shared catalog but is unavailable at runtime
|
||||
|
||||
- The manifest was not loaded successfully.
|
||||
|
||||
Reference in New Issue
Block a user