Auto-Registering Extractor System (#223)

* initial commit?

* Address PR feedback on extractor discovery and startup resilience

* Address latest PR review comments

* fix city resolution fallback when input parses empty

* address PR feedback on extractor registry and pipeline validation

* address copilot comments on manifests and registry startup

* fix extractor discovery export handling and env isolation in tests

* enforce duplicate manifest id failures in strict mode

* Fix remaining extractor registry and runtime review comments

* docs

* docs

* test all, logic remains in extractors

* Address PR review feedback on extractor registry and validation

* Revert extractor moduleResolution to bundler

* Enforce shared city filtering across all discovery sources

* Deduplicate extractor strict city post-filtering
This commit is contained in:
Shaheer Sarfaraz
2026-02-21 17:44:07 +00:00
committed by GitHub
parent cc7cacd7f5
commit 82e142a8a8
52 changed files with 2207 additions and 1515 deletions
+3
View File
@@ -7,6 +7,8 @@ sidebar_position: 1
This page helps you choose the right extractor for your run, understand key constraints, and navigate to detailed technical guides.
Extractor integrations are now registered through manifests and loaded automatically at orchestrator startup. Runtime discovery only scans `extractors/*/(manifest.ts|src/manifest.ts)` and does not read manifests from `orchestrator/**`. Extractor-specific run logic should also remain in `extractors/<name>/` so orchestrator stays source-agnostic. To add a new source, follow [Add an Extractor](/docs/next/workflows/add-an-extractor).
## Extractor chooser
| Extractor | Best use case | Core constraints/dependencies | Notable controls | Output/behavior notes |
@@ -37,3 +39,4 @@ Many runs combine sources: broad discovery first, then manual import for high-pr
- [Hiring Cafe](/docs/next/extractors/hiring-cafe)
- [UKVisaJobs](/docs/next/extractors/ukvisajobs)
- [Manual Import](/docs/next/extractors/manual)
- [Add an Extractor](/docs/next/workflows/add-an-extractor)