// how it works
Five steps before a posting reaches you.
Crawled, tagged, checked for duplicates, and rebuilt into the index you search — automatically, with no person touching an individual listing.
// where it starts
303K companies
Every open role traces back to one of them — crawled straight from where they actually posted it, not a copy of a copy.
// the pipeline
Follow one posting below — the picture at each step is that same job, at that stage.
Ingest
Crawled straight from the source, several times a day.
More detail— Ingest
Most of the catalogue comes straight from an employer's own ATS or careers page, not from a copy of a copy. Each source runs on its own schedule so one slow platform never holds up the rest, and a board that keeps failing backs off instead of being hammered every run.
Derive
Tagged by a fixed dictionary — never guessed.
More detail— Derive
This step never invents a value: it looks a term up in a closed dictionary and tags the posting if it finds a match, and emits nothing for anything it doesn't recognize. That is deliberate — a facet built on a guess degrades quietly, and quietly is the one way a filter is allowed to fail here.
Enrich
An LLM reads what a dictionary can't tag.
More detail— Enrich
This runs from a queue, not inline with the crawl, so a slow model call never holds up ingest. Every result is validated against a fixed schema before it reaches you; one that fails validation is dropped rather than shown half-formed.
Deduplicate
Three boards, one listing.
More detail— Deduplicate
This actually runs twice. Ingest itself never stores the same posting from the same source twice; separately, a later pass clusters near-duplicate postings by role and company across different boards and collapses them into one — including the aggregator's copy of a job we already have straight from the source. You see one listing; where it's actually posted is still there for you to check.
Reindex
Rebuilt and swapped in — search never goes down.
More detail— Reindex
The index is rebuilt from scratch and swapped in atomically, so a search you run mid-rebuild reads the old, complete index or the new one — never a half-written one in between.
// questions
How fresh is a posting when I see it?
Each source is crawled on its own schedule, several times a day for most boards. A posting typically clears ingest, tagging and dedup within the same day it goes up on the source.
Why do some postings have more detail than others?
Enrichment runs from a queue and reads the full posting text, so the depth of detail depends on how much the source itself wrote. A one-line listing stays a one-line listing — nothing here is invented to fill a gap.
If a job is on three boards, which one do I see?
One listing, picked by preferring an employer's own ATS or careers page over an aggregator's copy of the same role. The listing still tells you where it's actually hosted.
Does deduplication ever merge two different jobs by mistake?
It clusters on role and company together, not on title text alone, so two different openings with the same title at the same company stay separate. The clustering is re-run periodically as new postings arrive, which is also how a wrongly split pair gets caught and merged later.
Can search show a job that was already deleted?
No — the index is rebuilt from the current catalogue and swapped in as a whole, never patched in place, so a search can only return what was true at that rebuild.
See it running on the real catalogue
This pipeline runs on every job you'll find here.