Data Infrastructure Engineer, Knowledge Graph & Pipelines
Posted Updated
Description
Exa is building a knowledge platform that synthesizes information from the web and beyond into one of the world’s largest open-domain knowledge graphs. In this role, you’ll construct knowledge pipelines that turn docs, datasets and APIs into ontology-shaped claims with provenance and confidence.
This greenfield work involves decisions on representation, trust, and evolving schemas so the graph remains correct as data changes while supporting idempotent and resumable processing at scale.
Continuously synthesize live web crawl, vendor deliveries, APIs into a typed knowledge graph with provenance, timestamp and confidence. Bring new sources in with versioned history and replayable cursors. Decide what to trust via entity resolution and truth discovery across billions of records. Keep the graph current with streaming changes, ensuring eventual consistency and resumable updates. Measure precision, completeness and lineage to trace issues. Experience building and operating large-scale distributed data pipelines in production. Deep understanding of incremental processing, CDC, idempotent writes and backfills. Strong Python skills and ability to read/tune Spark query plans. Focus on correctness, reliability, and data quality at scale.Skills
Sign in to read and join the discussion.