Data Engineer
Posted Updated
Description
Responsibilities:
- Design and build enterprise knowledge foundations that enable accurate, governed, and context-aware AI and agentic systems.
- Assist with the data platform blueprint and design, encompassing the relevant data platform components.
- Collaborates with the Integration Architects and Data Architects to ensure cohesive integration between systems and data models.
- Architect ingestion, transformation, indexing, retrieval, and knowledge-enrichment pipelines.
- Build production RAG systems using structured, unstructured, graph, and metadata-driven retrieval.
- Define document parsing, chunking, taxonomy, ontology, entity resolution, and lineage strategies.
- Implement access-aware retrieval, freshness controls, quality checks, and auditability.
- Optimize retrieval quality, latency, scale, and cost.
- Partner with AI engineers on context construction and grounding.
- Mentor engineers and define reusable knowledge-engineering standards.
Requirements:
- Advanced proficiency with Databricks Assistant, GitHub Copilot, or Cursor, alongside LlamaIndex, LangChain, Neo4j, Pinecone, Weaviate, Elasticsearch/OpenSearch, and cloud vector-search services, to accelerate governed ingestion, knowledge graph, and production RAG engineering.
- Must have built and operated production data, search, knowledge, or retrieval platforms.
- Experience limited to basic vector-database prototypes is insufficient.
Professional and Technical Skills:
- Python, SQL, distributed processing, and orchestration.
- Vector search, hybrid retrieval, knowledge graphs, metadata, and semantic modeling.
- Document pipelines, search, data quality, lineage, and access control.
- Cloud, APIs, CI/CD, observability, and performance engineering.
- 15 years of full-time education.
Skills
Sign in to read and join the discussion.