Senior Data Engineer - AI/RAG
Description
About the Role:
We are looking for a Senior AI Data Engineer / Data Scientist who can turn messy enterprise data into AI-ready, high-quality knowledge assets. You will lead the cleanup, preparation, and enrichment of unstructured content (SharePoint/document repositories) and structured/semi-structured data (data lakes, databases) so our agents, copilots, and RAG systems are accurate, trustworthy, and scalable.
Key Responsibilities:
- Lead 'data triage' for AI use cases: identify authoritative sources, duplicates, outdated content, and low-quality documents.
- Build robust ingestion pipelines for SharePoint and file repositories: parsing, text extraction, structure recovery, and metadata capture.
- Design chunking strategies, metadata enrichment, and document structuring optimized for retrieval performance and cost.
- Define and implement data quality gates (freshness, completeness, relevance, dedupe rate, metadata coverage).
- Ensure traceability: lineage from source - processed artifact - retrieval corpus - production usage.
Tech Stack:
- Ingestion/ETL foundations, Postgres + pgvector, Redis, LLM gateway patterns, Langfuse, DeepEval/RAGAS, Presidio.
Required Experience:
- 5+ years in data engineering / applied data science / analytics engineering with ownership of production pipelines.
- Strong Python for data processing (parsing, cleaning, normalization, pipelines).
- Proven experience working with unstructured enterprise data (documents, PDFs, Office files, wikis, knowledge bases).
- Proven experience building data pipelines for retrieval/NLP/RAG-like use cases.
- Solid understanding of data quality engineering: validation, monitoring, lineage, refresh cycles.
- Strong stakeholder skill: can work with business to define what data matters and what 'good' looks like.
Skills
Sign in to read and join the discussion.