AI Engineer (Data Guardrails & LLM Ingestion Pipelines)

This is a remote position.

About the Role

We are looking for a highly skilled AI Engineer to design and build robust data ingestion, cleaning, validation, and LLM enhancement pipelines that power our AI applications. You will transform raw, unstructured data into high-quality, AI-ready datasets while implementing guardrails that ensure accuracy, consistency, and reliability.

Key Responsibilities

· Design and develop scalable data ingestion pipelines for structured and unstructured data.

· Build automated data cleaning, normalization, and preprocessing workflows.

· Develop AI-powered enrichment pipelines using LLMs (OpenAI, Claude, Gemini, etc.).

· Implement data quality validation and AI guardrails.

· Develop prompt engineering workflows for data transformation.

· Build document processing pipelines for PDFs, Word documents, CSVs, websites, and APIs.

· Develop Retrieval-Augmented Generation (RAG) pipelines.

· Create evaluation frameworks for LLM quality and accuracy.

· Build ETL/ELT workflows for AI-ready datasets.

· Integrate vector databases for semantic search.

· Monitor pipeline performance, cost, latency, and data quality.

· Collaborate with cross-functional teams to deliver production AI systems.

Required Technical Skills

Programming

· Python (Expert)

· SQL

· Git

AI & LLMs

· OpenAI API

· Anthropic Claude API

· Google Gemini API

· Prompt Engineering

· Function Calling

· Structured Outputs

AI Frameworks

· LangChain

· LlamaIndex

· DSPy (Preferred)

· PydanticAI (Nice to Have)

Data Engineering

· Pandas

· Polars

· ETL/ELT Pipelines

· Apache Airflow (Preferred)

· Data Validation Frameworks

Vector Databases

· Pinecone

· Weaviate

· Qdrant

· ChromaDB

· FAISS

Cloud & Infrastructure

· Docker

· Kubernetes (Preferred)

· AWS / Azure / GCP

· Linux

Databases

· PostgreSQL

· MongoDB

· Redis

Requirements

Preferred Qualifications

· Experience building production-grade AI systems.

· Strong understanding of RAG architectures.

· Experience implementing AI guardrails and hallucination mitigation.

· Experience with OCR and document parsing.

· Experience with embedding models and semantic search.

· Knowledge of data governance and security best practices.

Success Metrics

· Build scalable ingestion pipelines.

· Deliver automated data cleaning and LLM enhancement workflows.

· Implement AI guardrails to improve output quality.

· Develop evaluation pipelines for LLM performance.

· Contribute to a production-ready AI platform.