Mid/Senior Data Engineer
Posted Updated 1
view
Summary
A Data Engineer building robust data pipelines and vector storage systems to support AI agents using Python, PySpark, and big data tools like Spark and Kafka to ingest unstructured customer support data.
We're transforming our customer support data from static, disconnected reports into a live, AI-powered system. We're looking for an experienced Data Engineer to build the pipelines, storage, and logic that power our new AI agents — turning messy customer chats and tickets into clean, structured information our AI can reliably use.
Job Responsibilities
- Ingest and clean large volumes of unstructured support data from Bliss, Salesforce, Sprinklr, and JIRA
- Design storage and retrieval systems (including vector databases) that give our AI accurate, relevant context
- Build a centralized, reliable metrics layer for AI-driven analytics
- Build robust, monitored, failure-resistant pipelines — no fire drills
- Partner with ops, product, and engineering; push back on poor data practices at the source
Required Skills
- 5+ years in data engineering with big data tools (Hadoop, Hudi, Spark, Presto, Pinot, Flink, Kafka) and cloud data warehouses
- Strong hands-on Python and PySpark
- Proven use of AI tools (e.g., Claude, Codex) to accelerate development — automating checks, parsing messy text, building data logic layers
- Direct experience with vector databases (Pinecone, Milvus, Weaviate, pgvector) and AI-feeding data pipelines
- Bonus: experience building metric or semantic layers
If this opportunity aligns with your experience and interests, I'd be happy to discuss it further. Please send your updated resume to .