Sr Software Engineer
Posted Updated
Description
Job Title: Senior Software Engineer - Fullstack AI (Data Focus)
Location: Pune, MH
Experience: 4+ years in software engineering, including 1+ year building data-intensive or LLM-powered systems in production
Job Summary
Youll build the data and retrieval foundation that makes Clearwaters AI agents accurate, grounded, and measurable. This is a hands-on software engineering role, not a research role: youll write production pipelines and services, build retrieval and evaluation systems, and ship the product surfaces that expose them. Youll work across security and threat data at scale.
Responsibilities
- Build and operate data pipelines (batch and streaming) that ingest, normalize, and enrich large volumes of security and telemetry data
- Design retrieval systems for agents: chunking, embeddings, hybrid search, reranking, vector and graph stores, and freshness and access control
- Build evaluation harnesses and datasets to measure agent and retrieval quality, regressions, and cost
- Build backend APIs and services in Python, and the full-stack features that surface data and AI insights to users
- Design data models and manage relational, analytical, and vector stores for performance and cost
- Add data quality checks, lineage, observability, and governance, including PII and sensitive-data handling
- Use AI agents to automate data engineering work such as pipeline generation, data validation, and documentation
- Containerize and deploy on Kubernetes and cloud with CI/CD, and mentor engineers on data and AI practices
Requirements
- 4+ years of professional software engineering with strong data-engineering exposure and sound testing and CI/CD practice
- 1+ year building production systems involving LLMs, retrieval (RAG), or large-scale data pipelines
- Strong Python and SQL, including query optimization and data modeling
- Hands-on with pipeline or stream-processing tooling such as Airflow, Dagster, Spark, or Kafka, or an equivalent
- Experience with vector stores and retrieval design (pgvector, OpenSearch, or similar), and with evaluating retrieval quality
- Docker, working Kubernetes knowledge, and experience on AWS, Azure, or GCP
Nice to have
- Working JavaScript/TypeScript, enough to build and ship full-stack features
- Experience with security, threat intelligence, or SIEM/telemetry data
- Healthcare or compliance-driven data experience (HIPAA, SOC 2, HITRUST)
- Knowledge graphs or graph databases
- LLMOps and evaluation tooling experience
- Experience with a lakehouse or warehouse such as Snowflake, Databricks, or BigQuery
- Open-source contributions or public data or retrieval projects
First 12 months
- Ship production pipelines and retrieval services with clear quality and reliability targets
- Deliver a retrieval and evaluation framework that measurably improves agent accuracy
- Establish the teams patterns for data quality, grounding, and sensitive-data handling
Disclaimer: This job posting has been aggregated from external source. Role details, content, and availability are subject to change. Applicants are advised to confirm the latest information directly on the company website before applying.
Skills
- Agentic AI
- AI
- Airflow
- API
- AWS
- Azure
- BigQuery
- CI/CD
- Cloud
- Dagster
- Data Engineering
- Data Modeling
- Data Pipelines
- Data Quality
- Databricks
- Docker
- Embeddings
- GCP
- Hipaa
- JavaScript
- Kafka
- Kubernetes
- Lakehouse
- LLM
- LLMOps
- Observability
- OpenSearch
- pgvector
- Python
- SIEM
- Snowflake
- SOC 2
- Spark
- SQL
- Threat Intelligence
- TypeScript