Senior AI Engineer
Posted
Description
The Role
Take a working Claude-based multi-agent prototype that transforms clinical study documents and raw data into validated analysis datasets and reports, and make it production grade: reliable, testable, auditable and ready for a regulated environment. You architect, establish engineering standards, mentor and deliver hands-on across the stack.
Key Responsibilities
- Own and evolve the production architecture for multi-agent AI systems, balancing reliability, quality, performance, and cost.
- Lead the transition from prototype to production, implementing robust orchestration, validation, caching, retries, recovery, and operational controls.
- Build AI evaluation and testing frameworks to measure quality, prevent regressions, and support reliable releases.
- Ensure production excellence through monitoring, logging, tracing, cost tracking, incident response, and root-cause analysis.
- Design and maintain scalable data infrastructure using PostgreSQL and Azure Data Lake Storage/Blob, including schemas, access controls, lineage, retention, and auditability.
- Establish strong engineering practices and CI/CD standards, including code reviews, security checks, dependency management, and reproducible environments.
- Partner with clinical experts, statisticians, and other domain specialists to translate complex requirements into clear, testable technical specifications.
Required Skills & Experience
- 8+ years of senior-level engineering experience across Software, Data, Machine Learning Engineering, or related fields, with a strong track record of delivering and supporting production systems.
- Hands-on experience building AI-enabled pipelines, intelligent workflows, and production multi-agent systems, including orchestration, parallel execution, state management, retries, verification, failure recovery, and cost/latency optimization.
- Advanced Python 3.12 development skills, including typed, tested, maintainable production code using tools such as pytest, ruff, and pre-commit.
- Experience developing agents with Claude, including Claude Agent SDK or Claude Code, tool integration, prompt design, structured outputs, context management, and safeguards.
- Strong knowledge of LLM testing and evaluation, including golden datasets, deterministic testing, regression testing, evaluation thresholds, and release gates.
- Solid experience with Git-based development and CI/CD, including pull requests, code reviews, GitHub Actions or Azure DevOps, Docker, versioning, environment promotion, and rollback.
- Experience with PostgreSQL and Azure data platforms, including Azure Data Lake Storage/Blob, Parquet, pandas, pyarrow, schema design, migrations, data validation, and secure handling of sensitive information.
Preferred Qualifications
- Experience delivering software in regulated or validated environments, including GxP or similar settings, with strong knowledge of traceability, audit trails, change control, validation, access controls, and release management.
- Experience working within pharmaceutical or clinical environments, with the ability to understand and discuss SAS or R clinical programs.
- Familiarity with Domino Data Lab or similar compute platforms, as well as Azure Key Vault, managed identities, and Azure DevOps Test Plans.
- Exposure to advanced AI engineering capabilities such as MCP servers, long-document retrieval, prompt caching, agent observability, and distributed tracing
Skills
Sign in to read and join the discussion.