Staff AI Engineer: Build Production ML Evaluation Tools
Posted Updated
Who recruits here Pro
Up to three recruiters at Doctolib with their work email, so you can write to a person.
Part of Pro. How it works
Description
Doctolib is hiring for a senior software/ML engineer to help design and operate advanced evaluation tooling for AI products. You will collaborate with product, design, and business teams to assess current evaluation quality and implement improvements across APIs, services, and CI/CD integrations.
You will design strategies combining deterministic checks, LLM-based judgment, and human review, and build production sampling and distribution monitoring capabilities.
Design, ship, and operate production-grade evaluation tooling (APIs, services, CI/CD integrations). Collaborate with product teams to assess current evaluation methods and define improvements. Design evaluation strategies for agentic systems, combining deterministic checks and LLM-based review. Build capabilities for production sampling, dataset representativeness monitoring, and test generation. Define quality metrics, representative datasets, and reliable quality gates across the lifecycle. Create reusable, self-service platform capabilities and measure adoption and coverage. Contribute to cross-functional initiatives shaping Doctolib's AI platform and best practices. Have 6+ years of experience as a software engineer, with significant Python experience, or as a machine learning engineer. Experience building or operating evaluation for LLM-powered or agentic systems. Understand and promote software development best practices. Product-oriented, able to work with multiple teams and drive adoption. Health insurance Paid vacation 25 days RTT (reduction of working time) Mental health services Work from abroad up to 10 days per año Lunch vouchers Transport reimbursement 50% DoctoGrowth program ParentCare program Disability supportSkills
Sign in to read and join the discussion.