Senior Applied Machine Learning Engineer (Evals, Agents, Data) - Bengaluru
Summary
Owns AI quality for an agentic video editor, building evaluation datasets, offline/online evals, and feedback loops to improve agent performance and fix production failures.
Senior Applied ML Engineer to own AI quality for Cardboard’s agentic video editor, building evaluation datasets, offline/online evals, regression checks, and feedback loops that turn production failures into measurable improvements.
Company Details
Cardboard is an AI-first video editor building agentic tools that understand user requests, work with media, and make real edits on the timeline. It is backed by a Tier-1 global fund, YC, and founders of billion-dollar companies. Website:
Requirements
Experience shipping and operating an LLM or agent system used by real customers.
Strong software engineering skills in TypeScript or Python, with ability to work across both.
Experience building evaluations, datasets, experiments, or AI quality systems.
Strong product judgment and ability to turn vague AI quality issues into measurable problems.
Ability to work across data, evaluation methods, model selection, and fine-tuning.
Strong ownership as a senior individual contributor.
Bonus: Experience with multimodal AI, video, media, or creative software.
Bonus: Experience with human labeling, model graders, or fine-tuning.
Bonus: Strong understanding of experiment design and statistics.
Responsibilities
Define quality standards for Cardboard’s agent.
Build trusted evaluation datasets from real product usage.
Build offline and online evaluations using automated checks, model graders, and human review.
Analyze real agent runs and identify recurring failure patterns.
Improve agent quality through better data, evaluation methods, model selection, and fine-tuning.
Build regression checks and release gates for important agent changes.
Track AI quality alongside latency and cost.
Partner with product and engineering teams to ship measurable improvements.
Job Details
Bengaluru, India
Interview Process
Recruiter Screen
Technical Interview
ML & Evaluation Deep Dive
Product & Engineering Interview
Final Interview
Important Note
ClanX is a recruitment partner, helping Cardboard hire Senior Applied ML Engineer, Evals & Data.
As published by recruitee
Full name, Email, CV, Phone
- How many years of experience do you have as a Machine Learning Engineer? choose one
- Are you willing to relocate to Bengaluru and work from office? yes / no
- What is your notice period? (We are looking for candidates who can join within 30 days) choose one
- Is your notice period negotiable? choose one
- Do you have experience working at a product-based company? yes / no
- Are you a citizen of India and currently based in India? yes / no
- What is your current annual CTC range? (INR) choose one
- Please share the figures for your current CTC breakup: Fixed, Variable, ESOPs (with vesting), and any other components
- What is your expected annual CTC? (INR)
- Help us with your LinkedIn profile link
- Tell us about an agent or agentic system you have built or worked on. What was the agent expected to do, and how did you measure whether it was working well? written answer
- How have you designed or used evals for an LLM or agent? What did you evaluate, and how did the results influence your decisions? written answer
- Tell us about a time when your evals showed that an agent was failing. How did you identify the root cause and improve the system? written answer
- What interests you about the company Cardboard (http://usecardboard.com/)? Tip: Research the company and share a specific reason. Generic or copy-pasted answers will be rejected. written answer
- Why do you want to leave your current role? written answer
- Where did you find this job opportunity? (e.g., LinkedIn, Twitter, WhatsApp, etc). If someone referred you, please mention their name and contact number so we can thank them. (Write NA if not applicable). written answer