AI Research Engineer

We're seeking an AI Research Engineer to own model evaluations end-to-end, turning ambiguous capabilities into rigorous, reproducible benchmarks. Working closely with researchers, you will design datasets, analyze model behavior at scale, and directly shape the training lifecycle.

Responsibilities:

• Design rigorous benchmarks for reasoning, coding, and agentic capabilities across C/C++, Rust, embedded systems, hardware specifications, and engineering tasks.
• Develop deterministic evaluators using compilers, emulators, simulators, static analysis, formal verification, and hardware-in-the-loop systems.
• Build distributed infrastructure to run evaluations reliably against models and live training checkpoints.
• Diagnose regressions and anomalous results, separating model failures from issues in prompts, data, evaluators, or infrastructure.
• Design robust metrics, difficulty curricula, contamination-resistant tasks, and experiments around prompting, sampling, and scaffolding.
• Turn evaluation failures into targeted datasets, reward signals, and new training objectives in collaboration with RL researchers.
• Create dashboards, experiment tracking, and evaluation libraries that make model performance easy to understand and trust.
• Own the development of cutting-edge infrastructure from day one.
• Solve complex engineering challenges that directly shape our product roadmap.
• Accelerate your career alongside elite researchers in a fast-paced environment.
• Work closely with a brilliant, collaborative team of industry pioneers.

Requirements:

• Strong Python programming and software engineering fundamentals.
• Experience building benchmarks, evaluation systems, automated graders, research infrastructure, or similar testbeds.
• Strong understanding of LLMs and modern evaluation methodologies.
• Strong analytical and experimental thinking; you care deeply about whether a metric actually measures the intended capability.
• Experience debugging complex systems and investigating unexpected experimental results.
• Strong written and verbal communication.