Applied Research Evals and Data
You will work directly with customers to understand their workflows, data sources, and bottlenecks. You will prototype AI agents, data pipelines, and evaluation harnesses, then translate customer insights and evaluation results into research and product direction. You will design post-training and reinforcement-learning methods, build evaluation and verification systems, and integrate applied data into model improvement. You will also develop agent capabilities, distributed training and inference pipelines, and production observability.
Responsibilities
- Work with customers to understand workflows, data sources, and bottlenecks
- Prototype agents, data pipelines, and evaluation harnesses for customer use cases
- Translate customer insights and evaluation results into roadmap and research direction
- Design and implement RL and post-training methods for domain-specific tasks
- Build evaluation harnesses and verifiers for reasoning, robustness, and agentic behavior
- Integrate applied data collection and analytics into post-training workflows
- Prototype multi-agent and memory-augmented systems
- Extend and integrate agent frameworks
- Architect and maintain distributed training and inference pipelines
- Develop observability and monitoring for production deployments
Requirements
- Machine learning engineering experience in post-training, reinforcement learning, or large-scale model alignment
- Experience with applied data workflows and evaluation frameworks for large models or agents
- Expertise in distributed training and inference frameworks such as vLLM, sglang, Ray, or Accelerate
- Experience deploying containerized systems at scale using Docker, Kubernetes, and Terraform
- Research contributions through publications, open-source contributions, or benchmarks
- Knowledge of reasoning, measurement, and agentic AI systems
Benefits
- Equity incentives
- Flexible work, remote or San Francisco
- Visa sponsorship
- Relocation support
- Professional development budget
- Team off-sites
- Conference attendance