Agent Evaluation & Evolution Machine Learning Engineer Graduate (AML-Ark-US) - 2027 Start
Summary
Develop and optimize LLM agent evaluation systems and automated judging pipelines. The role involves analyzing execution traces, building benchmark datasets, and implementing closed-loop feedback to improve system reliability and alignment.
- Analyze agent execution traces
- Build LLM benchmark datasets
- Collaborate with research platform and product teams to deploy methods
- Design LLM agent evaluation systems
- Develop automated judging pipelines
- Extract failure patterns from logs
- Implement closed loop feedback optimization
- Improve system reliability and alignment
Perks/Benefits:
- 401k match
- Dental insurance
- Life insurance
- Long-term disability
- Medical insurance
- Paid Holidays
- Paid parental leave
- Paid personal time
- Paid sick days
- Short-term disability
- Vision insurance
- Wellbeing benefits