Senior AI Engineer
Summary
Senior AI Engineer in Tel Aviv building AI agent training and evaluation environments: simulated systems, tool-use/web/computer-use agent scenarios, automated evaluation and grading mechanisms, and scalable queue-driven infrastructure. Core stack is Python, Docker, Kubernetes and AWS, paired with LLM/agent and reinforcement-learning experience.
We are looking for a Senior AI Engineer to design, build, and scale advanced AI evaluation and training environments.
In this role, you will develop realistic, interactive environments that simulate real-world systems and scenarios, enabling advanced AI agents to be tested, evaluated, and improved. You will combine strong backend engineering skills with an interest in AI, reinforcement learning, and complex system design.
You will take ownership of end-to-end engineering challenges, from environment architecture and evaluation mechanisms to scalable infrastructure, working closely with AI researchers to turn research concepts into reliable, production-ready systems.
Responsibilities:
- Design and build end-to-end environments for training and evaluating AI agents, including simulated systems, scenarios, and automated evaluation mechanisms.
- Develop environments for tool-use, web, and computer-use agents.
- Ensure environments are secure, isolated, reliable, and resistant to unintended shortcuts or reward manipulation.
- Analyze agent execution traces to identify weaknesses, unexpected behavior, and opportunities for improvement.
- Design and develop evaluation systems that assess the quality, reliability, and effectiveness of AI training tasks.
- Establish quality standards for evaluation scenarios, focusing on realism, accuracy, and training value.
- Continuously improve evaluation mechanisms based on research findings, user feedback, and model performance.
- Build and scale infrastructure for running AI environments and evaluation pipelines.
- Develop queue-driven execution systems, isolated execution environments, and scalable grading mechanisms.
- Package and integrate environments into external training and evaluation pipelines.
- Work closely with AI researchers to translate research ideas into functional, production-ready environments.
- Use insights from model evaluation and training runs to improve environment design.
- Participate in technical design reviews, code reviews, and engineering best practices.
Requirements:
- 6+ years of experience as a Backend Engineer, building and maintaining production systems.
- BSc in Computer Science or a related field, preferably from a leading university.
- Strong Python skills and hands-on production experience with Docker, Kubernetes, and cloud infrastructure (AWS preferred).
- Experience designing distributed systems and working with message queues.
- Proven ability to own systems end-to-end, from architecture and design through production.
- High level of autonomy and ability to solve complex, undefined problems independently.
- Fluent written and spoken English.
- Academic background in AI or Machine Learning, including a degree, thesis, or research experience.
- Hands-on experience building AI-powered applications using LLMs, agents, or evaluation frameworks.
Skills
Sign in to read and join the discussion.