Software Engineer
Description
We are looking for AI Coding Experts to design and build complete, Dockerized Reinforcement Learning Environments (RLEs) used to evaluate and train AI coding agents. This is a hands-on engineering role focused on building realistic coding scenarios, reference solutions, verifiers, and test harnesses. It is not a prompt-writing or data-annotation role.
Responsibilities:
- Design realistic, self-contained Dockerized coding scenarios from scratch.
- Build repository fixtures and clear task instructions for AI coding agents.
- Write working Python reference solutions that demonstrate the correct implementation.
- Develop deterministic verifiers, pytest harnesses, and evaluation logic.
- Review AI-agent trajectories to identify where and why an agent failed.
- Validate whether solutions represent genuine fixes rather than superficial patches.
- Revise and improve scenarios based on structured quality-review feedback.
- Ensure environments are reproducible, offline, and technically reliable.
Requirements:
- Strong hands-on Python development skills.
- Experience writing real software, including reference solutions and automated tests.
- Strong proficiency with Docker and containerised development environments.
- Good understanding of software security and vulnerabilities.
- Ability to distinguish robust technical fixes from superficial patches.
- Strong debugging and code-review skills.
- Ability to read AI-agent execution traces and diagnose failures.
- Strong analytical and adversarial problem-solving mindset.
- Comfortable working independently on technically complex tasks.
Preferred Skills:
- RLHF or AI evaluation experience.
- Benchmark or evaluation-environment authoring.
- AI coding-agent evaluation.
- Red teaming, AppSec, or penetration testing.
- CTF or security challenge experience.
- Linux and shell scripting.
- Experience with Node.js, Go, or Java.
- Experience creating test harnesses, graders, or automated verification systems.
Ideal Profiles:
- Software Engineers
- Security Engineers
- Application Security Engineers
- ML / AI Engineers
- AI Evaluation Engineers
- Benchmark Authors
- Red Team Engineers
- Penetration Testers
- Coding Agent / AI Agent Engineers