Skip to content
Search jobs inside your AI agent. Claude, ChatGPT, Claude Code, Codex, Gemini, Cursor and Grok
Connect

Employment

Agent Evaluation Engineer

Posted Updated

Description

Employment seeks experienced software developers and testers to help build a realistic developer environment and evaluation tasks for AI coding agents. You will craft prompts, define success criteria, and write tests that accept multiple valid approaches.

This project uses Python/FastAPI, TypeScript/React, Docker, PostgreSQL, Kafka and Redis; collaboration with a remote team is required and English proficiency (B2+) is essential.

Create realistic developer environments including codebase, infra, tickets and docs. Design tasks from intermediate states and define what ‘solved’ means. Write tests that accept multiple valid approaches and reject incorrect ones. Iterate on tasks based on QA feedback to ensure fairness and robustness. 5+ years of professional software development experience. Strong hands-on with Python (FastAPI) and JS/TS (React). Experience with Docker, PostgreSQL, Kafka and Redis. Proven testing experience (functional/integration tests) and English proficiency (B2+).

Skills

Apply

See also