Skip to content
Search jobs inside your AI agent. Claude, ChatGPT, Claude Code, Codex, Gemini, Cursor and Grok
Connect

Mindrift

Senior Python Engineer - AI Task Designer & Evaluator

Posted Updated

Description

Mindrift is seeking experienced software developers to build and evaluate AI coding tasks for leading tech companies. You will create realistic environments, craft tasks from intermediate states, and write comprehensive tests to validate AI agent solutions.

This project-based role emphasizes quality and fairness, with flexible timing and an emphasis on English proficiency (B2+). You will work remotely on a global scale, designing innovative evaluation components for AI coding systems.

Build realistic developer environments - a virtual company with codebase, infrastructure, and context (tickets, docs, conversations) that forms a believable development history. Design tasks from intermediate states of these environments - craft the prompt, define what 'solved' means, and ensure the task is solvable by an AI agent. Write tests that verify agent solutions - accept all valid approaches and reject incorrect ones, neither too strict nor too lenient. Iterate on tasks and tests based on QA feedback - review agent solutions, analyze failures, and refine until the evaluation is fair and robust. 8+ years in software development. Experience writing tests (functional, integration). English proficiency – B2+.
What Senior Design jobs ask for — and how much of it you have →
Apply

See also

Design jobs by country — openings, pay and top skills →