Senior Python Engineer - AI Task Design (Project-based)
Summary
Design evaluation tasks and prompts for AI coding agents at Mindrift: create realistic developer environments, define success criteria, and write tests that accommodate multiple correct approaches. The role focuses on guiding task quality rather than writing full code, using Python and test-design skills.
Mindrift is building a dataset to evaluate AI coding agents by creating realistic developer environments and evaluation tasks. You design prompts, define success criteria, and write tests that accommodate multiple correct approaches.
This role focuses on guiding task quality, not writing full code from scratch, with engagement-based compensation. The project requires 8+ years in software development and experience writing tests, plus the ability to craft fair, robust evaluation scenarios and