Software Engineering, Data Science & Systems Design Experts (AI Model Evaluation)

As a Technical AI Evaluator, you will assess and improve AI-generated responses to coding and software engineering tasks. Your work will directly impact how accurately and effectively AI systems solve problems, write code, and communicate technical concepts.

This is a hands-on, intellectually rigorous role suited for professionals with strong coding, analytical, and problem-solving skills.

• Evaluate AI-generated responses for accuracy, reasoning, clarity, and completeness

• Execute and test code to validate correctness and performance

• Identify bugs, inefficiencies, and logical flaws in model outputs

• Annotate responses with detailed feedback and improvement suggestions

• Assess code quality, readability, and adherence to best practices

• Fact-check technical claims using reliable and authoritative sources

• Apply structured evaluation frameworks and guidelines consistently

• Bachelor’s, Master’s, or PhD in Computer Science or a related field

• 3+ years of professional experience in software engineering, data science, or related roles

• Strong proficiency in at least two programming languages (e.g., Python, JavaScript, C++, Java, SQL, Go, Rust, etc.)

• Ability to independently solve medium to hard algorithmic problems

• Experience working with or alongside LLMs in coding workflows

• Strong attention to detail and ability to evaluate complex technical reasoning

• Experience with AI model evaluation, RLHF, or data annotation

• Background in competitive programming

• Contributions to open-source projects (merged pull requests preferred)

• Experience reviewing production-level code

• Ability to explain complex technical concepts clearly to non-experts

• You consistently identify subtle bugs, flawed logic, and edge cases

• Your feedback improves the accuracy and clarity of AI-generated code

• You produce structured, reproducible evaluation outputs

• AI systems become more reliable for real-world engineering tasks

• Independent contractor engagement

• Fully remote with flexible working hours (open to US and non-US candidates)

• Full-time or part-time contract options available

• Weekly payments based on services rendered

We consider all qualified applicants without regard to legally protected characteristics and provide reasonable accommodations upon request.