Member of Technical Staff, Realm Lab AI Research
Description
- Own research and evaluation projects from task design and dataset development through benchmark validation and performance analysis.
- Design and validate capability evaluations, reinforcement learning environments, and benchmark suites for computer use, agent workflows, and emerging model capabilities.
- Develop datasets, task specifications, evaluation rubrics, and methods that improve post-training data quality and model assessment.
- Analyze model behavior through failure analysis, error taxonomies, and identification of capability gaps.
- Build reproducible evaluation pipelines and feedback loops for iterative model development.
- Contribute to benchmarks, research papers, technical reports, and open research artifacts.
- Pay: $100 to $130 per hour.
- Work arrangement: Contract work listed as part-time.
- Location: Open to candidates in the countries listed for this role across North America, South America, Europe, Asia, Africa, and the Middle East.
- Language: Professional fluency in English.
- Requirements: A strong technical background in computer science, machine learning, AI, or a related field; experience designing ML evaluations, benchmark suites, research datasets, or experimental frameworks; and proficiency in Python.
- Requirements: Experience building data pipelines, evaluation tools, or ML research infrastructure; understanding of large language models, foundation models, post-training techniques, and evaluation methods; and the ability to turn complex workflows into structured tasks, measurable criteria, and research experiments.
- Requirements: Strong analytical skills in model failure analysis, experimental design, and evaluation quality.
- Helpful background: Experience with reinforcement learning environments, reward modeling, simulators, feedback-driven training systems, computer use agents, tool use, browser automation, multi-step agent workflows, multimodal reasoning, long-context evaluation, instruction following, deep research benchmarks, human-in-the-loop evaluations, expert-generated datasets, post-training data, AI research publications, open-source benchmarks, or evaluation frameworks.
You apply here in about two minutes and finish on the hiring platform. The next step is a short AI interview (technical roles include a coding challenge). Passing it is how you qualify for paid tasks, so take it soon after you are invited.
The hiring platform's application for this job asks for your resume, your LinkedIn profile, your phone number, your location and answers to 4 screening questions.
About AI training workAI training work is the human side of building artificial intelligence. People create and review datasets, evaluate model responses, and design tests that help AI systems improve, so roles requiring research and evaluation expertise pay for that specialized knowledge.
Skills
Sign in to read and join the discussion.