Data Analyst (AI Evaluation)
Summary
Contractor Data Analyst reviewing datasets and AI model outputs for accuracy and guideline adherence, helping train next-generation AI systems. Day to day: applying evaluation rubrics, flagging errors and inconsistencies, and documenting feedback. Remote, US-based; spreadsheet fluency required, SQL a plus.
Role Title: Data Analyst (AI Evaluation)
Role Type: Contractor
Location: Remote, US-based.
micro1 is engaging Data Analysts to contribute their knowledge and expertise to a customer’s data quality project. In this role, you'll apply your expertise to help train next-generation AI systems. Your work will shape how models learn, reason, and perform through high-quality, real-world input. No prior experience in AI is required — your domain knowledge is what matters.
Scope of Work
- Review datasets and detailed task outputs for accuracy, completeness, and adherence to guidelines.
- Identify and flag inconsistencies, errors, and outputs that deviate from provided instructions.
- Apply structured rubrics and evaluation criteria consistently across large data sets.
- Exercise sound judgment and document rationale in ambiguous scenarios where guidelines are unclear.
- Escalate patterns or recurring data issues for further analysis rather than addressing them individually.
- Provide clear, actionable written and verbal feedback pinpointing issues and suggesting resolution paths.
- Maintain meticulous attention to detail while working through high volumes of repetitive data review assignments.
Preferred Qualifications
- 3–5+ years of experience in data review, data quality, quality assurance, or analytical roles.
- Proven expertise in reviewing, validating, and critiquing data or deliverables based on explicit criteria.
- Exceptional ability to follow detailed written instructions and apply complex evaluation rubrics precisely.
- Demonstrated comfort with ambiguity, including making justifiable decisions when guidelines are incomplete.
- Advanced attention to detail and error-spotting capabilities across repetitive or high-volume work.
- Strong written and verbal communication skills for documenting findings and providing feedback.
- Baseline fluency with spreadsheets (sorting, filtering, simple formulas); SQL or hands-on data querying is a plus.
- Experience with data annotation, labeling, or rubric-based evaluation is advantageous.