Skip to content
Search jobs inside your AI agent. Claude, ChatGPT, Claude Code, Codex, Gemini, Cursor and Grok
Connect

Matters

New

Senior Data Annotator

Posted Updated

Description

Responsibilities:

  • Own day-to-day annotation quality for document classification, sensitive data detection and other ML datasets.
  • Review, validate and correct annotations, and give annotators feedback that improves accuracy, consistency and productivity.
  • Resolve complex and ambiguous samples, and settle disagreements between annotators so similar cases are labelled consistently.
  • Develop and maintain annotation guidelines, labelling standards and decision documentation as recurring errors surface.
  • Build quality control processes, including sample-based audits, annotation reviews and consistency checks.
  • Build and maintain gold standard datasets for model evaluation, and support dataset sampling and edge case discovery.
  • Track quality metrics, error categories and progress, and coordinate review workflows to deliver on time without compromising quality.
  • Work with ML QA and ML engineers to investigate model errors caused by ambiguous labels, missing annotations or inconsistent ground truth.


Requirements:

  • 3 to 5 years in data annotation, annotation QA, data validation or a related field.
  • Bachelor's in Computer Science, IT, Data Science or a related discipline, or equivalent experience.
  • Proven experience annotating and reviewing complex datasets against labelling guidelines.
  • Strong grasp of data quality principles, ground truth creation and systematic quality audits.
  • Attention to detail, clear communication, and ease working with annotators and technical teams.


Nice to Have:

  • Experience with document classification, OCR, NLP, NER or information extraction.
  • Familiarity with detecting PII, credentials, secrets or other sensitive information.
  • Experience annotating documents, text and images on annotation platforms.
  • Experience with quality rubrics, inter-annotator agreement and sampling strategies.
  • Basic Python or SQL for dataset inspection.

Skills

Apply

See also