Senior Data Annotator
Posted Updated
Description
Responsibilities:
- Own day-to-day annotation quality for document classification, sensitive data detection and other ML datasets.
- Review, validate and correct annotations, and give annotators feedback that improves accuracy, consistency and productivity.
- Resolve complex and ambiguous samples, and settle disagreements between annotators so similar cases are labelled consistently.
- Develop and maintain annotation guidelines, labelling standards and decision documentation as recurring errors surface.
- Build quality control processes, including sample-based audits, annotation reviews and consistency checks.
- Build and maintain gold standard datasets for model evaluation, and support dataset sampling and edge case discovery.
- Track quality metrics, error categories and progress, and coordinate review workflows to deliver on time without compromising quality.
- Work with ML QA and ML engineers to investigate model errors caused by ambiguous labels, missing annotations or inconsistent ground truth.
Requirements:
- 3 to 5 years in data annotation, annotation QA, data validation or a related field.
- Bachelor's in Computer Science, IT, Data Science or a related discipline, or equivalent experience.
- Proven experience annotating and reviewing complex datasets against labelling guidelines.
- Strong grasp of data quality principles, ground truth creation and systematic quality audits.
- Attention to detail, clear communication, and ease working with annotators and technical teams.
Nice to Have:
- Experience with document classification, OCR, NLP, NER or information extraction.
- Familiarity with detecting PII, credentials, secrets or other sensitive information.
- Experience annotating documents, text and images on annotation platforms.
- Experience with quality rubrics, inter-annotator agreement and sampling strategies.
- Basic Python or SQL for dataset inspection.
Skills
Sign in to read and join the discussion.