Data Scientist (AI Data & LLM Specialist)

You will establish and document scalable data annotation strategies and enforce quality metrics including inter-annotator agreement. You will research and prototype optimal data formats and preprocessing for fine-tuning LLMs, prepare datasets for tokenization, embedding generation, and NER, and build automated quality analysis and feedback loops. You will use APIs/SDKs to automate annotation and active learning and collaborate with engineers to implement data processing pipelines, producing clear documentation for technical and non-technical audiences.

Responsibilities

• Design and document formal data annotation strategies and guidelines
• Define and enforce quality metrics including inter-annotator agreement
• Research, define, and prototype data formats and preprocessing for LLM training and fine-tuning
• Establish automated processes and metrics for data quality analysis and feedback
• Use APIs and SDKs to automate data annotation and active learning loops
• Collaborate with engineering to guide implementation of data processing pipelines

Requirements

• Proven experience as a Data Scientist or Machine Learning Engineer focused on data quality and preparation
• Strong understanding of data labeling methodologies and experience with data annotation platforms and workflows
• Experience preparing datasets for training and fine-tuning LLMs, including tokenization, embedding generation, and NER
• Proficiency in Python and data science libraries such as Pandas, NumPy, and Scikit-learn
• Experience with spaCy and Hugging Face
• Experience using APIs and SDKs to automate data annotation and active learning loops
• Excellent communication skills and ability to create clear documentation for technical and non-technical audiences
• Nice-to-have: experience with audio data processing
• Nice-to-have: familiarity with data annotation platforms and tools
• Nice-to-have: knowledge of modern MLOps principles and practices
• Nice-to-have: experience with RLHF and large language model data curation

Benefits

• Flexible work schedule with synchronous and asynchronous collaboration
• Quarterly in-person meetups
• Equity
• Benefits package