Software Engineer
Summary
On-site Software Engineer in San Francisco (hired via Raydar on behalf of a technology client) who designs datasets, scoring rubrics, and evaluation frameworks for large-scale ML model training. Day-to-day is Python-based data pipeline work, LLM evaluation/post-training, and reward modeling with external research partners.
About the company
Our client is a technology company.
The role
Raydar is recruiting for this role on behalf of our client. Design datasets, evaluation methods and measurement frameworks that shape how machine learning models are trained and assessed. Collaborate with external research partners, run fast experiments and convert findings into concrete specifications for large-scale training work.
What you'll do
- Design targeted data samples that reveal weaknesses in model behavior across a range of subject areas.
- Build and refine scoring rubrics and reward signals used in model training pipelines.
- Study how human annotators behave and run experiments to improve model capabilities.
- Develop quantitative methods for assessing dataset quality, diversity and downstream impact.
- Own the data pipelines that supply both naturally occurring and artificially generated training material.
- Work with research partners to turn their training goals into clear data and evaluation specifications.
Requirements
What we're looking for
- 1 to 4 years of experience in software or machine learning engineering.
- Strong Python skills and experience building dependable data pipelines, experimentation tooling or evaluation systems.
- Hands-on experience with language model evaluation, post-training, reward modeling or closely related ML infrastructure.
- Sound scientific judgment, including setting up well-controlled tests and drawing conclusions from imperfect results.
- Background in building or auditing training data and judging how well it supports model performance.
- Ability to learn unfamiliar domains quickly and reason across them.
Benefits
Compensation and benefits
- Base salary: USD 260,000 to 290,000 per year
- Equity
- Profit sharing
Location and work model
- San Francisco, CA, United States
- On-site
- Full-time
Skills
As published by workable · 3 questions
Basics
First name, Last name, Email, Headline, Phone, Address, Photo, Education, Experience, Summary, Resume, Cover letter
Short answers (1)
- Public LinkedIn profile URL (https://www.linkedin.com/in/...)
Pick from a list (2)
- Will you require work authorization of any kind?
- Are you able to work onsite in San Francisco?