AI Researcher - Pre-training
Posted Updated
Description
You will design and run experiments on training recipes, optimizers, schedules, initialization, and PyTorch-based model architectures. You will evaluate convergence, stability, and efficiency; analyze proxy experiments and scaling behavior; monitor and debug large-scale distributed training runs; reproduce relevant research; and collaborate across compute, data, evaluation, and post-training work to inform pre-training decisions.
Responsibilities
- Design and run ablations on training recipes and implement PyTorch-based architectural changes.
- Evaluate convergence, stability, and training efficiency.
- Run and analyze small-scale proxy experiments and contribute to scaling methods.
- Monitor large-scale training runs and investigate convergence and distributed training failures.
- Follow research literature, reproduce promising results, and propose ideas for testing.
- Collaborate with compute performance, data, evaluation, and post-training functions.
Requirements
- Proficiency in Python and hands-on PyTorch experience.
- Strong machine-learning foundation demonstrated through research, open-source contributions, or substantial projects.
- Solid mathematical background in linear algebra, probability, and optimization.
- Understanding of transformers and experience pre-training LLMs from scratch.
- Ability to design careful experiments and communicate evidence and results clearly.
- Ability to learn unfamiliar technical areas quickly.
- Ability to write clean, maintainable code and produce reproducible experiments.
- Clear communication and responsiveness to feedback.
Skills
Sign in to read and join the discussion.