Skip to content
Search jobs inside your AI agent. Claude, ChatGPT, Claude Code, Codex, Gemini, Cursor and Grok
Connect

Aleph Alpha GmbH

Senior AI Researcher – Pre-training

Posted Updated

Description

You will own technical decisions for model architecture, optimization, stability, and scaling. You will turn mathematical reasoning and experimental results into training decisions, from proxy experiments to multi-thousand-GPU runs. You will optimize training recipes and PyTorch architectures, develop scaling methodologies, investigate convergence and distributed-system failures, and align model design with hardware constraints. You will write maintainable, well-tested code, support reproducible experiments, and collaborate regularly on site in Heidelberg.

Responsibilities

  • Optimize training recipes, including optimizers, schedules, and initialization
  • Design PyTorch-based architectural improvements
  • Develop hyperparameter scaling laws and scale-up methodologies
  • Use proxy experiments to predict large-scale training behavior
  • Investigate convergence issues and training instability
  • Resolve distributed-system failures, including communication bottlenecks, race conditions, and synchronization errors
  • Align model lifecycle decisions with hardware constraints, memory bandwidth, and communication topologies
  • Implement complex model architectures efficiently and reliably
  • Write maintainable, well-tested code
  • Support reproducible experimentation workflows
  • Collaborate regularly on site in Heidelberg

Requirements

  • Python proficiency
  • PyTorch training workflows
  • Machine learning research
  • Software engineering
  • Mathematical reasoning about optimization, scaling behavior, and training dynamics
  • Transformer training dynamics
  • Large distributed training jobs
  • Rigorous experimental design
  • Pre-training large models with 7B+ parameters on 100+ GPU clusters
  • Maintainable and well-tested code
  • Reproducible experimentation
  • Complex model architecture implementation
  • Distributed-system debugging
  • Ability to work in Germany
  • Regular on-site collaboration in Heidelberg

Benefits

  • 30 days of paid vacation
  • Fitness and wellness offerings via Wellhub
  • Mental health support through nilo.health
  • Subsidized company pension plan
  • Subsidized Germany-wide transportation ticket
  • Budget for additional technical equipment
  • Flexible working hours
  • Hybrid working model
  • JobRad bike lease

Skills

Apply

See also