AI Research Engineer: Distributed Training & MoE Models
Posted Updated
Description
A cutting-edge AI safety startup is seeking an AI Research Engineer in Paris to tackle complex training challenges for LLMs and VLMs. The role focuses on designing next-generation architectures, building multimodal training pipelines, and optimizing performance through advanced distributed training frameworks like Megatron and DeepSpeed. Candidates should possess a deep understanding of distributed training concepts and Triton kernels. This is a full-time position with a mid-senior level requirement, offering the opportunity to drive innovation in advanced AI systems.
Train LLMs and VLMs using advanced distributed training frameworks.
Design and implement next-generation architectures, including Mixture-of-Experts (MoE).
Build and maintain complex multimodal training pipelines for text, images, and audio data.
Develop custom Triton kernels to optimize training and inference bottlenecks.
Drive innovation exploring new model architectures and hyperparameter strategies.
Deep understanding of tensor parallelism, pipeline parallelism, expert parallelism, ZeRO, and FSDP.
Ability to write Triton kernels for custom operators or fundamental layers like attention and MLP.
Experience working with multimodal systems like Llava-style and Flamingo-style.
Skills
Sign in to read and join the discussion.