Staff ML Systems Engineer, Distributed Systems
We are seeking a Staff ML Systems Engineer to architect and build the distributed infrastructure that powers large-scale machine learning workflows across the organization.
This role sits at the intersection of machine learning, distributed systems, and platform engineering. You will be responsible for designing scalable systems that support data processing, model training, evaluation, and post-processing pipelines while enabling ML teams to efficiently develop, operate, and scale production-grade workflows.
You will play a critical role in defining the architectural patterns, tooling, and infrastructure that underpin our machine learning platform.
What You'll Get To Do
- Design and build scalable distributed machine learning pipelines across data processing, model training, evaluation, and post-processing workflows.
- Architect distributed execution systems, including parallelization strategies, workload scheduling, resource allocation, and fault tolerance mechanisms.
- Develop reusable abstractions, frameworks, and libraries that simplify distributed pipeline development.
- Optimize performance across distributed CPU and GPU environments, improving throughput, utilization, and reliability.
- Design systems that effectively manage data partitioning, memory utilization, serialization overhead, and compute efficiency.
- Partner closely with ML engineers, data engineers, and infrastructure teams to productionize research workflows and enable large-scale model development.
- Establish best practices and engineering standards for distributed machine learning infrastructure.
- Evaluate and guide decisions around distributed computing frameworks, infrastructure technologies, and system design trade-offs.
- Improve observability, debugging, monitoring, and operational tooling for distributed systems at scale.
What You Have
- 5+ years of experience building distributed systems, backend infrastructure, machine learning platforms, or large-scale data processing systems.
- Strong Python programming skills, including experience with concurrency, performance optimization, and systems development.
- Experience with distributed computing frameworks such as Ray, Spark, Dask, Flink, or similar technologies.
- Experience designing and scaling data pipelines or machine learning workflows.
- Strong system design skills with demonstrated expertise in scalability, reliability, and performance optimization.
- Experience diagnosing and resolving bottlenecks in distributed environments.
- Ability to work cross-functionally and drive technical decisions across multiple teams.
The Extras That Set You Apart
- Experience building infrastructure for machine learning training and inference systems.
- Familiarity with modern ML frameworks such as PyTorch or TensorFlow.
- Experience with multi-node or multi-GPU training architectures, including DDP, FSDP, DeepSpeed, or similar technologies.
- Experience operating Kubernetes-based infrastructure and large-scale cloud systems.
- Deep understanding of distributed systems concepts including data locality, serialization costs, scheduling, and resource management.
- Experience with distributed debugging, observability, and workflow orchestration platforms.
- Proven ability to establish technical direction and influence architecture across organizations.
Skills
As published by lever · 12 questions
Basics
Which location are you applying for?, Resume/CV, Full name, Email, Phone, Current location, Current company, LinkedIn URL, Twitter URL, GitHub URL, Portfolio URL, Field AI
Short answers (8)
- First Name
- Middle Name optional
- Last Name
- Address Line 1
- Address Line 2 optional
- City
- State
- Zip Code
Pick from a list (4)
- Are you legally authorized to work in the country where this position is located?
- Will you now or in the future require employer sponsorship for a work visa or employment authorization to work in the country where this position is located?
- Are you at least 18 years of age and legally eligible to perform the duties of this position?
- I certify that the information provided in this application is true, complete, and accurate to the best of my knowledge. I understand that providing false or misleading information may result in the rejection of my application or termination of employment if discovered after hire. optional