Lead ML Engineer (USA)
Summary
Lead ML Engineer owning the full ML lifecycle—training, evaluation, inference, and deployment of large AI models into production using Python, PyTorch/JAX on GPU infrastructure.
Lead ML Engineer
Location: Remote, Nationwide
Our client is building a new generation of intelligent software designed to turn advanced machine learning into dependable, everyday product experiences. This is an opportunity to take a leadership role in the systems that move sophisticated models out of experimentation and into production. The Lead ML Engineer will have substantial ownership across model adaptation, training infrastructure, evaluation, inference, and deployment while helping a small, highly capable engineering team solve difficult problems at scale.
This Role Offers:
- A hands-on technical leadership role with meaningful influence over engineering decisions and delivery.
- A remote-first environment built around autonomy, trust, rapid iteration, and strong technical judgment.
- Cash and equity compensation structured around the individual candidate.
- Health, dental, and life insurance options, along with a 401(k) match and equity participation.
- The chance to work across research, infrastructure, and product engineering rather than being limited to a single layer of the ML stack.
Focus:
- Lead the machine learning lifecycle from data preparation and training through evaluation, serving, deployment, and continuous production improvement.
- Develop and refine processes for tailoring advanced AI models through specialized training, optimization, and knowledge-transfer techniques.
- Engineer GPU-based training and serving environments with careful attention to memory utilization, throughput, response time, scalability, reliability, and infrastructure cost.
- Establish rigorous evaluation practices for measuring model quality, resilience, safety, and behavioral performance throughout the release lifecycle.
- Build data workflows that effectively use both real-world and synthetically generated information to improve training quality.
- Work alongside research and product engineering teams to convert promising model capabilities into maintainable, user-facing functionality.
- Drive production readiness through deployment planning, monitoring, troubleshooting, capacity management, and fast resolution of model and infrastructure problems.
- Balance technical quality with practical delivery needs, using measurable production results to determine where models and systems should improve next.
- Provide hands-on technical leadership by helping engineers solve difficult problems, maintain alignment, and uphold a strong engineering standard.
Skill Set:
- Demonstrated success delivering machine learning systems that have operated in production and served actual users.
- Broad ownership experience spanning ML data workflows, training, evaluation, inference, deployment, and production operations.
- Strong understanding of large models, including practical experience identifying and addressing model limitations and unexpected production behavior.
- Practical experience applying advanced techniques to customize, refine, and optimize AI models for specialized use cases and performance requirements.
- Advanced Python development skills with experience using PyTorch, JAX, or comparable machine learning frameworks.
- Strong knowledge of GPU training and inference, including techniques for improving memory utilization, latency, throughput, and scalability.
- Production engineering discipline with an emphasis on software correctness, maintainability, monitoring, reliability, operating cost, and safety.
- Ability to independently navigate ambiguous technical challenges, make sound tradeoffs, and carry complex initiatives through production delivery.
- Clear communication skills and the ability to provide technical direction, unblock colleagues, and collaborate effectively within a small, high-trust engineering environment.
About Blue Signal:
Blue Signal is an award-winning, executive search firm specializing in various specialties. Our recruiters have a proven track record of placing top-tier talent across industry verticals, with deep expertise in numerous professional services. Learn more at bit.ly/46Gs4yS