Lead/Senior Inference Engineer
Posted
Description
This position is listed on behalf of a partner company, which manages all applications and next steps. Our partner is looking for a Lead/Senior Inference Engineer based in Canada.
The Lead/Senior Inference Engineer will build and operate the production inference infrastructure powering advanced large language models and AI-driven professional applications. This hands-on engineering role focuses on developing a reliable, scalable model-serving platform while optimizing performance, latency, and infrastructure costs. Working within a newly established engineering team, you will help shape its technical direction, engineering practices, and operational standards from the ground up. You will collaborate closely with model training, evaluation, research, and product teams to bring new model releases into production. The role offers significant autonomy and the opportunity to influence infrastructure strategy, reliability, and international platform expansion in a high-stakes, compliance-focused environment.
Accountabilities
We use an AI-powered matching process to ensure your application is reviewed quickly, objectively, and fairly against the role's core requirements. Our system identifies the top-fitting candidates, and this shortlist is then shared directly with the hiring company. The final decision and next steps (interviews, assessments) are managed by their internal team.
We appreciate your interest and wish you the best!
Why Apply Through Jobgether?
Data Privacy Notice: By submitting your application, you acknowledge that Jobgether will process your personal data to evaluate your candidacy and share relevant information with the hiring employer. This processing is based on legitimate interest and pre-contractual measures under applicable data protection laws (including GDPR). You may exercise your rights (access, rectification, erasure, objection) at any time.
We may use artificial intelligence (AI) tools to support parts of the hiring process, such as reviewing applications, analyzing resumes, or assessing responses and identifying potential inconsistencies or verification signals in application materials based on available information. These tools assist our recruitment team but do not replace human judgment. Final hiring decisions are ultimately made by humans. If you would like more information about how your data is processed, please contact us.
The Lead/Senior Inference Engineer will build and operate the production inference infrastructure powering advanced large language models and AI-driven professional applications. This hands-on engineering role focuses on developing a reliable, scalable model-serving platform while optimizing performance, latency, and infrastructure costs. Working within a newly established engineering team, you will help shape its technical direction, engineering practices, and operational standards from the ground up. You will collaborate closely with model training, evaluation, research, and product teams to bring new model releases into production. The role offers significant autonomy and the opportunity to influence infrastructure strategy, reliability, and international platform expansion in a high-stakes, compliance-focused environment.
Accountabilities
- Build, maintain, and operate production-grade model-serving infrastructure supporting large-scale AI workloads.
- Implement and optimize inference techniques, including batching, caching, quantization, and comparable approaches, to improve latency and reduce operating costs without compromising model quality.
- Strengthen platform observability, monitoring, and operational visibility as traffic volumes and deployment requirements increase.
- Develop the operational and compliance foundations required for production growth, including controls and evidence supporting the ISO 27001 compliance path.
- Establish cost-tracking and attribution mechanisms that support financial planning, as well as GPU-capacity contingency plans that help prevent service interruptions and unnecessary parallel infrastructure costs.
- Evaluate and document trade-offs between cost, latency, and reliability, communicating technical recommendations to engineering leadership and partner teams.
- Work directly with model training and evaluation teams to integrate new model releases into the serving infrastructure.
- Help establish engineering practices and tools, including on-call processes, incident reviews, onboarding, and operational documentation.
- Collaborate with research, product, and engineering stakeholders to ensure the serving platform evolves alongside model capabilities and product requirements.
- Contribute to the team's technical strategy and mentor other engineers as the function grows.
- Hands-on infrastructure engineering experience with Kubernetes or comparable technologies, including ownership of production systems beyond prototyping.
- Experience operating and maintaining model services across different workloads, including peak traffic, scaling, disaster recovery, and service reliability.
- Direct experience with model-serving infrastructure and/or inference optimization for large language models, at a scale appropriate to previous roles.
- Strong background in large-scale distributed systems, particularly production scalability, availability, and reliability.
- Proven ability to troubleshoot distributed systems and resolve production incidents under real traffic conditions.
- Demonstrated experience delivering software end to end, from technical design and implementation through deployment and ongoing operations.
- Ability to establish and uphold engineering best practices while adapting to evolving product and business requirements.
- Strong autonomy, technical judgment, and ownership, with the ability to operate as an early lead engineer and mentor colleagues as the team expands.
- Preferred qualifications:
- Experience with LLM-serving technologies such as vLLM, TensorRT-LLM, Triton, SGLang, or similar frameworks.
- Exposure to GPU-cluster capacity planning and/or multi-region or multi-cloud deployment environments.
- Experience working in regulated or compliance-intensive environments, including ISO 27001, SOC 2, or comparable standards.
- Familiarity with systems programming languages such as Rust or Go.
- Knowledge of systems-level performance optimization, including rate limiting, request routing, and identifying bottlenecks across networking, hardware, and lower-level software.
- Competitive compensation: CAD $140,000–$175,000 annual base salary for Ontario, Canada.
- Potential eligibility for an annual performance bonus based on individual and organizational results.
- Comprehensive benefits and wellbeing programs, including retirement savings, flexible vacation, and employee incentive programs.
- Flexible work arrangements, including the possibility of working from anywhere for up to eight weeks per year.
- Two company-wide mental health days annually and access to the Headspace app.
- Learning and career development opportunities, including skills development and tuition reimbursement.
- Resources supporting mental, physical, and financial wellbeing.
- Two paid volunteer days per year, along with opportunities to participate in community projects and environmental and social impact initiatives.
- Inclusive workplace culture focused on collaboration, professional growth, flexibility, and work-life balance.
We use an AI-powered matching process to ensure your application is reviewed quickly, objectively, and fairly against the role's core requirements. Our system identifies the top-fitting candidates, and this shortlist is then shared directly with the hiring company. The final decision and next steps (interviews, assessments) are managed by their internal team.
We appreciate your interest and wish you the best!
Why Apply Through Jobgether?
Data Privacy Notice: By submitting your application, you acknowledge that Jobgether will process your personal data to evaluate your candidacy and share relevant information with the hiring employer. This processing is based on legitimate interest and pre-contractual measures under applicable data protection laws (including GDPR). You may exercise your rights (access, rectification, erasure, objection) at any time.
We may use artificial intelligence (AI) tools to support parts of the hiring process, such as reviewing applications, analyzing resumes, or assessing responses and identifying potential inconsistencies or verification signals in application materials based on available information. These tools assist our recruitment team but do not replace human judgment. Final hiring decisions are ultimately made by humans. If you would like more information about how your data is processed, please contact us.
Skills
Sign in to read and join the discussion.