Senior MLOps Engineer

We are seeking a Senior MLOps Engineer to own end-to-end Machine Learning infrastructure, with a strong focus on high-scale LLM inference serving, distributed fine-tuning pipelines, and multi-provider gateway routing. In this role, you will bridge the gap between ML models and production reliability—deploying self-hosted open-weight models (ranging from ~7B to ~376B parameters), optimizing multi-GPU/distributed topologies, managing cloud cost governance, and treating the internal engineering platform as a product.

Details

• Role: Senior MLOps Engineer / ML Infrastructure Engineer

• Seniority: Senior / Lead (5+ years in production MLOps/ML Infrastructure)

• Allocation: Full-Time (1 FTE)

• Rate Cap: Max 145 PLN/h net B2B

• Work Model: Remote / Cloud-native across major hyperscalers (AWS, GCP, Azure)

Responsibilities

• High-Scale Inference Serving

  • Deploy, scale, and optimize self-hosted open-weight models (~7B to ~376B parameters) using engines such as vLLM, Triton Inference Server, or TGI.

  • Apply continuous batching, tensor/pipeline parallelism, and quantization strategies (AWQ, GPTQ, FP8) tailored to model size and strict SLA constraints.

• Multi-Provider Gateway & Cost Governance

  • Operate and enhance the intelligent routing layer spanning self-hosted models and external APIs (OpenAI, Anthropic, OpenRouter).

  • Build token accounting, rate limiting, budget controls, and cost/latency/quality-aware routing logic.

• Distributed Training & Fine-Tuning Infrastructure

  • Build and maintain automated pipelines for fine-tuning, evaluation, versioning, and continuous delivery (MLflow, SageMaker Pipelines, or Kubeflow).

  • Manage distributed training workloads using DeepSpeed, FSDP, or Accelerate.

• Reliability, Observability & Production Ownership

  • Own production reliability, monitoring, logging, and incident response for ML services (handling GPU OOMs, degraded inference, and latency spikes).

  • Participate in on-call rotation for the inference serving platform.

• Evaluation Harnesses & Platform Engineering

  • Stand up automated evaluation and verification harnesses to catch quality/performance regressions before deployment.

  • Deliver infrastructure-as-code (IaC) and CI/CD pipelines to provide self-serve tooling for internal engineering teams.

Requirements

• Production Experience: 5+ years of hands-on experience in MLOps, ML Infrastructure, or ML Engineering owning end-to-end model lifecycles in production.

• On-Call & Incident Handling: Proven experience being on-call for live ML services and resolving real-world incidents (e.g., GPU OOMs, memory leaks, routing failures, cost blowups).

• Inference & Quantization Depth: Deep understanding of quantization and parallelism trade-offs under latency, throughput, and hardware cost constraints.

• Core Tech Stack:

  • Languages: Advanced Python (C/C++ for performance-sensitive paths is a plus).

  • Containerization & Orchestration: Docker, Kubernetes (EKS/GKE), Helm, and IaC tools (Terraform/CloudFormation).

  • Cloud Ecosystems: Deep experience with hyperscaler ML services (AWS SageMaker, EC2 GPU instances, Lambda).

  • MLOps & Training Frameworks: MLflow, Kubeflow, PyTorch, DeepSpeed, FSDP, or Hugging Face Accelerate.

Nice to Have / Bonus

• Prior experience building multi-provider LLM gateways with token billing and cost controls.

• Track record of building programmatic LLM evaluation/verification frameworks.