ML Ops Enigneer / 1
We are seeking an advanced ML Ops Engineer to design and implement the infrastructure required to host, orchestrate, and manage up to 1,500 ML scoring processes within a new Databricks environment. The focus of the role is on operationalizing the ML scoring pipelines by setting up a scalable, secure, and well‑monitored platform for data science teams to deploy their models.
Environment Configuration
- Set up Databricks clusters, jobs, and workflows for large-scale ML scoring use cases.
- Infrastructure as Code is used for reproducibility and governance (e.g., Terraform).
- Implement scalable infrastructure capable of running thousands of ML scoring tasks.
- Configure job scheduling, parallel execution strategies, and resource optimization.
- Monitoring and alerting are integrated into the platform using cloud-native tools.
- Security, compliance, and cost-efficiency are key pillars of the operational setup.
ML Ops Pipeline Integration
- Develop deployment processes for ML models using Databricks MLflow or equivalent.
- Implement version control and tracking for models, scoring code, and configuration files.
Execution Management
- Build frameworks to orchestrate scoring of >1,500 ML models or scoring jobs.
- Ensure resilience, fault tolerance, and restart capabilities for failed jobs.
- Monitoring & Observability Integrate logging, alerting, and dashboards to monitor scoring throughput, latency, and failures.
- Establish model performance monitoring hooks for post‑scoring analytics.
Automation
- Work alongside Dev Ops Engineers to ensure common infrastructure and processes (e.g., shared storage, Delta Lake tables) serve both ML and BI use cases.
- Automate provisioning of resources and deployments from CI/CD pipelines.
- Utilize Infrastructure as Code (IaC) where feasible for reproducibility.
Collaboration
- Work closely with data scientists, solution architects, and platform engineers to ensure smooth handover from model development to operational scoring.
- Define operational SLAs for scoring workloads.
Work 3 times a week from an office in Warsaw, Lublin or Poznań.
We hereby inform you that Inetum Polska sp. z o.o. has implemented an internal reporting (whistleblowing) procedure. The content of the procedure and the possibility to submit an internal report are available at:
https://inetum.whispli.com/speakup?locale=pl