Platform Engineer – Data Science & AI Platform
Summary
Build and maintain the AI/ML platform infrastructure, including data pipelines, observability, and governance, using Python, Spark, and IaC while collaborating with data scientists and engineers.
- Add observability for agent traces responses evaluations and monitoring
- Apply security and compliance best practices with RBAC and ACLs and identity and access management
- Build and evolve data and ML platform infrastructure
- Collaborate with data engineers data scientists and ML engineers
- Create documentation standards and best practices
- Develop data processing components with Python and Spark
- Enhance observability for data quality pipeline reliability and model performance
- Implement data and ML governance with lineage and permissions
- Implement evaluation pipelines and safety guardrails
- Implement infrastructure-as-code
- Improve platform reliability, performance, and security
- Maintain shared datasets and platform services
- Manage Dev/Test/Prod environments
- Manage access control and secure configuration
- Manage data and ML lifecycle with pipelines experiment tracking and versioning
- Operate agentops capabilities including LLM gateway tool integration and prompt version management
- Set up CI/CD pipelines
Perks/Benefits:
- Employee discount
- Employee sample sales
- Fixed annual payment
- Paid annual leave
- Personalised learning
- Private medical care scheme