Lead AI Engineer, Databricks
Summary
Designs and builds scalable data engineering frameworks and AI-ready data products using Databricks (PySpark, Delta Lake) while integrating Agentic AI solutions like LangGraph to accelerate development for a global team of data engineers.
We are seeking a Lead AI Engineer, Databricks, to design and build data engineering frameworks, reusable components, and best practices that empower hundreds of SAP Data Engineers to build AI-ready Data Products at scale, while driving innovation in AI-assisted development and Agentic AI solutions.
Responsibilities
- Design and build the data engineering framework, reusable components and templates (e.g. CI/CD), and best practices to boost productivity of Data Engineers building AI-ready Data Products
- Extend the data engineering framework to make it easy for AI Agents to discover and consume Data Products at scale
- Provide a powerful AI-assistant to accelerate Data Product development and operations
- Establish and promote engineering excellence practices across agile team peers
- Set up robust, scalable pipelines leveraging CI/CD frameworks
- Collaborate with international teams to align on technical strategy and delivery
- Guide Data Product Engineers with actionable technical direction
Requirements
- 5+ years of experience in Data Engineering with a successful track record of building enterprise-quality Data Products, preferably in Lakehouse architecture
- Proficiency in Python development
- Proficiency in version control systems and CI/CD frameworks, such as GitHub and GitHub Actions
- Expertise in Databricks technologies, including PySpark, Delta Lake and Unity Catalog, or equivalent technology
- Expertise in Agentic AI and frameworks such as LangGraph, MCP and A2A
- Knowledge of industry standard microservices technologies, including Kubernetes and Docker
- Strong English communication skills, both written and spoken, for collaboration with international teams
- Capability to provide actionable technical guidance and implement engineering excellence practices among agile team peers
Benefits
We connect like-minded people:
- Delivering innovative solutions to industry leaders, making a global impact
- Enjoyable working environment, whether it is the vibrant office or the comfort of your own home
- Opportunity to work abroad for up to two months per year
- Relocation opportunities within our offices in 55+ countries
- Corporate and social events
We invest in your growth:
- Leadership development, career advising, soft skills and well-being programs
- Certifications, including GCP, Azure and AWS
- Unlimited access to EPAM's internal learning database
- Free English classes with certified teachers
- Discounts in local language schools, including online courses for the Kazakh language
We cover it all:
- Participation in the Employee Stock Purchase Plan
- Monetary bonuses for engaging in the referral program
- Medical & family care package
- Six trust days per year (sick leave without a medical certificate)
- Coverage of psychology sessions of your choice
- Benefits package (sports activities, a variety of stores and services)
- Housing support program: preferential mortgage access via Otbasy Bank partnership