Data Engineer
Summary
Builds and operates modern data platforms on Azure Databricks, owning solutions end-to-end from ingestion and transformation to deployment, monitoring and production support alongside a senior architect. Core stack: PySpark, SQL, Delta Lake, Unity Catalog, Structured Streaming and Azure DevOps, with heavy use of AI coding agents.
We’re looking for a Data Engineer to build and operate modern data platforms on Azure Databricks.
You’ll work with a Senior Architect on solution design and own the implementation end-to-end—from ingestion and transformation to deployment, monitoring and production support. Alongside strong data engineering skills, you’ll also take ownership of the platform/DevOps aspects of the solution.
We’re building for an AI-native world, so we expect you to use AI coding agents, AI-assisted development, reusable skills/workflows and agentic automation to make engineering faster, smarter and more reliable.
What You'll Do
- Build scalable batch and real-time data pipelines using Databricks, PySpark, SQL and Delta Lake.
- Implement Medallion Architecture (Bronze/Silver/Gold) and governed data products using Unity Catalog.
- Build streaming solutions using Structured Streaming and Azure Event Hubs.
- Work with ADLS Gen2, Azure Key Vault and Azure DevOps.
- Own CI/CD, deployment automation, monitoring and production troubleshooting.
- Optimize data pipelines and Databricks workloads for performance, reliability and cost.
- Build analytical datasets and dashboards using BI/visualization tools.
- Develop data products that can support analytics, ML, GenAI and agentic applications.
- Leverage AI agents and AI-native developer tools for coding, testing, documentation, troubleshooting and automation.
- Collaborate with architects, analysts and business stakeholders to deliver production-ready solutions.
What We're Looking For
- 4-6 years of hands-on experience in Data Engineering / Data Platforms.
- Strong Python/PySpark and SQL skills.
- Strong hands-on experience with Azure Databricks, Delta Lake and Medallion Architecture.
- Experience with batch + real-time/streaming data engineering.
- Hands-on experience with Unity Catalog, Databricks Workflows/Jobs and Structured Streaming.
- Experience with ADLS Gen2, Event Hubs, Azure DevOps/Git and Key Vault.
- Good understanding of CI/CD, cloud security and production operations.
- Hands-on experience with a BI/dashboarding tool.
- Strong problem-solving and ownership mindset.
- Comfortable using AI coding assistants/agents and eager to adopt emerging AI-native engineering practices.
Good to Have
- Databricks or Azure certifications.
- Lakeflow/DLT, Terraform/IaC or Kafka.
- Experience with AWS/GCP and Databricks.
- Exposure to GenAI, RAG or agentic applications.
- Experience in Telecom, Media, Advertising/AdTech.
If you enjoy building production-grade data platforms - and using AI to fundamentally improve how you build and operate them, we’d love to hear from you.