Senior Azure Data Engineer
Posted Updated
Description
Role: Sr. Azure Data Engineer
Experience: 8–10 Years
Work Timings: 1:30 PM – 10:30 PM IST
Location: Bellandur Bengaluru (Work from Office)
Company: Chevron
Employment Type: 6- 12 months Contract
Role Overview
We are seeking an experienced Senior Data Engineer to design and deliver scalable cloud data solutions on Azure. The ideal candidate will have strong expertise in Databricks, PySpark, and modern data architectures, with exposure to energy domain standards like OSDU.
Key Responsibilities
- Architect and design robust Azure-based data solutions using Databricks, ADLS, and PaaS services
- Define and implement scalable data Lakehouse architectures aligned with OSDU standards
- Build and manage end-to-end data pipelines for batch and real-time processing using PySpark
- Establish data governance frameworks including metadata, lineage, security, and access control
- Implement DevOps best practices (CI/CD, Azure Pipelines, GitHub, automated deployments)
- Collaborate with stakeholders to translate business needs into technical solutions
- Develop and maintain architecture documentation, solution patterns, and standards
- Provide technical leadership and mentorship to engineering teams
- Optimize solutions for performance, cost, reliability, and security
- Ensure alignment with enterprise architecture and compliance standards
- Drive adoption of modular and reusable cloud data components
Required Skills & Qualifications
Core Technical Skills
- Azure Databricks, Apache Spark (PySpark), Delta Lake, Unity Catalog
- Azure Data Lake Storage (ADLS), Azure Data Factory, Synapse Analytics
- Strong experience in Python-based data engineering
- Data pipeline development (batch + real-time)
Architecture & Advanced Skills
- Data Lakehouse architecture and distributed systems
- Microservices, APIs, and integration frameworks
- OSDU (Open Subsurface Data Universe) or similar energy data models
DevOps & Tools
- CI/CD tools: Azure Pipelines, GitHub Actions
- Infrastructure as Code: Terraform or similar
Other Skills
- Data governance, security, compliance, and cost optimization
- Strong analytical and problem-solving skills
- Excellent communication and stakeholder management
Skills
- Analytics
- API
- Azure
- Azure Data Factory
- Azure Synapse
- CI/CD
- Cloud
- Data Engineering
- Data Governance
- Data Lake
- Data Pipelines
- Databricks
- Delta Lake
- DevOps
- Distributed Systems
- GitHub
- GitHub Actions
- Infrastructure as Code
- Lakehouse
- Microservices
- PySpark
- Python
- Spark
- Stakeholder Management
- Terraform
- Unity
- Unity Catalog
Sign in to read and join the discussion.