Senior Data Engineer
Posted 1
view
Summary
Senior Data Engineer who designs, builds, and owns scalable data platforms on Azure for retail, wholesale, and distribution clients — developing enterprise data warehouses, ETL/ELT pipelines (ADF, Databricks, Airflow), PySpark/Delta Lake transformations, and Power BI models.
Senior Data Engineer
Location - India (Coimbatore preferred)
Work Mode - WFO preferred, hybrid or remote will also be considered for the right candidate
Experience - 5+ years in Data Engineering, including hands-on cloud data platform delivery
Employment Type - Full-Time
ROLE OVERVIEW
Vserve is looking for a Senior Data Engineer to design, build, and own scalable data platforms on Microsoft Azure
ecosystems. This role will lead the development of Enterprise Data Warehousing, ETL/ELT pipelines, and analytics-
ready data models that power reporting and decision-making for our retail, wholesale, and distribution clients. The
ideal candidate is equally comfortable architecting cloud-native pipelines and partnering directly with business
stakeholders to translate requirements into reliable, production-grade data solutions.
KEY RESPONSIBILITIES
• Design, build, and maintain scalable Enterprise Data Warehouse (EDW) solutions on Azure (ADF, Databricks, Azure SQL, Microsoft Fabric).
• Develop and orchestrate end-to-end ETL/ELT pipelines using Apache Airflow, Azure Data Factory, and Databricks, ensuring reliability and minimal manual intervention.
• Build and optimize PySpark transformations and Delta Lake pipelines to process large-scale datasets into curated, analytics-ready data products.
• Design dimensional data models (star/snowflake schemas), fact and dimension tables, and data marts to support sales, inventory, demand, and profitability reporting.
• Develop Power BI semantic models, datasets, and interactive dashboards enabling self-service analytics and KPI reporting for business stakeholders.
• Implement data quality and validation frameworks, including reconciliation checks, exception reporting, and automated alerting.
• Own data reconciliation between ERP/business systems (e.g., Microsoft Dynamics AX / D365 F&O, POS platforms) and the enterprise data warehouse.
• Implement complex business logic for margin calculations, cost attribution, and promotional/discount allocation to
support accurate financial reporting.
• Build automated data ingestion frameworks using Python and REST APIs, and implement incremental loading
strategies for efficient, near real-time reporting.
• Tune and optimize SQL/PL-SQL queries and data workflows for performance across high-volume, regulated, or
compliance-sensitive datasets.
• Implement CI/CD for data pipelines (GitHub Actions, Git) to improve release consistency and reduce deployment
effort.
• Partner with business users, analysts, and SMEs to gather requirements and translate them into scalable, well-
documented data engineering solutions.
• Mentor junior data engineers and contribute to data architecture and engineering best practices across the team.
CANDIDATE PROFILE — SKILLS & EXPERIENCE
Must-Have
• 5+ years of hands-on Data Engineering experience, with demonstrated ownership of production data pipelines
end to end.
• Strong experience with Microsoft Azure data services: Azure Data Factory (ADF), Databricks, Azure Data
Lake, Azure SQL.
• Hands-on expertise in Apache Spark / PySpark and Delta Lake for large-scale data processing.
• Proven experience building and orchestrating pipelines with Apache Airflow or equivalent workflow
orchestration tools.
• Advanced SQL skills and solid Python (Pandas, NumPy) programming ability.
• Strong grounding in dimensional data modeling (star/snowflake schemas, normalization, ER-diagram design).
• Experience with Power BI (DAX, Power Query) and/or Microsoft Fabric for reporting and dashboard delivery.
• Experience implementing data quality, validation, and reconciliation frameworks in a production
environment.
Good-to-Have
• Exposure to Microsoft Dynamics AX / Dynamics 365 Finance & Operations (D365 F&O) or similar ERP/POS
platforms (e.g., Aptos POS).
• Experience with Informatica PowerCenter or other enterprise ETL tools.
• Familiarity with MongoDB, PostgreSQL, or DynamoDB alongside relational/warehouse platforms.
• Experience in retail, wholesale distribution, or financial services data domains.
• Working knowledge of CI/CD practices (GitHub Actions, Git) and Agile delivery methodologies.
• Prior experience mentoring engineers or leading a small data engineering pod.
Education
• Bachelor degree in Computer Science, Information Technology, or a related field is required.
• Master degree is a plus.
Requirements
Azure, ADF, Databricks, PySpark, Python, SQL, Airflow, Delta Lake, ETL/ELT, Data Warehousing, Data Modeling, Power BI, DAX, Data Quality, REST API, Git/CI-CD, D365 F&O
Skills
- Agile
- Airflow
- Analytics
- API
- Azure
- Azure Data Factory
- Candidate Experience
- CI/CD
- Cloud
- Cloud Native
- Data Engineering
- Data Ingestion
- Data Lake
- Data Modeling
- Data Pipelines
- Data Quality
- Data Warehousing
- Databricks
- Delta Lake
- DynamoDB
- ELT
- ERP
- ETL
- Financial Reporting
- Git
- GitHub
- GitHub Actions
- Microsoft Dynamics
- Microsoft Fabric
- MongoDB
- NumPy
- pandas
- PostgreSQL
- Power Query
- Power BI
- PySpark
- Python
- REST
- Snowflake
- Spark
- SQL
- Workflow Orchestration