Senior Data Engineer-(Immediate Joiners with LWD (Sept Month) Only Apply!
Spectral Consultants Senior Data Engineer-(Immediate Joiners with LWD (Sept Month) Only Apply!
Summary
Senior Data Engineer for a Professional Services & Accounting client, designing and running scalable ETL/ELT pipelines and governed Lakehouse data models. Core stack: Azure Databricks, PySpark, SQL, Delta Lake, Unity Catalog, ADLS Gen2; hybrid role in Bangalore for candidates with 5–9 years of experience.
Spectral Consultants is currently hiring for a Senior Data Engineer – Azure Databricks for a leading Professional Services & Accounting organization.
This is an opportunity to work on modern Lakehouse platforms and next-generation AI & Analytics initiatives, building scalable, governed, and AI-ready data solutions using Azure Databricks and the broader Azure ecosystem.
Location: Bangalore
Experience: 5–9 Years
Work Mode: Hybrid
About the Role:-
As a Senior Data Engineer, you will design and develop scalable ETL/ELT pipelines and curated Lakehouse data models using Azure Databricks, PySpark, SQL, Delta Lake, Unity Catalog, and ADLS Gen2. You will collaborate with Analytics, AI/ML, Product, and Business teams to deliver reliable, governed, and AI-ready datasets.
Key Responsibilities:-
- Design and maintain scalable ETL/ELT pipelines using Azure Databricks.
- Build data pipelines using Medallion Architecture (Bronze, Silver, Gold).
- Develop and optimize Delta Lake data models for Analytics and AI/ML use cases.
- Implement data ingestion using Spark Structured Streaming, Auto Loader, connectors, and Delta Sharing.
- Manage governed data using Unity Catalog and integrate with ADLS Gen2.
- Implement data quality, lineage, audit, reconciliation, and governance frameworks.
- Orchestrate workflows using Databricks Jobs/Workflows and DLT/Lakeflow.
- Implement CI/CD using Git and Databricks Asset Bundles.
- Troubleshoot production issues and collaborate with cross-functional teams to deliver reliable, consumption-ready datasets.
Must-Have Skills:-
- 5–8 years of hands-on experience in Data Engineering/Data Integration/Analytics Engineering.
- Strong expertise in Azure Databricks, PySpark/Spark, and SQL.
- Hands-on experience with Delta Lake and Medallion Architecture.
- Strong experience with Unity Catalog and ADLS Gen2.
- Experience building scalable ETL/ELT pipelines and Databricks Jobs/Workflows.
- Strong understanding of Data Governance, Data Quality, Lineage, Access Control, and PII handling.
- Experience with Git, CI/CD, and modern data engineering practices.
- Strong analytical, problem-solving, and stakeholder management skills.
Important Eligibility: Only Immediate Joiners or candidates whose Last Working Day (LWD) falls in September are eligible to apply.
