Data Engineering Lead

Role & responsibilities

Lead the design and implementation of scalable data architectures on Azure, leveraging Databricks, Delta

Lake, and related Azure data services.

• Build and optimize highvolume ETL/ELT pipelines using PySpark, SQL, Databricks Workflows, and Delta Live

Tables.

• Define and enforce best practices for data engineering across notebooks, jobs, CI/CD, Unity Catalog,

security, and workspace governance.

• Integrate and orchestrate data pipelines using Azure Data Factory, Azure Synapse pipelines, or Azure

Databricks Workflows.

• Drive performance tuning for Spark jobs, cluster configurations, and data storage layers to balance speed

and cost efficiency.

• Implement robust data quality and validation frameworks using tools like Great Expectations, DLT

expectations, or custom PySpark checks.

Candidate should be Gradaute from IIT/IIM

Interested candidate can share resume at Rajwinder@bottechus.com