Senior Databricks Developer II Immediate II Remote

Experience: 8+ Years
Role: Senior Databricks Developer

Job Summary

We are looking for an experienced Senior Databricks Developer with 8+ years of overall experience and strong hands-on expertise in designing, developing, and optimizing enterprise-scale data engineering solutions using Databricks.

The ideal candidate should have strong proficiency in Databricks, PySpark, Python, SQL, Delta Lake, Unity Catalog, data modelling, and ETL/ELT pipelines. Experience in modernizing legacy data warehouses and migrating traditional ETL workloads to Databricks is essential.

Key Responsibilities

• Design, develop, and optimize scalable data pipelines using Databricks, PySpark, Python, and SQL.
• Implement Medallion Architecture (Bronze, Silver, and Gold layers).
• Develop reusable frameworks for data ingestion, transformation, validation, and processing.
• Design, implement, and optimize Delta Lake tables and large-scale Spark workloads.
• Build and manage data workflows using Databricks Workflows.
• Implement data governance and security using Unity Catalog, including RBAC, row/column-level security, and data masking.
• Develop dimensional, reference, master, and curated data models.
• Modernize legacy data warehouse solutions from SQL Server, Oracle, and Teradata to Databricks.
• Migrate stored procedures and traditional ETL processes into scalable cloud-based ETL/ELT pipelines.
• Perform Spark and SQL performance tuning, cost optimization, troubleshooting, and root-cause analysis.
• Implement pipeline monitoring, error handling, and recovery mechanisms.
• Support CI/CD, code reviews, testing, deployment, and production support.
• Mentor junior developers and participate in technical and design discussions.
• Collaborate with architects, business stakeholders, and onshore/offshore delivery teams.

Mandatory Technical Skills

• Databricks
• PySpark
• Python
• Advanced SQL
• Delta Lake
• Unity Catalog
• Medallion Architecture (Bronze/Silver/Gold)
• ETL/ELT Development
• Data Modelling
• Spark Performance Tuning
• Databricks Workflows

Good to Have

• Databricks AI capabilities.
• Experience with SQL Server, Oracle, and Teradata migration.
• Cloud data engineering experience.
• CI/CD and production support.

Desired Candidate Profile

• 8+ years of overall IT experience.
• Strong hands-on experience in Databricks development.
• Experience in enterprise-scale data engineering and warehouse modernization.
• Strong problem-solving, analytical, and communication skills.
• Ability to work independently and mentor junior team members.