Data Engineer – Databricks
Summary
Build and optimize scalable data pipelines using Databricks, PySpark, and SQL, focusing on ETL/ELT, Delta Live Tables, and Unity Catalog.
Job Description
We are looking for an experienced Data Engineer with strong expertise in Databricks to design, develop, and maintain scalable data solutions. The role will focus on building and optimizing data pipelines, implementing ETL processes, ensuring data quality, and supporting data integration across various systems.
The successful candidate will work closely with multiple teams, provide technical solutions, and serve as a subject matter expert in data engineering and Databricks technologies.
Responsibilities
Design, develop, and optimize scalable data pipelines using Databricks.
Develop and maintain ETL/ELT processes for data integration and transformation.
Implement data engineering solutions using Delta Live Tables, Auto Loader, and Unity Catalog.
Monitor data pipelines and troubleshoot data quality, performance, and processing issues.
Apply data modeling and database design principles to support business and technical requirements.
Work with Python, PySpark, and SQL to process and transform large datasets.
Integrate data using Azure Data Factory and other data integration technologies.
Collaborate with multiple teams and stakeholders to understand requirements and provide effective data solutions.
Serve as a subject matter expert and contribute to technical and team-level decisions.
Document data processes, workflows, and technical solutions following best practices.
Identify opportunities to improve data pipeline performance, reliability, and accessibility.
Provide technical guidance and solutions to issues affecting the immediate team and other teams.
Requirements
5+ years of hands‑on experience with Databricks.
Expert-level proficiency in the Databricks Unified Data Analytics Platform.
Strong experience with:
Databricks Unity Catalog
Delta Live Tables (DLT)
Auto Loader
Python
PySpark
SQL
Azure Data Factory
Strong understanding of data modeling and database design.
Solid experience with ETL/ELT and data integration techniques.
Knowledge of data warehousing concepts and practices.
Familiarity with Azure cloud data services is an advantage.
Strong analytical, problem-solving, and troubleshooting skills.
Ability to collaborate with multiple teams and provide technical guidance.
Work Details
Experience: 5+ years
Work Location: Taguig
Project Location: Uptown Tower 3
Work Setup: Hybrid
Onsite Requirement: 2 days per week
Shift: Mid Shift, 2:00 PM – 12:00 AM
Work Hours: 45 hours per week