Data Engineer (Azure Databricks & Spark)
Summary
Build and maintain large-scale data pipelines using Azure Databricks, PySpark, and SQL to process TB-scale datasets for a multinational insurance provider.
On-site - Taguig 5-10 Yrs Exp Bachelor Full-time
Job Description
Insurance Health & Wellness
Dental Insurance, Health Insurance, Life Insurance, Occupational Accident Insurance, Retiree Health & Medical
Others
Company Social Events
Bereavement Leave, Sick Leave, Wedding Leave
Work-Life Balance
Work from Home
Work Setup: Onsite - Taguig, Philippines
Industry: Multinational Company (MNC)
Employment Type: Permanent
Only for local candidates in the Philippines
Arbeit Group is currently looking for an experienced Data Engineer to join a leading multinational company in Taguig, Philippines.
We're looking for candidates with proven production-level experience in Azure Databricks and Spark. Strong expertise in PySpark, SQL, Data Lakes, ETL, and data quality is essential for this role.
Key Responsibilities
- Build, develop, and maintain large-scale data pipelines using Azure Databricks and Spark
- Design and optimize ETL processes and data ingestion workflows
- Develop high-performance Python/PySpark and SQL solutions for TB-scale datasets
- Design and maintain modern Data Lake architectures
- Ensure data quality, validation, integrity, and governance across data platforms
- Utilize Azure data services for analytics, storage, and processing
- Prepare technical documentation, requirements documents, and testing artifacts
- Collaborate with business users, stakeholders, and technical teams to deliver data solutions
- Support production environments and performance optimization initiatives
- Manage source control and code versioning using Git
Requirements
- Minimum 5+ years of hands-on experience with Azure Databricks and Apache Spark
- Strong expertise in Python, PySpark, and SQL
- Proven experience handling and optimizing TB-scale data processing workloads
- Strong knowledge of Data Lake architecture and ETL frameworks
- Experience with Azure Data Analytics services
- Strong understanding of data quality, validation, and governance practices
- Experience creating technical documentation and solution designs
- Excellent communication and stakeholder management skills
- Experience with Git version control
- Ability to work effectively with cross-functional teams
Preferred Qualifications
- Strong requirements gathering and testing documentation experience
- Exposure to complex analytics and business intelligence environments