Data Engineer
Posted 8
views
Summary
Senior data engineer (6+ years) in Makati who designs and builds data storage, ingestion, and transformation pipelines on Azure Databricks, automates and optimizes data jobs, and keeps the EDP platform stable and compliant. Core stack: Azure, Databricks, SQL, Python, PySpark.
- Design and develop data storage solutions, including database objects, data structures, and data ingestion pipelines.
- Build and maintain data transformation, cleansing, and preparation processes.
- Optimize and automate data jobs through code enhancements, configuration improvements, and new technologies.
- Perform unit testing and system integration testing to ensure data quality and reliability.
- Monitor and maintain EDP platform stability and performance, including updates, patches, and operational enhancements.
- Ensure compliance with data governance, security policies, and SDLC standards.
- Collaborate with cross-functional teams to identify data requirements and improvement opportunities.
- Develop and maintain technical documentation for data engineering processes and procedures.
- Conduct Proof of Concepts (POCs) for new data integration and analytics solutions.
- Contribute to data engineering best practices, standards, and continuous improvement initiatives.
Qualifications:
- At least 6 years of experience in Data Engineering or Data-related projects.
- Strong hands-on experience with Databricks, Azure, SQL, Python, and PySpark.
- Strong background in Data Warehousing and Data Mart concepts.
- Experience designing and developing data ingestion, transformation, and integration pipelines.
- Good understanding of data governance tools and practices.
- Familiarity with Azure DevOps.
- Experience in job automation, performance optimization, and data platform monitoring is preferred.