Databricks Data Engineer
Description
++Senior MDM / Databricks Data Engineer++
Experience: 6--10+ Years
Key Skills: MDM, Databricks, PySpark, SQL, Delta Lake, Unity Catalog, Data Quality, Data Governance
Role Summary
We are looking for an experienced MDM / Databricks Data Engineer to build scalable master data and integration solutions on a modern Databricks Lakehouse. The role will focus on common IDs, Golden Records, entity resolution, data quality, governance, and cross-platform data integration.
Key Responsibilities
· Design and implement MDM solutions, including common IDs, Golden Records, entity matching, deduplication, and survivorship rules.
- Integrate data from multiple source systems into the Databricks Lakehouse.
- Develop and optimize data pipelines using Databricks, PySpark, SQL, and Delta Lake.
- Build and enhance Bronze, Silver, and Gold data layers.
- Implement data quality, standardization, reconciliation, and validation processes.
- Support Unity Catalog, data lineage, access controls, cataloguing, and data governance.
- Implement data masking and anonymization for sensitive data.
- Build trusted, reusable data layers for cross-platform analytics and intelligence.
- Collaborate with business, architecture, and engineering teams to define MDM and data integration requirements.
Required Skills
· Strong experience in Master Data Management (MDM) and data engineering.
- Hands-on expertise in Databricks, PySpark, SQL, Delta Lake, and Lakehouse architecture.
- Experience with Golden Records, common IDs, entity resolution, deduplication, and data quality.
- Good understanding of Unity Catalog, data governance, metadata, and lineage.
- Experience integrating data from multiple enterprise platforms.
- Knowledge of Medallion Architecture and data privacy/anonymization.
Good to Have
· Experience with Informatica MDM/IDMC, Reltio, Profisee, Semarchy, or similar MDM platforms.
- Cloud experience with Azure, AWS, or GCP.
- Exposure to large-scale data modernization or migration programs.