Data Engineer
Summary
Hands-on data engineer at MAS Global Consulting building an end-to-end governed data platform on Microsoft Fabric for a US industrial manufacturer: metadata-driven ingestion, a medallion lakehouse, and a master data hub with a PySpark entity-matching engine that syncs golden employee records back to Workday, payroll, Epicor and security systems.
About the project. Our client, a US-based industrial manufacturer, runs on more than a dozen systems with no shared keys between them. MAS Global is building a governed data platform on Microsoft Fabric with two parts: a medallion lakehouse for analytics in Power BI, and a master data hub that keeps one golden record per key entity and syncs corrected values back to the operational systems. The first domain is Workforce: one trusted employee record across Workday HCM, payroll, Epicor ERP and the physical security system.
About the role. You will be the hands-on engineer who builds the platform end to end, from ingestion to the master hub and the integrations that send master updates back to each system. You will work with the Data Architect, a Power BI Developer, and the client's system owners and data stewards.
Responsibilities
- Build metadata-driven ingestion from Workday, payroll, Epicor and security into bronze.
- Standardize names, nicknames, suffixes, dates and codes in silver, with data quality checks.
- Implement the matching engine in PySpark: shared-ID and exact rules, probabilistic scoring such as Jaro-Winkler, and blocking. Tune it against known cases.
- Build the master hub in a Fabric SQL database: golden records, cross-reference IDs, survivorship from rules tables, and lineage.
- Build the stewardship workflow: a review queue, steward edits and a full audit trail.
- Build the sync back of create, update and deactivate events through Fabric pipelines or MuleSoft, with retries and delivery confirmation.
- Publish gold tables and data quality findings, ready for Power BI Direct Lake.
- Schedule and monitor the daily runs, and handle personal and compensation data securely.
- Work with Git and deployment pipelines, and write tests that protect the matching rules.
Qualifications
Required
- 5+ years in data engineering, including 2+ on the Microsoft data stack.
- Hands-on Microsoft Fabric experience, or deep Synapse or Databricks on Azure plus active Fabric adoption.
- Built a medallion architecture in production.
- Strong PySpark, Spark SQL and T-SQL.
- Entity resolution or MDM: fuzzy matching, survivorship, golden records.
- API integration (REST, OAuth, files, SFTP) and write-back to operational systems.
- Git and CI/CD for data workloads.
Nice to have
- Workday, payroll or Epicor data extraction.