Data Engineer
Posted Updated
Description
Datamatics Technologies LLC in the UAE is seeking a Data Engineer to design and operate data pipelines that move data from source systems into the lakehouse and on to analytics and AI consumers. The role focuses on automated, performant, secure pipelines and governance using Azure / Fabric with metadata captured in Microsoft Purview.
You will collaborate across the data architecture, data quality, business analysis and AI engineering teams to deliver production-grade pipelines, maintaining CI/CD
Build batch, incremental, CDC and streaming pipelines into Bronze using Azure Data Factory and Fabric. Develop Silver- and Gold-layer transformations in SQL, PySpark; perform cleansing and conformance. Implement dimensional models, SCDs and surrogate keys. Integrate DQ checks and data quality governance with the team. Publish metadata and lineage to Purview; ensure security and access controls. Advanced SQL, Python and PySpark expertise. Hands-on with Azure Data Factory, Fabric, Databricks or Synapse. Experience with CI/CD and DataOps practices. Knowledge of Microsoft Purview metadata & lineage.Skills
Sign in to read and join the discussion.