Data Engineer
Posted Updated
Description
Datamatics Technologies in the United Arab Emirates is seeking a Data Engineer to design, build and operate data pipelines that move data from source systems into the lakehouse and onward to analytics and AI consumers.
You will work with Azure Data Factory and Fabric, Databricks or Synapse, implementing security, governance and CI/CD practices to deliver production-grade solutions. The role requires 4+ years in data engineering and excellent English communication.
Build batch, incremental, CDC and streaming ingestion pipelines from sources into Bronze layer. Develop Silver- and Gold-layer transformations in SQL, PySpark and Spark notebooks. Implement dimensional models, SCDs, surrogate keys and data marts according to the architect's designs. Integrate DQ checks, validation, quarantine and reconciliation into pipelines with the Senior Data Quality Engineer. Ensure that pipelines publish metadata and lineage to Microsoft Purview. Implement scheduling, dependency management, parameterization, error handling and alerting. Optimize pipelines, partitioning, file formats (Delta/Parquet) and compute usage for performance and cost efficiency. Apply access controls, masking, encryption and secure credential management in line with UAE PDPL, Dubai Data Law and DESC ISR. Maintain code in source control and deploy through CI/CD pipelines, with automated tests and environment promotion. Support production operations, incident resolution and runbooks, and coach counterparts for handover. 4+ years in data engineering, UAE government or regulated environments. Advanced SQL, Python and PySpark/Spark skills. Hands-on experience with Azure Data Factory, Fabric, Databricks or Synapse. Strong understanding of Lakehouse/Medallion architecture, Delta/Parquet formats and dimensional modelling, including SCDs and surrogate keys. Experience with CI/CD (Azure DevOps or GitHub), source control and DataOps practices. Familiarity with Microsoft Purview metadata and lineage, and with DQ tooling. Awareness of UAE data regulation, including UAE PDPL, the Dubai Data Law and DESC ISR. Excellent English communication and documentation skills.Skills
Sign in to read and join the discussion.