Data Engineer - Medallion Architecture

Job Description :

- Design and implement scalable data ingestion pipelines from a variety of systems including on-premise, Azure Databricks and c3.ai

- Build and maintain Bronze-to-Silver transformation pipelines following the Medallion architecture, using a config-driven approach

- Optimise BigQuery data organisation through effective partitioning, clustering, and schema design for large-scale, high-performance querying

- Implement and maintain workload identity federation and APIM integrations for secure cross-platform data movement

- Collaborate with Data Governance teams to embed data quality checks within various layer transformations, targeting defined quality thresholds

- Contribute to pipeline design reviews and actively participate in medallion architecture planning sessions with platform and architecture teams

- Document data flows, transformation rules, and design decisions to support knowledge transfer and team continuity