Senior Data Engineer

What the role is about

We are seeking for a Senior Data Engineer to join our Data Management team. In this role, you will lead engineering capability for its data platform and data integration work. You will be responsible for setting Data Engineering standards within the client environment. You will design and maintain reliable data architecture, pipelines and integrations across multiple source systems, establish data engineering standards, and provide technical diagnosis for data pipeline incidents. You will work closely with business and use-case owners, IT/cybersecurity stakeholders, source-system owners and vendors to turn business requirements into maintainable data solutions. You may also provide technical guidance to a junior/mid-level Data Engineer and support business continuity through production runbooks, recovery procedures and knowledge transfer.

What you will be working on

Data Architecture and Pipeline Engineering

• Design and maintain data architecture, models, pipelines and integration patterns across multiple source systems to a centralized Data Platform and governed analytical datasets.
• Build, review and improve automated ingestion and transformation workflows with robust failure handling, reconciliation, testing and observability.
• Act as the primary data engineering subject matter expert for Data Engineering tasks
• Handle data pipeline incidents and perform root-cause analysis and coordinate remediation.
• Establish engineering standards for code structure, version control, testing, documentation, deployment, monitoring and maintainability.
• Operationalize Data Governance policies, such as automated data-quality controls and monitoring for incomplete, inconsistent, duplicated or unexpected data.
• Define and review APIs, file/data-transfer specifications, schemas and technical acceptance criteria with source-system owners and vendors.
• Review vendor-developed pipeline designs, code, documentation and change proposals for reliability, security, supportability and technical debt.
• Translate validated analytical and business requirements into reusable datasets and technical designs in partnership with Business Analysts.
• Work hands-on in the AWS environment, including AWS Glue, and coordinate access-control, security, logging and governance requirements with IT/cybersecurity stakeholders.
• Own production runbooks, recovery procedures, technical documentation and knowledge transfer needed for business continuity.
• Guide any junior/mid-level Data Engineer and review technical work without formal people-management responsibility.

3LoDGovernance Responsibilities

• Operationalize 3LoDgovernance controls. Design and implement automated data governance controls — including data quality monitoring, classification enforcement, access logging, and reconciliation checks. Ensure these controls are embedded from inception in the ADAPT platform and all governed datasets, inline with the governance-by-design principle.
• Compliance monitoring support. Provide the CDO-led data governance function with the technical monitoring mechanisms needed to track adherence to data policies and standards across client's data platform, including dashboards or alerts for data quality thresholds, pipeline failures, and access anomalies, in support of Line 2's compliance monitoring obligations.
• Line 3 audit enablement. Own and maintain production runbooks, data lineage documentation, pipeline architecture records, and change logs to a standard that enables Line3 to conduct periodic audits of data governance processes, controls, and compliance. Serve as the primary technical point of contact during data governance audits and support root-cause analysis for audit findings.
• Risk escalation and issue resolution. Proactively identify, document, and escalate data engineering risks — including architecture risks, data quality failures, and security or access control gaps — to the CDO-led data governance function and IT/cybersecurity stakeholders, in line with Line 2's issue resolution and strategic recommendations obligations under the 3LoD framework.
• Governance by design. Integrate governance and control measures into pipeline and platform design from the outset, ensuring that business-as-usual engineering workflows embed data classification, quality, and access controls rather than treating them as post-hoc additions.
• Capability contribution. Support the data governance function's capability-building efforts by providing technical guidance to the Data Engineer and contributing to the development of engineering standards that reflect evolving governance requirements and regulatory changes, in support of Line 2's continuous improvement mandate.

Operational expectations

• The role provides first-line technical Production support during normal operations. Regular after-hours or on-call work is not expected; planned after-hours support may be required for scheduled deployments.

What we are looking for

Must-have

• Possess knowledge or experience in Computer Science, Information Systems, Data Engineering, Software Engineering, and/or subjects with substantial technical or quantitative content.
• Demonstrated end-to-end ownership of production data pipelines and integrations, with approximately 5–8years of relevant professional experience in data engineering or a closely related field.
• Demonstrated end-to-end ownership of production or business-critical data pipelines and integrations.
• Strong SQL, data modelling and Python/data-engineering programming fundamentals.
• Strong technical fundamentals in SQL, data modelling and Python, with hands-on experience building and maintaining ETL/ELT pipelines, automated tests, data-quality checks and failure-handling approaches in a production environment.
• Hands-on experience with AWS services in a production context, including AWS Glue, with the ability to operate safely and coordinate access-control, security and governance requirements across services.
• Ability to design maintainable ETL/ELT pipelines, automated tests, data-quality checks, logging/monitoring and failure-handling approaches.
• Experience integrating multiple source systems through APIs, files, databases or equivalent interfaces, and the ability to define and review technical specifications and acceptance criteria with system owners and vendors.
• Ability to establish and uphold engineering standards — including code structure, version control, testing, documentation and monitoring — in an environment where data engineering practices are still maturing.
• Good communication skills with the ability to convey technical risk, incidents and trade-offs clearly to both technical and non-technical audiences, including business stakeholders and vendors.
• Demonstrated ability to operationalize data governance policies through technical controls — including automated data quality monitoring, access logging, classification enforcement, and audit-ready documentation — in support of a structured governance framework such as the Three Lines of Defense (3LoD).

Good-to-have

• Exposure to data platform or architecture design, beyond pipeline-level work — relevant since the role involves setting standards as the first Data Engineer hire.
• Experience reviewing vendor-delivered data platforms or integrations.
• Experience working in or with public sector or heritage/cultural institutions
• Exposure to data platform or architecture design, beyond pipeline-level work — relevant since the role involves setting standards as the first Data Engineer hire.
• Able to deploy via CI/CD, and have experience with data observability and data governance experience.
• Experience with visualization tools like Tableau, Amazon QuickSight, Power BI, to understand the end-to-end data pipeline and visualization
• Hands-on experience with large-scale distributed data processing using Apache Spark or its Python variant, PySpark
• Experience working with regulated/public-sector environments.
• Experience mentoring or technically guiding another engineer.
• Experience working within or alongside a formal data governance function, including supporting internal or external audit activities related to data pipelines, data quality, or access management.