Data Engineering Manager

We are looking for an experienced Data Engineering Manager to lead the design, development, and optimisation of enterprise-scale data platforms. The ideal candidate should have a strong background in data engineering, ETL development, data architecture, and cloud technologies (AWS or GCP) with experience building scalable, high-performance data platforms. This role requires a hands-on technical leader who can drive data platform initiatives, mentor engineering teams, and collaborate with cross-functional stakeholders to deliver reliable and scalable data solutions.

Responsibilities:

• Design, build, and maintain scalable enterprise data platforms.
• Architect and develop robust ETL/ELT pipelines for batch and real-time data processing.
• Design data architectures, including data lakes, lakehouses, and data warehouses.
• Develop and optimise large-scale data ingestion, transformation, and processing frameworks.
• Work with structured and unstructured data across multiple data sources.
• Collaborate with product managers, architects, and business stakeholders to understand data requirements and deliver scalable solutions.
• Drive cloud-based data engineering initiatives on AWS or GCP.
• Ensure high availability, performance, scalability, and reliability of data platforms.
• Implement data governance, data quality, security, and monitoring best practices.
• Lead technical discussions, perform architecture reviews, mentor engineers, and promote engineering best practices.

Leadership Expectations:

• Lead and mentor data engineering teams.
• Drive technical design and architecture decisions.
• Collaborate effectively with cross-functional and global teams.
• Own end-to-end delivery of data platform initiatives.
• Foster engineering excellence through code reviews, architecture reviews, and best practices.

Requirements:

• 10-14 years of experience in data engineering or data platform engineering.
• Strong experience in building enterprise-scale data platforms.
• Hands-on experience with ETL/ELT pipeline development.
• Strong understanding of data architecture, data modelling, and data warehousing concepts.
• Experience with Big Data technologies such as Spark, Kafka, Hadoop, Hive, Flink, Databricks, Trino, or similar.
• Strong programming skills in Python, Java, or Scala.
• Hands-on experience with cloud platforms - AWS or GCP (either one is mandatory).
• Experience with data lakes, lakehouse architectures, or distributed data processing systems.
• Strong knowledge of SQL and distributed databases.
• Experience working in Agile development environments.

Preferred Skills:

• Experience with Delta Lake, Iceberg, or similar modern data lake technologies.
• Exposure to CDC (Change Data Capture) frameworks such as Debezium.
• Experience with orchestration tools like Airflow.
• Familiarity with data governance, metadata management, and data quality frameworks.
• Experience in migrating on-premise data platforms to the cloud.
• Knowledge of CI/CD, Infrastructure as Code, and container technologies is a plus.

Ideal Candidate Profile:

• The ideal candidate has extensive experience building and managing large-scale data platforms with expertise in ETL, data architecture, and cloud technologies.
• They should possess strong technical leadership capabilities, a problem-solving mindset, and a track record of delivering scalable, high-performance data engineering solutions in enterprise environments.