Data Engineering Lead
Posted Updated
Who recruits here Pro
Up to three recruiters at Blue Pearl PTY with their work email, so you can write to a person.
Part of Pro. How it works
Summary
Hands-on technical lead role designing and building enterprise-scale cloud data platforms: architecting scalable ETL/ELT pipelines and warehouse/lakehouse solutions on AWS using Snowflake, Databricks, Spark, Airflow and advanced SQL, while enforcing governance and mentoring engineers to serve BI, analytics and AI initiatives.
Johannesburg, South Africa | Posted on 07/14/2026
Our client is seeking an experienced Data Engineering Lead to design, build, and lead enterprise‑scale data platforms that enable business intelligence, advanced analytics, reporting, and data‑driven decision‑making.
The successful candidate will provide technical leadership across the full data engineering lifecycle, ensuring scalable, secure, reliable, and high‑performing data solutions. This role requires close collaboration with business stakeholders, solution architects, BI teams, governance specialists, and software engineering teams to deliver trusted and analytics‑ready data assets.
The ideal candidate is a hands‑on technical leader with extensive experience in cloud data platforms, ETL/ELT development, data architecture, SQL optimisation, and modern data engineering practices.
Key Responsibilities
Data Engineering Leadership
Lead the design, development, implementation, and continuous improvement of enterprise‑grade data engineering solutions.
Architect scalable, secure, reusable, and high‑performance data pipelines supporting batch and near real‑time processing.
Provide technical leadership across multiple data engineering initiatives and mentor junior and intermediate engineers.
Establish and enforce enterprise data engineering standards, frameworks, and best practices.
Drive innovation and continuous improvement across the organisation's data platform capabilities.
Design robust ingestion frameworks for structured, semi‑structured, and unstructured data.
Develop scalable storage architectures supporting data warehousing and lake‑house environments.
Implement efficient ETL and ELT processing frameworks.
Optimise cloud‑native data architectures for performance, scalability, reliability, and cost efficiency.
Design resilient solutions supporting analytics, reporting, AI, and machine learning initiatives.
Build, maintain, and optimise enterprise data pipelines.
Develop complex SQL queries, stored procedures, and transformation logic.
Design and maintain reusable data integration components.
Perform data transformation, cleansing, enrichment, and validation processes.
Ensure consistent data availability and integrity across enterprise platforms.
Data Governance & Quality
Embed data quality controls throughout the engineering lifecycle.
Implement automated validation, monitoring, lineage, metadata management, and observability.
Ensure compliance with organisational governance frameworks and regulatory requirements.
Secure sensitive and confidential information through appropriate engineering controls.
Maintain technical documentation including source‑to‑target mappings and technical specifications.
Stakeholder Management
Collaborate closely with business stakeholders, business intelligence teams, data analysts, solution architects, application development teams, governance teams, and infrastructure teams.
Translate business requirements into scalable technical solutions.
Communicate technical concepts clearly to both technical and non‑technical audiences.
Provide regular project updates, technical recommendations, and risk assessments.
Operational Excellence
Troubleshoot pipeline failures and performance issues.
Optimise processing times and infrastructure costs.
Lead root‑cause analysis and incident resolution.
Support production deployments and platform upgrades.
Requirements Education
Bachelor's Degree in Computer Science
Bachelor's Degree in Information Systems
Bachelor's Degree in Information Technology
Bachelor's Degree in Software Engineering
Bachelor's Degree in Data Science
Bachelor's Degree in Mathematics
Bachelor's Degree in Statistics
Or a related discipline
Relevant postgraduate qualifications will be advantageous.
Experience
8–10+ years' experience in Data Engineering.
Minimum 3 years leading technical data engineering teams.
Proven experience building scalable ETL/ELT solutions.
Experience working within enterprise environments.
Experience supporting reporting, analytics, and data science initiatives.
Experience implementing data governance frameworks.
Experience in Financial Services, Mining, Retail, Telecommunications, or Consulting environments will be advantageous.
Technical Knowledge
Modern Data Engineering architectures
Data Warehousing
Lake house Architecture
ETL / ELT Frameworks
Data Integration
Cloud Computing
SQL Performance Optimisation
Metadata Management
Data Governance
Master Data Management
Data Quality
CI/CD
Infrastructure as Code
Software Development Lifecycle (SDLC)
Information Security
Data Privacy Regulations
Technical Skills
AWS
Snowflake
Databricks
Advanced SQL
Python
Spark
Py Spark
AWS Glue
Airflow
Power BI
Qlik Sense
Qlik View
Tableau
SQL Server
Oracle
Amazon Redshift
Git
Terraform
API Integration
REST Services
Data Security
Data Encryption
Desired Attributes
Technical Leadership
Strategic Thinking
Analytical Thinking
Strong Problem‑Solving Ability
Stakeholder Management
Decision‑Making Ability
Accountability
Attention to Detail
Results Orientation
Planning and Organising
Mentoring and Coaching
Preferred Certifications
Microsoft Azure Solutions Architect Expert
AWS Certified Solutions Architect – Associate / Professional
Databricks
Snowflake
Certified Data Management Professional (CDMP)
TOGAF (Advantageous)
Key Success Measures
Delivery of scalable, secure data platforms.
Data pipeline reliability and availability.
Data quality and integrity.
Platform performance and optimisation.
Successful project delivery within agreed timelines.
Reduction in operational incidents.
Engineering automation and efficiency improvements.
Compliance with enterprise governance standards.
Mentoring and development of engineering team members.
Why Join This Opportunity? This is an exciting opportunity to lead the development of modern enterprise data platforms within a collaborative and innovative environment. You will play a key role in shaping the organisation's data engineering capability, enabling business intelligence, advanced analytics, AI initiatives, and digital transformation through scalable, secure, and high‑performing data solutions.
Our client is seeking an experienced Data Engineering Lead to design, build, and lead enterprise‑scale data platforms that enable business intelligence, advanced analytics, reporting, and data‑driven decision‑making.
The successful candidate will provide technical leadership across the full data engineering lifecycle, ensuring scalable, secure, reliable, and high‑performing data solutions. This role requires close collaboration with business stakeholders, solution architects, BI teams, governance specialists, and software engineering teams to deliver trusted and analytics‑ready data assets.
The ideal candidate is a hands‑on technical leader with extensive experience in cloud data platforms, ETL/ELT development, data architecture, SQL optimisation, and modern data engineering practices.
Key Responsibilities
Data Engineering Leadership
Lead the design, development, implementation, and continuous improvement of enterprise‑grade data engineering solutions.
Architect scalable, secure, reusable, and high‑performance data pipelines supporting batch and near real‑time processing.
Provide technical leadership across multiple data engineering initiatives and mentor junior and intermediate engineers.
Establish and enforce enterprise data engineering standards, frameworks, and best practices.
Drive innovation and continuous improvement across the organisation's data platform capabilities.
Design robust ingestion frameworks for structured, semi‑structured, and unstructured data.
Develop scalable storage architectures supporting data warehousing and lake‑house environments.
Implement efficient ETL and ELT processing frameworks.
Optimise cloud‑native data architectures for performance, scalability, reliability, and cost efficiency.
Design resilient solutions supporting analytics, reporting, AI, and machine learning initiatives.
Build, maintain, and optimise enterprise data pipelines.
Develop complex SQL queries, stored procedures, and transformation logic.
Design and maintain reusable data integration components.
Perform data transformation, cleansing, enrichment, and validation processes.
Ensure consistent data availability and integrity across enterprise platforms.
Data Governance & Quality
Embed data quality controls throughout the engineering lifecycle.
Implement automated validation, monitoring, lineage, metadata management, and observability.
Ensure compliance with organisational governance frameworks and regulatory requirements.
Secure sensitive and confidential information through appropriate engineering controls.
Maintain technical documentation including source‑to‑target mappings and technical specifications.
Stakeholder Management
Collaborate closely with business stakeholders, business intelligence teams, data analysts, solution architects, application development teams, governance teams, and infrastructure teams.
Translate business requirements into scalable technical solutions.
Communicate technical concepts clearly to both technical and non‑technical audiences.
Provide regular project updates, technical recommendations, and risk assessments.
Operational Excellence
Troubleshoot pipeline failures and performance issues.
Optimise processing times and infrastructure costs.
Lead root‑cause analysis and incident resolution.
Support production deployments and platform upgrades.
Requirements Education
Bachelor's Degree in Computer Science
Bachelor's Degree in Information Systems
Bachelor's Degree in Information Technology
Bachelor's Degree in Software Engineering
Bachelor's Degree in Data Science
Bachelor's Degree in Mathematics
Bachelor's Degree in Statistics
Or a related discipline
Relevant postgraduate qualifications will be advantageous.
Experience
8–10+ years' experience in Data Engineering.
Minimum 3 years leading technical data engineering teams.
Proven experience building scalable ETL/ELT solutions.
Experience working within enterprise environments.
Experience supporting reporting, analytics, and data science initiatives.
Experience implementing data governance frameworks.
Experience in Financial Services, Mining, Retail, Telecommunications, or Consulting environments will be advantageous.
Technical Knowledge
Modern Data Engineering architectures
Data Warehousing
Lake house Architecture
ETL / ELT Frameworks
Data Integration
Cloud Computing
SQL Performance Optimisation
Metadata Management
Data Governance
Master Data Management
Data Quality
CI/CD
Infrastructure as Code
Software Development Lifecycle (SDLC)
Information Security
Data Privacy Regulations
Technical Skills
AWS
Snowflake
Databricks
Advanced SQL
Python
Spark
Py Spark
AWS Glue
Airflow
Power BI
Qlik Sense
Qlik View
Tableau
SQL Server
Oracle
Amazon Redshift
Git
Terraform
API Integration
REST Services
Data Security
Data Encryption
Desired Attributes
Technical Leadership
Strategic Thinking
Analytical Thinking
Strong Problem‑Solving Ability
Stakeholder Management
Decision‑Making Ability
Accountability
Attention to Detail
Results Orientation
Planning and Organising
Mentoring and Coaching
Preferred Certifications
Microsoft Azure Solutions Architect Expert
AWS Certified Solutions Architect – Associate / Professional
Databricks
Snowflake
Certified Data Management Professional (CDMP)
TOGAF (Advantageous)
Key Success Measures
Delivery of scalable, secure data platforms.
Data pipeline reliability and availability.
Data quality and integrity.
Platform performance and optimisation.
Successful project delivery within agreed timelines.
Reduction in operational incidents.
Engineering automation and efficiency improvements.
Compliance with enterprise governance standards.
Mentoring and development of engineering team members.
Why Join This Opportunity? This is an exciting opportunity to lead the development of modern enterprise data platforms within a collaborative and innovative environment. You will play a key role in shaping the organisation's data engineering capability, enabling business intelligence, advanced analytics, AI initiatives, and digital transformation through scalable, secure, and high‑performing data solutions.
Skills
- AI
- Airflow
- Analytics
- API
- Automation
- AWS
- Aws Glue
- Azure
- Business Intelligence
- CI/CD
- Cloud
- Data Engineering
- Data Governance
- Data Pipelines
- Data Quality
- Data Science
- Data Warehousing
- Databricks
- ELT
- ETL
- Git
- Infrastructure as Code
- Machine Learning
- Master Data Management
- Metadata Management
- Observability
- Oracle
- Power BI
- Python
- Qlik
- Redshift
- Risk Assessment
- SDLC
- Snowflake
- Spark
- SQL
- SQL Server
- Stakeholder Management
- Statistics
- Tableau
- Terraform
- TOGAF
Sign in to read and join the discussion.