Lead Data Engineer
Summary
Lead a data engineering team in Nelspruit for a data analytics and business intelligence company, designing scalable data pipelines, warehouses and data lakes using Spark/Hadoop, Python or Java, SQL/NoSQL and cloud data services (AWS, Azure, GCP), while mentoring engineers and enforcing data quality and governance.
Our client, a pioneering force in data analytics and business intelligence, is seeking a talented Lead Data Engineer to join their expanding team in Nelspruit. This role is central to designing, building, and maintaining robust data pipelines and infrastructure that power critical business decisions. You will lead a team of data engineers, architecting scalable data solutions, and ensuring data quality and accessibility. The ideal candidate possesses a strong background in data warehousing, ETL/ELT processes, and big data technologies, coupled with excellent leadership and problem-solving skills. This position offers the opportunity to work with cutting‑edge technologies and influence the data strategy of a growing organization. You will be instrumental in optimizing data processes, enabling advanced analytics, and ensuring our client remains at the forefront of data‑driven innovation. The role demands a hands‑on approach to technical challenges and a commitment to delivering high‑quality data solutions that meet business needs. You will collaborate closely with data scientists, analysts, and business stakeholders to deliver data‑centric solutions. The position offers a challenging and rewarding career path for an experienced data professional.
Key Responsibilities- Design, develop, and maintain scalable and reliable data pipelines and ETL/ELT processes.
- Lead the architecture and implementation of data warehousing solutions and data lakes.
- Manage and optimize data infrastructure, ensuring data availability, quality, and performance.
- Collaborate with data scientists and analysts to understand data needs and provide data solutions.
- Develop and enforce data governance policies and standards.
- Mentor and guide junior data engineers, fostering technical excellence within the team.
- Evaluate and implement new data technologies and tools to enhance data capabilities.
- Troubleshoot and resolve issues related to data pipelines and infrastructure.
- Ensure data security and compliance with relevant regulations.
- Optimize data queries and database performance for analytical workloads.
- Bachelor's or Master's degree in Computer Science, Engineering, or a related quantitative field.
- Minimum of 7 years of experience in data engineering, with at least 3 years in a lead or senior role.
- Proven experience in designing and building complex data pipelines using technologies like Spark, Hadoop, or similar big data frameworks.
- Strong expertise in SQL and experience with various database technologies (relational and No SQL).
- Proficiency in programming languages such as Python or Java.
- Experience with cloud platforms (AWS, Azure, GCP) and their data services.
- Solid understanding of data warehousing concepts and dimensional modeling.
- Excellent problem-solving, analytical, and communication skills.
- Demonstrated ability to lead technical projects and mentor team members.
- Experience with BI tools and data visualization platforms is a plus.
- Competitive salary and performance-based incentives.
- Comprehensive benefits package including health and retirement plans.
- Opportunities for professional development and training in the latest data technologies.
- Lead key data initiatives for a growing organization in Mpumalanga.
- Work with a dedicated team focused on data innovation.
- Chance to shape the future of data infrastructure in the region.