Senior Data Engineer (Big Data & Analytics)
Summary
Senior Data Engineer designing, building, and maintaining big-data pipelines, warehouses, and data lakes for a telecommunications company, turning massive datasets into analytics-ready insights. Core stack: Spark, Hadoop, Kafka, SQL, Python/Scala/Java, and cloud data platforms (AWS, Azure, GCP). Remote-first within South Africa.
About the Role Our client, a leading telecommunications innovator in Klerksdorp, North West , is seeking a Senior Data Engineer specializing in Big Data and Analytics. This is a remote-first position, offering the flexibility to work from anywhere within South Africa . You will be responsible for designing, building, and maintaining robust data pipelines and infrastructure that support our advanced analytics capabilities. This role is critical in transforming raw data into actionable insights, enabling data-driven decision-making across the organization. You will work with massive datasets, leveraging cutting-edge big data technologies and cloud platforms. The ideal candidate possesses a strong technical background in data engineering, a passion for solving complex data challenges, and the ability to collaborate effectively in a remote team environment. This is an exciting opportunity to shape the data landscape of a dynamic telecommunications company and contribute to its strategic growth through powerful data insights. We foster an environment of continuous learning and innovation for our remote workforce.
Key Responsibilities
- Design, develop, and optimize scalable data pipelines for ingesting, processing, and transforming large volumes of structured and unstructured data.
- Build and maintain data warehousing solutions and data lakes using big data technologies (e.g., Hadoop, Spark, Kafka).
- Implement data quality frameworks and ensure the accuracy, consistency, and reliability of data.
- Develop ETL/ELT processes and solutions to support business intelligence and analytics requirements.
- Collaborate with data scientists, analysts, and business stakeholders to understand data needs and deliver solutions.
- Monitor and troubleshoot data pipeline performance and data infrastructure issues.
- Optimize data storage and processing for cost-efficiency and performance.
- Stay abreast of emerging big data technologies and best practices, recommending and implementing new tools and techniques.
- Ensure data security and compliance with relevant regulations.
Requirements
- Bachelor's or Master's degree in Computer Science, Data Science, Engineering, or a related quantitative field.
- Minimum of 5 years of experience in data engineering, with a strong focus on big data technologies.
- Proficiency in programming languages such as Python, Scala, or Java.
- Hands-on experience with distributed data processing frameworks like Apache Spark, Hadoop ecosystem (HDFS, MapReduce, Hive).
- Experience with cloud data platforms (e.g., AWS EMR, S3, Redshift; Azure Data Lake, Databricks; GCP BigQuery).
- Strong SQL skills and experience with various database technologies (relational and NoSQL).
- Familiarity with data warehousing concepts and ETL/ELT tools.
- Understanding of data modeling techniques and best practices.
- Excellent problem-solving skills and ability to work independently.
- Strong communication skills for collaborating with technical and non-technical teams.
Benefits
- Competitive salary and comprehensive benefits package.
- Fully remote work arrangement with flexible hours.
- Allowance for home office setup.
- Opportunities for professional development, training, and conferences.
- Cutting-edge technology stack and challenging projects.
- Collaborative and innovative team culture.
- Generous paid time off.
- Retirement fund participation.