Senior Data Engineer - Big Data Technologies
Summary
A senior data engineer in Randburg designs, builds, and maintains scalable data pipelines, ETL/ELT processes, and data models that power analytics and business intelligence. Core tech includes big data tools like Spark, Hadoop, and Kafka, plus SQL, Python or Scala, and cloud platforms such as AWS, Azure, or GCP.
About the Role
Our client is seeking a skilled Senior Data Engineer to join their analytics division in Randburg . This role is vital in designing, building, and maintaining scalable data pipelines and infrastructure that support advanced analytics and business intelligence initiatives. You will work with large datasets and cutting-edge big data technologies to transform raw data into actionable insights. This is an excellent opportunity for an experienced data professional to contribute to data-driven decision-making within a forward-thinking organization that values innovation and professional development.
Key Responsibilities
- Design, construct, install, test, and maintain highly scalable data management systems and pipelines.
- Develop and implement ETL/ELT processes to ingest, transform, and load data from various sources into data warehouses and data lakes.
- Optimize data retrieval and processing for performance and cost-efficiency using big data technologies like Spark and Hadoop.
- Collaborate with data scientists, analysts, and business stakeholders to understand data requirements and deliver solutions.
- Ensure data quality, integrity, and security across all data systems.
- Build and maintain data models that support reporting, analytics, and machine learning applications.
Requirements
- Master's or Bachelor's degree in Computer Science, Engineering, Statistics, or a related quantitative field.
- 5+ years of experience in data engineering or a related role.
- Proficiency in SQL and experience with relational and NoSQL databases.
- Hands-on experience with big data technologies such as Apache Spark , Hadoop, Kafka, or similar.
- Strong programming skills in languages like Python or Scala.
- Experience with cloud platforms (AWS, Azure, GCP) and data warehousing solutions.
Benefits
- Competitive salary and potential for annual bonuses.
- Comprehensive medical, dental, and vision benefits.
- Retirement savings plan with employer matching.
- Generous opportunities for training and professional development.
- A supportive hybrid work environment that promotes work-life balance.