Senior data engineer - big data technologies
Summary
Senior data engineer in the client's analytics division in Randburg, designing and maintaining scalable data pipelines, ETL/ELT processes, and data models using Spark, Hadoop, Kafka, SQL, Python/Scala, and cloud data warehouses to support analytics and BI. Hybrid work with benefits and bonuses.
Our client is seeking a skilled Senior Data Engineer to join their analytics division in Randburg. This role is vital in designing, building, and maintaining scalable data pipelines and infrastructure that support advanced analytics and business intelligence initiatives. You will work with large datasets and cutting-edge big data technologies to transform raw data into actionable insights. This is an excellent opportunity for an experienced data professional to contribute to data-driven decision-making within a forward-thinking organization that values innovation and professional development.
Key Responsibilities Design, construct, install, test, and maintain highly scalable data management systems and pipelines. Develop and implement ETL/ELT processes to ingest, transform, and load data from various sources into data warehouses and data lakes. Optimize data retrieval and processing for performance and cost-efficiency using big data technologies like Spark and Hadoop. Collaborate with data scientists, analysts, and business stakeholders to understand data requirements and deliver solutions. Ensure data quality, integrity, and security across all data systems. Build and maintain data models that support reporting, analytics, and machine learning applications. Requirements Master's or Bachelor's degree in Computer Science, Engineering, Statistics, or a related quantitative field. 5+ years of experience in data engineering or a related role. Proficiency in SQL and experience with relational and No SQL databases. Hands-on experience with big data technologies such as Apache Spark , Hadoop, Kafka, or similar. Strong programming skills in languages like Python or Scala. Experience with cloud platforms (AWS, Azure, GCP) and data warehousing solutions. Benefits Competitive salary and potential for annual bonuses. Comprehensive medical, dental, and vision benefits. Retirement savings plan with employer matching. Generous opportunities for training and professional development. A supportive hybrid work environment that promotes work-life balance.