Senior Data Engineer - Big Data Solutions
Summary
A Senior Data Engineer will design, build, and maintain scalable ETL pipelines and large-scale data warehouses/lakes using big data technologies like Spark and Hadoop, cloud services (AWS, Azure, GCP), and SQL/Python, supporting analytics and BI in a hybrid Port Elizabeth role.
About the Role
Our client is looking for a highly skilled Senior Data Engineer to join their expanding analytics team in Port Elizabeth . This role is crucial for building and optimizing robust data pipelines and infrastructure that support complex data analysis and business intelligence initiatives. You will work with large, diverse datasets, ensuring data quality, accessibility, and performance. The position offers a stimulating hybrid work environment , allowing for effective collaboration with team members while maintaining a degree of remote flexibility. If you are passionate about leveraging data to drive business insights, this is an excellent opportunity to make a significant impact.
Key Responsibilities
- Design, construct, install, and maintain scalable data pipelines and ETL processes.
- Develop and manage large-scale data warehouses and data lakes using big data technologies.
- Ensure data integrity, quality, and security across all data systems.
- Optimize data retrieval and processing for performance and efficiency.
- Collaborate with data scientists and analysts to understand data needs and provide solutions.
- Implement best practices for data governance, data modeling, and data management.
Requirements
- Bachelor's or Master's degree in Computer Science, Engineering, or a related quantitative field.
- 5+ years of experience in data engineering or a related role, with expertise in big data technologies.
- Proficiency in SQL and programming languages such as Python, Scala, or Java.
- Hands-on experience with distributed data processing frameworks (e.g., Spark, Hadoop).
- Experience with cloud data services (AWS Redshift, S3, Azure Data Factory, Google BigQuery).
- Strong analytical and problem-solving skills, comfortable working in a hybrid setup in Port Elizabeth .
Benefits
- Competitive salary and performance-based incentives.
- Comprehensive medical and retirement benefits.
- Professional development opportunities and support for certifications.
- Flexible hybrid work schedule balancing office and remote work.
- Access to cutting-edge data technologies and challenging projects.