Data Engineer
Description
Our client, a rapidly expanding e-commerce platform, is seeking a talented and motivated Data Engineer to join their team in **Benoni, Gauteng**. This role is essential for building and maintaining the data pipelines and infrastructure that support our data-driven decision-making processes. You will be responsible for designing, developing, and optimizing ETL/ELT processes, ensuring the availability and integrity of data for analysis, reporting, and machine learning applications. This is a remote-first position, enabling you to contribute effectively from your chosen location within **South Africa**. We are looking for an individual with a strong understanding of data warehousing concepts, big data technologies, and a passion for building scalable data solutions. You will work closely with data scientists, analysts, and business stakeholders to ensure data meets the needs of the organization. This is a fantastic opportunity to make a significant impact in a fast-paced, innovative environment.
Key Responsibilities- Design, build, and maintain robust and scalable data pipelines (ETL/ELT processes).
- Develop and manage data warehouses and data lakes using cloud-based technologies.
- Ensure data quality, integrity, and reliability across all data systems.
- Optimize data storage, retrieval, and processing for performance and cost-efficiency.
- Collaborate with data scientists and analysts to understand their data requirements.
- Implement data governance and security best practices.
- Automate data processes and monitoring systems.
- Work with various data sources, including databases, APIs, and streaming data.
- Troubleshoot and resolve issues related to data pipelines and infrastructure.
- Stay updated with emerging data engineering technologies and methodologies.
- Bachelor's degree in Computer Science, Engineering, or a related quantitative field.
- 3+ years of experience in data engineering or a related role.
- Proficiency in SQL and experience with relational and No SQL databases.
- Experience with cloud platforms such as AWS (S3, Redshift, Glue, EMR), Azure (Data Lake, Synapse), or GCP (Big Query, Dataflow).
- Hands-on experience with big data technologies (e.g., Spark, Hadoop, Kafka).
- Proficiency in at least one programming language (e.g., Python, Scala, Java).
- Experience with ETL/ELT tools and data warehousing concepts.
- Understanding of data modeling techniques.
- Strong problem-solving and analytical skills.
- Excellent communication and collaboration skills, especially in a remote setting.
- Familiarity with workflow orchestration tools (e.g., Airflow).
- Our client offers a competitive salary, performance bonuses, and comprehensive medical aid and retirement benefits.
- As a remote position, you enjoy the flexibility to work from anywhere.
- We provide a professional development allowance for training, conferences, and certifications.
- Our client fosters a supportive and inclusive remote team culture, with regular virtual team-building events and knowledge-sharing sessions.
- Generous annual leave and wellness programs are part of our commitment to employee well-being.
This role in **Benoni, Gauteng** offers an exciting opportunity to build and manage critical data infrastructure for a leading e-commerce business.