Data Engineer
Summary
A Data Engineer in Johannesburg (hybrid) who designs and maintains scalable cloud data pipelines and ETL processes across AWS and Azure services (Glue, Redshift, S3, Lambda, Databricks, Microsoft Fabric), ensures data quality and security, and collaborates with data scientists and clients. Core stack: Python, SQL, big data tools, and IaC.
\"Join Us as a Data Alchemist: Turn Raw Data into Golden Insights!\"
Location:Johannesburg, SA - Hybrid | Salary:Competitive (Cause you’re worth it!)
Are you a data virtuoso with a flair for crafting seamless datahighways?We're on the hunt for a Data Engineer extraordinaire to join our client’s dynamic Technology Team. This role offers the uniqueopportunity to collaborate with clients, designing secure, scalable,and cost-effective solutions that harness the power of both on-premises technologies and Cloud platforms like Azure and AWS. If youpossess a blend of Cloud expertise, hands-on data engineeringexperience, and the ability to communicate complex ideas withcreativity and clarity, we want to hear from you!
Your Mission, Should You Choose to Accept:
- Design and implement scalable data pipelines using Cloud services such asGlue, Redshift, S3, Lambda, EMR, Athena, Microsoft Fabric, andDatabricks.
- Develop and maintain ETL processes to transform and integrate data fromdiverse sources.
- Collaborate with data scientists, analysts, and other stakeholderstounderstand data requirements and deliver top-notch data solutions.
- Optimize and fine-tune the performance of data pipelines and queries.
- Ensure data quality and integrity through rigorous testing andvalidation processes.
- Implement data security and compliance best practices.
- Monitor and troubleshoot data pipeline issues, ensuring timelyresolutions.
- Stay abreast of the latest developments in data engineeringtechnologies and best practices.
Your Toolkit:
- A Bachelor's degree from an accredited university.
- 3+ years, hands-on experience in data engineering; prior experience inthe financial services industry is a plus but not mandatory. Bonuspoints if you have experience with banks and insurers!
- Proven skills in building and managing data solutions using on-premisestechnologies or Cloud platforms.
- Familiarity with core Cloud Data Services, including Glue, Redshift, S3,Lambda, EMR, Athena, Microsoft Fabric, or Databricks.
- Knowledge of big data technologies such as Apache Spark, Hadoop, orKafka.
- Proficiency in programming languages like Python (with Pandas) andSQL.
- Experience working with relational databases like AWS RDS, MS SQL,Azure SQL DB, or Postgres.
- Solid understanding of data modeling, ETL processes, and datawarehousing.
- Experience with Infrastructure as Code (IaC) tools like AWSCloudFormation, Terraform, or Azure ARM/Bicep.
- Familiarity with CI/CD tools to streamline software development anddelivery.
- Excellent communication, problem-solving, and analytical skills, withthe ability to present complex technical concepts clearly andconcisely.
- Relevant Cloud certifications are a plus, showcasing your commitmentto professional development and mastery of Cloudservices and best practices.
- Strong desire to learn and enhance business knowledge.