Aws data engineer (senior) 1221 / 1154 tt
Summary
Senior AWS Data Engineer in Pretoria who builds and maintains big-data ETL pipelines on the client's AWS data platforms and acts as a custodian of data governance. Core stack: Terraform, Python/PySpark, Boto3, Docker, and AWS services like Glue, Lambda, S3, and DynamoDB.
Our client in Pretoria is recruiting for an AWS Data Engineer (Senior) to join their team. Data Engineers are responsible for building and maintaining Big Data Pipelines using our client’s Data Platforms. They are custodians of data and must ensure that data is shared in line with the information classification requirements on a need-to-know basis.
Qualifications/Experience: Relevant IT / Business / Engineering Degree Candidates with one or more of the following certifications are preferred: AWS Certified Cloud Practitioner AWS Certified Sys Ops Associate AWS Certified Developer Associate AWS Certified Architect Associate AWS Certified Architect Professional Hashicorp Certified Terraform Associate
Technical Skills: Terraform Python 3.x Py Spark Boto3 ETL Docker Powershell / Bash
Advantageous Skills Requirements: Exceptional analytical skills analyzing large and complex data sets Perform thorough testing and data validation to ensure the accuracy of data transformations Strong written and verbal communication skills, with precise documentation Self-driven team player with the ability to work independently and multi-task Experience in working with Enterprise Collaboration tools such as Confluence, JIRA, etc. Experience developing technical documentation and artifacts Knowledge of data formats such as Parquet, AVRO, JSON, XML, CSV, etc. Experience working with Data Quality Tools such as Great Expectations Experience developing and working with REST APIs is a bonus Experience building data pipelines using AWS Glue or Data Pipeline, or similar platforms Familiar with data stores such as AWS S3, AWS RDS, or Dynamo DB Experience and solid understanding of various software design patterns Experience preparing specifications from which programs will be written, designed, coded, tested, and debugged Strong organizational skills
Basic experience/understanding of AWS Components (in order of importance): Glue Cloud Watch SNS S3 Lambda Dynamo DB Step Function Param Store Secrets Manager Cloud Formation Business Intelligence (BI) Experience Technical data modeling and schema design (“not drag and drop”) Kafka AWS EMR Redshift Basic experience in Networking and troubleshooting network issues Knowledge of the Agile Working Model Package & Remuneration
Salary Market Related #J-18808-Ljbffr