Data Engineer
Posted
Summary
Senior data engineer in Gauteng, South Africa, designing Kimball-methodology data warehouses, ETL pipelines, and dimensional databases while ensuring data quality, documentation, and stakeholder reporting. Core stack: SQL (ideally SAS Data Studio), AWS, Python or R, and CSV/JSON data handling.
Background:
- 3 Year IT related Degree
- Post graduate qualification (advantageous)
- 5-10 Years’ experience and understanding in designing and developing data warehouses according to the Kimball methodology
- Adept at design and development of ETL processes
- SQL development experience, preferably SAS data studio and AWS experience
- Ability to ingest/output CSV, JSON and other flat file types and any related data sources
- Proficient in Python or R
- Experience within Retail, Financial Services and Logistics environments
Responsibilities:
- Ensure highest data quality assurance, data accuracy, and data completeness through regular and in-depth review and testing of work
- Create easily understandable technical documentation that are kept up to date
- Conduct data design, database architecture, metadata and repository creation activities and tasks as required by business Stakeholder
- Translate business needs into long-term architecture solutions
- Define, design, and build dimensional databases
- Design the ETL pipelines
- Develop data warehousing blueprints, evaluating hardware and software platforms, and integrating systems
- Evaluate reusability of current data for additional analysis
- Translate Meta data into explanatory reports and visuals for easy understanding to end user
- Perform data pre-processing which includes data manipulation, transformation, normalisation, standardisation, visualisation, and derivation of new variables/features, as applicable to developing specific algorithms or models
- Build data quality metrics and conduct data validation testing
- Follow the IT governance process when implementing a change to ensure governance standards and protocols are followed
- Provide feedback to business owners on identified problems to ensure quality of data is rectified
- Conduct training and upskilling on new reports and or self-service analytics platforms to relevant Stakeholders
- Communicate with Stakeholders to keep them abreast of current developments within the function and to manage expectations
- Deal effectively and timeously with Customer complaints and ensure that complaints are resolved or escalated in line with agreed standards
- Deliver services and products to the customer within the parameters of the agreed SLA