Data engineer

The Data Engineer will architect, build, and continuously enhance enterprise-grade data infrastructure and pipelines, ensuring the delivery of high-quality, trusted, and scalable data that underpins effective decision‑making across Solar Africa’s expanding portfolio of IPP plants, off‑takers, and commercial partners.

Position of the job in the organisation:

The Data Engineer will report directly to the Head of IT.

Principal responsibilities Architect and implement scalable ELT/ETL pipelines on AWS and/or Azure, leveraging cloud‑native services (e.g., AWS Glue/S3/Redshift, Azure Data Factory/ADLS/Synapse) and Databricks where applicable Integrate and transform data from Financial, ERP (Dynamics 365), CRM, Sales, and operational systems into a unified analytical data platform Implement and enforce Medallion Architecture (Bronze → Silver → Gold) patterns, ensuring clean separation of raw ingestion, cleansed/conformed, and business‑ready data layers Develop rigorous pipeline testing frameworks including unit tests, data reconciliation, schema validation, row count checks, and end‑to‑end UAT coordination with business stakeholders Implement data quality monitoring, alerting, and governance controls — maintaining full lineage and documentation across all data assets Build and maintain Power BI semantic models and enterprise reports for operational, investor, and executive reporting Optimise cloud resource usage to ensure solutions are cost‑effective — including compute right‑sizing, scheduling optimisation, storage tiering, and active cloud spend monitoring Engage independently with internal and external stakeholders to translate business requirements into technical data solutions, managing delivery from scoping through to production deployment Maintain data documentation, including pipeline runbooks, schema registries, and architectural decision records (ADRs) Any other job duties associated with this role. Experience & Skills Bachelor’s degree in Computer Science, Information Systems, Engineering, Mathematics, or a related field 5–8 years of hands‑on Data Engineering experience with demonstrable ownership of senior‑level projects Strong SQL skills, including complex query optimisation, window functions, and performance tuning on large datasets Proven experience building production‑grade ELT/ETL pipelines using Python (Py Spark preferred) Hands‑on experience with AWS (Glue, S3, Redshift, Lambda, Step Functions) and/or Azure (ADF, ADLS Gen2, Synapse Analytics, Azure Databricks) Experience implementing Medallion Architecture (Bronze/Silver/Gold) in a cloud lakehouse environment Proficiency in Power BI, including Power Query, DAX modelling, semantic layer design, and publishing enterprise‑grade reports Demonstrated experience integrating ERP (Dynamics 365 preferred), CRM, and Financial/Sales data platforms into analytical environments Strong understanding of data governance, data quality frameworks, and access control best practices Experience with data warehousing solutions (Redshift, Synapse, Snowflake, or equivalent) Ability to produce architectural documentation: ERDs, data flow diagrams, source‑to‑target mappings Previous experience with ERP (preferably Dynamics 365) Strong Differentiators Hands‑on experience with Databricks (Delta Lake, Unity Catalog, Py Spark notebooks, Databricks Workflows) Experience in renewable energy, IPP, asset management, or infrastructure environments Specific attributes An architectural thinker who approaches new integrations with a design‑first mentality before writing code Applies test‑driven practices to data pipelines, not satisfied until data accuracy is independently verified Proactively manages cloud costs and flags inefficiencies without being prompted Can operate fully independently in an ambiguous environment, self‑managing priorities across concurrent projects Communicates technical concepts clearly to non‑technical stakeholders (finance, operations, executives) Desirable attributes Familiarity with Prefect 3 for orchestrating data ingestion workflows. Exposure to creating monitoring dashboards/alerts in Grafana Cloud What Does Success Look Like? By month 3 I will understand all core data sources, pipelines, and dependencies I will have documented existing data architecture and known data issues I can independentlymaintainand troubleshoot existing data pipelines I will be able to engage confidently with stakeholders on data requirements By month 6 I will have improved data reliability, accuracy, and pipeline stability I can design and deploy new data pipelines with minimal oversight I will have implemented basic data quality checks and monitoring I will be able to support analysts and business users with trusted datasets By month 12 I will have contributed to scalable, future‑ready data architecture I can proactivelyidentifyand resolve data risks before theyimpactthe business I will have reduced data‑related incidents and rework I will be able to enable faster, better decision‑making through reliable data

We are currently only considering candidates based in South Africa with valid working rights. Unfortunately, we are unable to consider applications from outside the country at this time. #J-18808-Ljbffr