Data Engineer
Posted Updated
Description
ATS Client in Johannesburg is seeking an experienced Data Engineer to lead the delivery of a global Datalake and build data pipelines that support commercial data propositions and business performance management. You will work with Product managers and stakeholders to enable revenue growth through data-driven decision making.
Responsibilities include designing data architectures, ETL development, and ensuring data quality across on-premise and cloud platforms (AWS, Azure, Oracle Cloud).
Implement and evolve the strategic data architecture for the business. Manage the data delivery roadmap across the function. Design, develop, and maintain scalable data pipelines and ETL processes using Databricks. Write efficient SQL queries for data extraction and transformation. Develop and maintain data integration workflows using Python, Scala, and Java. Collaborate with data scientists, analysts, and stakeholders to understand data requirements and deliver solutions. Ensure data quality and integrity through testing and validation. Monitor and troubleshoot data pipeline performance and reliability. Stay up-to-date with industry trends and technologies in data engineering. Bachelor's degree in Computer Science, Engineering, or a related field. Proven experience as a Data Engineer or in a similar role. Proficiency in Databricks, SQL, Python, Scala, and Java, Oracle. Experience with building Graph, Vector, NoSQL and data lakehouse architectures. Strong understanding of data warehousing concepts and ETL processes. Experience with on-premise and cloud platform architectures (AWS, Azure and Oracle Cloud). Excellent problem-solving skills and attention to detail. Strong communication and collaboration skills. Ability to work in a fast-paced, dynamic environment. Outstanding verbal and written communication skills. Excellent interpersonal and organisational skills. Strong commercial awareness in all areas.Skills
Sign in to read and join the discussion.