Senior Data Engineer - Cloud Platforms

About the Role

Our client, a leading enterprise in data analytics, is seeking a highly skilled Senior Data Engineer to join their robust technology team. This role is critical for designing, building, and maintaining scalable and reliable data pipelines and infrastructure. You will be instrumental in transforming raw data into actionable insights, supporting various business intelligence and machine learning initiatives. This position is based in our modern Benoni office and requires a strong background in cloud-based data solutions and big data technologies. The ideal candidate is passionate about data, possesses exceptional problem-solving skills, and thrives in a collaborative, fast-paced environment. You will work with a team of talented engineers and data scientists to architect and implement cutting-edge data solutions.

Key Responsibilities

• Design, develop, and optimize robust, scalable data pipelines and ETL/ELT processes.
• Build and maintain data warehousing solutions and data lakes on cloud platforms (AWS, Azure, or GCP).
• Implement data governance, data quality, and data security best practices.
• Collaborate with data scientists, analysts, and business stakeholders to understand data requirements.
• Develop and manage infrastructure for big data processing using technologies like Spark, Kafka, or Flink.
• Monitor data system performance, troubleshoot issues, and implement solutions to ensure high availability and efficiency.
• Automate data processes and deployments using CI/CD principles.
• Stay current with emerging data technologies and best practices, recommending and implementing new tools and techniques.
• Document data architecture, processes, and standards.
• Contribute to the architectural design of data platforms and solutions.

Requirements

• Bachelor's degree in Computer Science, Engineering, Information Technology, or a related quantitative field.
• Master's degree is a plus.
• 5+ years of experience in data engineering, with a strong focus on building and managing data infrastructure.
• Proven experience with cloud data services (e.g., AWS S3, Redshift, Glue; Azure Data Lake, Databricks, Synapse; GCP Cloud Storage, BigQuery, Dataflow).
• Proficiency in programming languages such as Python, Scala, or Java.
• Strong SQL skills and experience with various database technologies (relational and NoSQL).
• Experience with big data technologies like Apache Spark, Hadoop ecosystem.
• Familiarity with data orchestration tools (e.g., Airflow, Prefect).
• Understanding of data warehousing concepts and dimensional modeling.
• Excellent problem-solving, analytical, and communication skills.
• Experience working in an Agile development environment.
• Ability to work effectively in a team and collaborate with cross-functional stakeholders.

Benefits

Our client offers a competitive salary, performance-based bonuses, and comprehensive health, dental, and vision benefits. We provide a retirement savings plan, ample paid time off, and continuous opportunities for professional development, including training, certifications, and conference attendance. This role is based in our state-of-the-art Benoni office, fostering a collaborative and innovative work environment. Join our client and contribute to building the future of data infrastructure in a company that values innovation and employee growth.