Skip to content
Search jobs inside your AI agent. Claude, ChatGPT, Claude Code, Codex, Gemini, Cursor and Grok
Connect

Cf Consulting

NewBe an early applicant

Senior Data Engineer — Azure Data Platform & Analytics

Posted

Description

  • What Senior Data Engineer Azure Data Platform...

Showing 17 Senior Data Engineer Azure Data Platform Analytics jobs in Centurion

An established insurance company is seeking to hire a highly skilled and experienced Data Engineer to join their team. Your:

Formal Education:

  • Degree in Data Science, Information Technology, Computer Science or equivalent

Advantageous :

  • Cloud Data Certifications
  • Exposure to regulated environments,financial services, fintech

Experience:

  • Minimum of 2 years in a data engineer role or a similar technical role.

Responsibilities:

  • Build and maintain ETL pipelines supporting a multi-tenant data platform, ingesting data from APIs, databases, and event sources.
  • Build and maintain Data Platform APIs that allow teams to ingest, process, and access data easily and reliably.
  • Implemented tenant-specific logic by following existing configuration and naming conventions.
  • Apply tenant-level data isolation using schemas, partitions, or access controls
  • Build models from existing templates used for financial and operational reporting. Develop models for analytics and reporting, maintaining consistency with shared data models.
  • Monitor scheduled pipelines, investigate failures, and resolve data quality issues and inconsistencies.
  • Maintain daily and incremental data loads into the data warehouse.
  • Assist with onboarding new clients by validating source data and testing pipeline outputs.
  • Work closely with senior data engineers to learn patterns for multi-tenant data isolation.
  • Collaborate with analytics, product, and customer facing teams to understand reporting needs.
  • Support strict regulatory and audit requirements by following data handling,retention, and audit guidelines.
  • Handle financial and sensitive data (PII) according to company policies and regulatory standards (e.g. POPIA)
  • Apply least-privilege access and rolebased access controls, and support data protection through masking, encryption,and established security standards.

Technical Skills:

  • Programming languages Good knowledge of programming languages such as Python, especially used for pipeline and data manipulation.
  • SQL working experience using SQL for data cleaning, aggregation, data transformation and integration.
  • Data Warehouse - have a fundamental understanding of data warehousing solutions and platforms.
  • Databases hands on experience working with relational and non-relational databases.
  • Cloud computing comfortable building data solutions using cloud hosted services or data platforms.
  • Analytics skills strong problem solving skills, understand data characteristics, identify patterns, and data quality issues.
  • Data modelling and ETL Is able to communicate and translate business requirements into existing data models.
  • Data Pipeline Development build and validate smaller scale data pipelines independently.
  • CI/CD and Version Control apply best practice for managing pipelines and data workflows.
  • Bachelor's degree in Computer Science, Information Systems, Data Engineering or a related field.
  • Relevant Microsoft Fabric and/or Azure data certification.
REQUIREMENTS Minimum Education (essential)
  • Bachelor's degree in Computer Science, Information Systems, Data Engineering or a related field.
  • Relevant Microsoft Fabric and/or Azure data certification.
Minimum applicable experience (years):
  • 0-1 years of practical data engineering experience, including strong recent hands-on experience with Microsoft Fabric.
Required nature of experience:
  • Strong hands-on experience with Microsoft Fabric, including One Lake, Lakehouse, Warehouse, Data Pipelines/Data Factory, notebooks and Dataflows Gen2.
  • Strong SQL Server and T-SQL capability, including complex query development, schema design, indexing and performance optimisation.
  • Practical experience developing, maintaining and supporting production ETL/ELT pipelines.
  • Experience integrating and extracting data from REST/SOAP APIs, databases, flat files and other structured or unstructured data sources.
  • Proficiency in data transformation using SQL and Python/Py Spark, with an understanding of scalable data processing practices.
  • Practical experience in data warehousing, dimensional modelling, incremental loading, orchestration and schema evolution.
  • Experience with troubleshooting pipeline failures, data quality issues and performance bottlenecks, including the ability to restore service efficiently.
  • Experience with source control, CI/CD and deployment practices using Git, Azure Dev Ops or equivalent tools.
  • Experience supporting Power BI and other downstream analytical or reporting requirements.
  • Demonstrated ability to take ownership of an existing technical environment with limited hand-holding.
  • Strong documentation, communication and stakeholder engagement skills.
Skills and Knowledge (essential):
  • Microsoft Fabric: One Lake, Lakehouse, Warehouse, Data Pipelines/Data Factory, notebooks and Dataflows Gen2.
  • SQL Server / T-SQL
  • Python / Py Spark
  • REST/SOAP APIs and structured/unstructured data ingestion.
  • ETL/ELT, incremental loading, orchestration and scheduling.
  • Dimensional modelling, medallion architecture, schema evolution and data warehousing.
  • Power BI integration and understanding of downstream analytical requirements.
  • Git / Azure Dev Ops, CI/CD and environment deployment practices.
  • Monitoring, data quality, performance optimisation, security and operational support.
Other:
  • Proficient in Afrikaans and English.
  • Own transport and valid drivers license.
Remuneration Offered Market related

Our client is seeking a meticulous and experienced Data Engineer to build and optimize robust data pipelines for AI & Emerging Technologies initiatives in Centurion . In this critical role, you will be responsible for ensuring the availability, quality, and accessibility of data required for machine learning models and AI-driven applications. You will work closely with data scientists and ML engineers to architect scalable data solutions, fostering a data-centric culture. This on-site position provides a unique opportunity to work with a dedicated team in a collaborative and innovative environment.

About the Role

Our client is seeking a meticulous and experienced Data Engineer to build and optimize robust data pipelines for AI & Emerging Technologies initiatives in Centurion . In this critical role, you will be responsible for ensuring the availability, quality, and accessibility of data required for machine learning models and AI-driven applications. You will work closely with data scientists and ML engineers to architect scalable data solutions, fostering a data-centric culture. This on-site position provides a unique opportunity to work with a dedicated team in a collaborative and innovative environment.

Key Responsibilities
  • Design, construct, install, test, and maintain highly scalable data management systems and pipelines.
  • Develop processes and systems to monitor data quality, ensure consistency, and manage data flow.
  • Build and manage ETL/ELT processes for large, diverse datasets.
  • Optimize data infrastructure for performance, reliability, and cost-effectiveness.
  • Collaborate with data scientists and ML engineers to understand their data requirements and provide solutions.
  • Implement data security and governance best practices.
Requirements
  • Bachelor's or Master's degree in Computer Science, Engineering, or a related field.
  • 3+ years of experience in data engineering, with a focus on building large-scale data pipelines.
  • Proficiency in SQL and programming languages like Python or Scala.
  • Experience with big data technologies such as Spark, Hadoop, Kafka, and distributed storage systems.
  • Hands-on experience with cloud data services (AWS, Azure, GCP).
  • Strong understanding of data warehousing concepts and database design.
Benefits
  • Competitive annual salary and performance incentives.
  • On-site work environment with access to advanced facilities.
  • Comprehensive medical, dental, and vision insurance plans.
  • Opportunities for professional development and certifications.
  • Collaborative team atmosphere focused on innovation and growth.

Skills

What Senior Data Engineering jobs ask for — and how much of it you have →
Apply

See also

Data Engineering jobs by country — openings, pay and top skills →