Azure Data Engineer
Summary
Builds and maintains Azure-based data pipelines for a bank using Databricks, PySpark, and Kafka to ingest, process, and report on financial data in real time.
Role: Data Engineer
Location: Johannesburg
Job Type: Contract
Duration: 12+ Months
Client: Banking Domain
Please Note: Must be certified in Databricks
Experience:
- Data Collection and Analysis: Works with full competence to determine and analyze trends from collected data to assist in compiling reports that support business decisions. Typically works without supervision and may provide technical guidance.
- Engineering: Deep expertise In major cloud Azure platform, Azure Data Factory, Microsoft Fabric, Databricks, Python, PySpark. Infrastructure as Code
- Database User Interfaces and Queries: Works with full competence to create and run queries and interact with various database interfaces and query languages. Typically works without supervision and may provide technical guidance. Experience with low latency data Ingestion and processing using Message queues, Kafka, Azure Event Hub, Real time Intelligence.
- Data Conversion: Works with full competence to use data conversion tools and techniques to encode data in various formats. Typically works without supervision and may provide technical guidance.
- Database Reporting: Works with full competence to use database reporting tools and techniques. Typically works without supervision and may provide technical guidance.
- Orchestration and Data Ops proven experience with workflow orchestration and Implementing CI/Cd pipelines for data solutions
- Application Development: Works with full competence to develop software through use of programming languages. Typically works without supervision and may provide technical guidance.