Data Platform Engineer/ Data Engineer
Summary
Design and run scalable data pipelines and platform infrastructure: orchestrating workflows with Apache Airflow, managing an enterprise Data Lake, deploying containerized services on Kubernetes, and automating CI/CD with DevOps practices. Collaborates with data scientists, analysts, and IT infra teams; on-call for pipeline health and incident troubleshooting.
Design, construct, and maintain scalable data ingestion and processing workflows using Apache Airflow.
Manage, optimize, and expand enterprise Data Lake infrastructure to ensure high availability, performance, and data integrity.
Maintain and deploy containerized data services and pipeline workloads on Kubernetes clusters.
Integrate DevOps best practices, automated CI/CD pipelines, and infrastructure management across data platforms.
Monitor data pipeline health, troubleshoot production incidents, and perform root-cause analysis to ensure continuous operational uptime.
Collaborate closely with data scientists, business analysts, and IT infrastructure teams to refine data integration capabilities.
Requirements
Bachelor’s degree in Computer Science, Information Technology, Software Engineering, or a related discipline.
2 to 4 years of hands‑on experience in data engineering, data platform engineering, or DevOps roles.
Strong, hands‑on expertise with Apache Airflow for workflow orchestration and pipeline management.
Proven experience managing and optimizing Data Lake architectures.
Working knowledge of Kubernetes and containerization technologies (Docker).
Practical experience with DevOps methodologies, CI/CD tools, and automated deployment pipelines.
Proficiency in Python, SQL, and Shell scripting for system automation and data processing.
Good communication, problem‑solving, and analytical skills.
Desirable Qualifications
Experience working within financial institutions, banking environments, or public sector bodies.
Exposure to cloud platforms (AWS, Azure, GCP) or hybrid cloud infrastructure.