Data Engineer
Summary
Designs, builds, and optimizes scalable ETL/ELT data pipelines and cloud data platforms (Snowflake, Spark, Airflow) using medallion architecture for a financial services client. Day to day: monitoring production pipelines, troubleshooting failures, and delivering well-modelled datasets for analytics teams in Colombo.
About Our Client
Our client is a well-established financial services organization seeking a skilled professional to support its data engineering and cloud data initiatives.
Job Overview
The Data Engineer will design, build, and optimize scalable data pipelines and cloud data platforms to support real-time analytics and data-driven decision-making.
Key Responsibilities
- Build and optimize batch and incremental ETL/ELT pipelines.
- Design data layers using the medallion architecture (Bronze/Silver/Gold), including CDC and SCD patterns.
- Monitor production pipelines, troubleshoot failures, and optimize compute and storage costs.
- Deliver high-quality, well-modelled datasets for analytics and data science teams.
- Write technical documentation, review code, and collaborate within an Agile team.
Qualifications & Requirements
- Degree in Computer Science, IT, Big Data, or a related field.
- 1–2+ years of experience with big data, on-premises, or cloud data platforms.
- Experience with Snowflake (preferred), Hadoop (HDFS), or AWS S3.
- Proficiency in Python or Java, with PySpark, Spark SQL, or Snowpark.
- Strong SQL skills, including MySQL, SQL Server, or Oracle.
- Experience with Airflow (preferred), Oozie, or Informatica.
- Familiarity with Git.
Interested candidates may send their CV to jobs@mindplus.global