Senior Data Engineer
Summary
Senior Data Engineer in Colombo for a financial-services client of MindPlus: designing and optimizing batch/incremental ETL/ELT pipelines and cloud data platforms (Snowflake, Hadoop/S3), building medallion-architecture datasets with CDC/SCD using Python/Java, PySpark, SQL, and Airflow to power real-time analytics.
About Our Client
Our client is a well-established financial services organization focused on delivering technology-driven solutions and data-led business insights.
Job Overview
The Senior Data Engineer will design, build, and optimize scalable data pipelines and cloud data platforms to support real-time analytics and business decision-making.
Key Responsibilities
- Build and optimize batch and incremental ETL/ELT pipelines.
- Design medallion data architecture (Bronze, Silver, and Gold) using CDC and SCD.
- Monitor and troubleshoot production data pipelines while optimizing compute and storage costs.
- Develop high-quality, well-modelled datasets for analytics and data science teams.
- Document technical solutions, review code, and collaborate within an Agile team.
Qualifications & Requirements
- Degree in Computer Science, IT, Big Data, or a related field, or equivalent experience.
- Minimum 5 years of hands-on experience in data engineering using big data or cloud data platforms.
- Strong production experience with Snowflake (preferred), Hadoop (HDFS), or AWS S3.
- Proficiency in Python or Java, with hands-on experience in PySpark, Spark SQL, or Snowpark.
- Advanced SQL skills, including query tuning and optimization.
- Experience with Airflow (preferred), Oozie, or Informatica.
- Proven knowledge of CDC, SCD, and medallion architecture.
- Experience with Git, CI/CD, and code reviews.
- Strong troubleshooting and problem-solving skills.
Interested candidates may send their CV to jobs@mindplus.global