Senior Data Engineer
Posted 36
views
Summary
Senior Data Engineer in Colombo building and optimizing scalable batch and incremental ETL/ELT pipelines and cloud data platforms (Snowflake, Spark, Airflow), designing medallion data layers with CDC/SCD to power real-time analytics and decision-making while optimizing compute and storage costs.
We're looking for a Senior Data Engineer to build and optimize scalable data pipelines and cloud data platforms that power real-time analytics and decision-making. If you're passionate about modern data engineering, we'd love to hear from you.
What You'll Do
- Build and optimize batch and incremental ETL/ELT pipelines
- Design medallion (bronze/silver/gold) data layers with CDC and SCD
- Monitor and troubleshoot production pipelines, and optimize compute and storage costs
- Deliver high-quality, well-modeled datasets for analytics and data science teams
- Document your work, review code and collaborate in an Agile team
Candidate Profile
- Degree in Computer Science, IT, Big Data or a related field, or equivalent experience
- 5+ years of hands-on data engineering on big data or cloud data platforms
- Strong production experience with Snowflake (preferred), Hadoop (HDFS) or AWS (S3)
- Strong Python or Java; hands-on with PySpark, Spark SQL or Snowpark
- Advanced SQL (MySQL, SQL Server or Oracle), including query tuning and optimization
- Production experience with Airflow (preferred), Oozie or Informatica
- Proven experience implementing CDC, SCD and medallion architectures
- Track record of troubleshooting production pipelines and reducing compute and storage costs
- Git, CI/CD and code review experience