Middle Data Engineer

# Middle Data EngineerHybrid,RemoteColombiaMedellínWe are looking for a Middle Data Engineer to help build and maintain a data platform that supports data engineering, BI, Machine Learning, and AI initiatives. In this role, you will design batch and real-time data pipelines, work with distributed data processing tools, and collaborate with business and technical stakeholders to deliver reliable data solutions.## Required for this role* **3–5 years of experience** building **batch and real-time data pipelines** using big data technologies.* Experience with distributed data processing tools such as **Spark**, **Hadoop**, **Airflow**, **NiFi**, and **Kafka**.* Hands-on experience designing, developing, and maintaining **ETL processes** and data pipelines.* Proficiency in **Python** and/or **Scala**, including the ability to optimize data processing scripts.* Strong knowledge of **SQL** and experience designing scalable **data models**.* Familiarity with cloud-based data platforms such as **Snowflake**, **Databricks**, **AWS**, **Azure**, or **GCP**.* Experience using **CI/CD** tools and working with **Docker** and **Kubernetes** for data pipeline automation.* Ability to troubleshoot, monitor, and optimize data workflows for reliability and performance.* Basic understanding of **data governance**, **data quality frameworks**, and compliance requirements.## Nice to have* Experience with **OpenShift**, **Trino**, **Ranger**, and **Hive**.* Knowledge of **Machine Learning** and data science concepts and tools.* Familiarity with BI and analytics tools such as **Tableau** and **Superset**.* Good communication skills, attention to detail, Agile mindset, and flexibility.## Your responsibilities* Work with business stakeholders and cross-functional teams to understand data requirements and deliver scalable data solutions.* Design, develop, and maintain ETL processes that move data from different sources into the data platform.* Build batch and event-driven data pipelines using cloud and on-premises hybrid data platforms.* Collaborate with data architects to review solutions and data models, ensuring alignment with best practices.* Take ownership of assigned deliverables and ensure high-quality implementation within agreed timelines.* Implement data quality standards and collaborate with governance teams to support compliance with data policies.* Optimize data integration workflows for performance, reliability, and maintainability.* Troubleshoot and resolve data integration and processing issues.* Apply an agile mindset when working with engineers and stakeholders to experiment, iterate, and deliver new initiatives.## What you get### Your time off* Up to 15 paid vacation days annually* Additional unpaid days off* 18 Colombian public holidays### Learning & growth* Sombra University workshops and internal learning programs* Tech Communities and knowledge sharing sessions* Language courses and workshops* Mentorship opportunities* Access to the Platzi online learning platform### And even more* Company-provided work equipment* Internal referral program* Bonuses for getting married or the birth of a child* Sombra events and internal initiatives## Before you applyOur recruitment team will carefully review your profile, and if we see a good match with the role, we’ll reach out to you shortly.If you don’t hear from us within 5 business days, it means we’ve decided to continue the process with other candidates for this position. Thanks for understanding.Kateryna KyryiaRecruitment PartnerSend CV## Apply now! #J-18808-Ljbffr