Strong Middle Data Engineer
Posted Updated 4
views
Summary
Designs and builds scalable data platforms and pipelines for analytics and operations using Azure (Data Factory, Synapse, Event Hub, Data Lake Storage), Apache Spark/pySpark, and Databricks. Day to day: modeling data, integrating sources, and monitoring and optimizing pipeline performance with analysts, scientists, and engineers.
- Design and develop scalable, reliable data platforms and pipelines for analytics and operations
- Implement data processing workflows using distributed frameworks (Apache Spark/pySpark, Databricks)
- Evolve the organization’s cloud data platform leveraging Azure technologies (Data Factory, Synapse, Event Hub, Azure Data Lake Storage)
- Model data to support analytics, reporting, and downstream consumption
- Integrate and process data from multiple sources ensuring quality and consistency
- Monitor and optimize pipeline performance, ensuring reliability and scalability
- Collaborate with analysts, scientists, and engineers to translate requirements into solutions
- Improve data engineering standards, testing, and operational reliability
- Evaluate and introduce new tools and technologies in the data engineering ecosystem
- 4+ years of hands-on experience in data engineering or building data processing platforms
- Strong SQL skills and solid Python experience for pipeline development
- Experience with distributed processing frameworks (Apache Spark/pySpark, Databricks preferred)
- Practical experience designing and implementing pipelines in cloud environments (Azure preferred)
- Experience with production-scale analytical systems and data modeling
- Understanding of ETL/ELT design, dimensional modeling, and data warehousing principles
- Experience with modern data lake architectures and formats (Parquet, JSON)
- Familiarity with workflow orchestration tools (Airflow)
- Experience with database development and optimization
- Ability to design scalable solutions with minimal supervision
- Strong collaboration and communication skills
- Professional proficiency in English