Lead Data Engineer - Python/SQL/ETL

Requirements :

- Strong experience in AWS Data Engineering: S3, Redshift, EMR, Glue, Lambda

- Hands-on expertise in Spark, PySpark, Spark SQL

- Experience with Hadoop, Hive, Kafka

- Strong proficiency in Python, SQL, and preferably Scala

- Experience building ETL/ELT pipelines and large-scale data platforms

- Strong knowledge of Data Warehousing, Data Modeling (Star/Snowflake Schema

- Experience with Apache Airflow or similar orchestration tools

- Exposure to Databricks, Snowflake, Delta Lake is a plus

- Experience in performance tuning, data quality, and production support

- Ability to lead technical discussions, mentor developers, and collaborate with stakeholders

Key Responsibilities :

- Architect and maintain scalable ETL/ELT pipelines to ingest, process, and transform massive datasets for downstream analytical consumption.

- Design sophisticated data models that ensure data integrity, performance, and ease of access for business intelligence reporting.

- Optimize data warehouse performance by fine-tuning storage strategies and query execution plans to support high-concurrency environments.

- Lead the migration and integration of data workloads onto AWS cloud infrastructure to improve system elasticity and cost-efficiency.

- Implement real-time data streaming solutions using Kafka to enable low-latency data availability for critical business applications.

- Mentor junior engineers and conduct code reviews to enforce best practices in software engineering and data quality standards.