Lead Data Engineer - Spark / Scala
Posted Updated
Summary
Lead Data Engineer building ETL data pipelines that pull from flat files, databases, and APIs using Spark, Hive, and Scala. The role involves Python/Pandas data transformation, advanced SQL, Airflow pipeline orchestration, and BI dashboards in tools like Power BI, Tableau, or Looker.
Notice Period : Immediate to 15 Days.
Job Description :
- 6+ years of overall Data Analytics and BI experience
- Experience in Spark, Hive, Scala.
- Build data pipelines for ETL that fetch data from variety of sources such as flat files relational databases and APIs
- Python scripting with focus on data transformation and manipulation libraries such as Pandas and numpy
- Strong knowledge and hands-on experience of SQL (should be able to write advance level SQL queries)
- Good hands-on experience on data visualization tools such as Power BI, Tableau, Looker
- Good understanding and hands-on experience of Data engineering pipeline management tools such as Airflow
- Good Communication skills.