Azure Data Engineer
Summary
Build and maintain Azure Data Factory pipelines, architect ETL frameworks, and transform data using PySpark and SQL for a data-driven product company.
Experience Required: 8 Years
Role Description:
Looking for a Data Engineer with strong experience in ADF, ADB, PySpark, SQL, and architecting ETL frameworks.
PySpark
SQL
Architecting ETL Frameworks
Required Skills:
Strong communication skills with the ability to liaise and coordinate with SMEs and technical partners
Strong ability to break down technical requirements into actionable tasks
Proficient with relational databases such as MS SQL and SQL scripting
Proficient in building and maintaining Azure Data Factory pipelines for data ingestion orchestration
Good OOP understanding with experience in Python or Java
Proficiency with Git (Bitbucket or GitHub)
Good understanding of CI/CD concepts for building and deploying artifacts
Understanding of synchronous and asynchronous data encryption/decryption
Nice to Have Skills:
Azure Databricks Notebooks or Apache Spark for data transformations
Datadog dashboard and monitoring experience
Understanding of Parquet and Delta Lake concepts
Experience with XLR for CI/CD
Experience with Flyway for data migrations