Sr. Data Engineer (PySpark, Azure Databricks)
Posted 34
views
Description
We are seeking a highly skilled Senior Data Engineer with extensive experience in designing, building, and optimizing high-volume data pipelines.
This is a fully remote job opportunity!
Responsibilities
- Adhere to coding and Numerator technology standards
- Build suitable automation test suites within Azure DevOps
- Maintain and update automation test suites as required
- Carry out manual testing, load testing, exploratory testing as required
- Work closely with Business Analysts and Senior Data Developers to consistently achieve sprint goals
- Assist in estimation of sprint-by-sprint stories and tasks
- Pro-actively take a responsible approach to product delivery
Requirements
- 7-10 years of experience in data engineering roles
- Good Python skills
- Experience working with Microsoft Azure Cloud
- Experience in Agile methodologies (Scrum/Kanban)
- Experience with Apache Spark, PySpark, Databricks
- Experience working with Devops pipeline, preferably Azure DevOps
Preferred Qualifications
- Bachelor's or master's degree in computer science, Information Technology, Data Science, or a related field.
- Experience working in a support focused role
- Knowledge/experience in AI ML techniques
- Certification in relevant Data Engineering discipline or related fields.