Data Engineer - Azure, Python, Pyspark
Posted Updated
Description
Role Summary
Join our team as a Data Engineer, where your expertise will directly influence our data-driven initiatives. Your role will be pivotal in shaping how our organization manages and optimizes its data workflows.
Responsibilities
- Design and implement scalable data pipelines using Azure, Python, and PySpark to drive data processing efficiency.
- Collaborate with data scientists and analysts to gather requirements and develop effective data solutions.
- Enhance performance and reduce costs through optimized data storage and retrieval processes.
- Set up data quality controls and monitoring systems to maintain the integrity of data across platforms.
- Utilize Data Bricks for the management and analysis of extensive datasets, maximizing collaborative efforts in analytics.
- Keep abreast of emerging trends in the data engineering field to refine practices and adopt new technologies.
Must-Have Skills
- Expertise in Azure cloud services, with hands-on experience in relevant data engineering tools.
- Strong programming capabilities in Python, particularly for tasks involving data manipulation and analysis.
- Proficiency in PySpark for processing large-scale datasets effectively.
- Familiarity with Data Bricks as a platform for collaborative data analytics work.
- Excellent analytical skills with a keen attention to detail.
- Strong communication skills for effective collaboration with cross-functional stakeholders.
Good-to-Have Skills
- Knowledge of additional big data technologies or frameworks.
- Experience in implementing machine learning algorithms or advanced analytics techniques.
Skills
Sign in to read and join the discussion.