Data Engineer
Summary
Designs, builds, and maintains scalable ETL/ELT data pipelines using Python, PySpark, Databricks, and Snowflake to ensure data quality and enable analytics. Collaborates with cross-functional teams to deliver data solutions.
- Design, build, and maintain scalable ETL/ELT data pipelines.
- Develop data processing workflows using Python and PySpark.
- Work with Databricks and Snowflake for data storage, transformation, and analytics.
- Integrate data from multiple sources while ensuring data quality and integrity.
- Optimize SQL queries and improve data pipeline performance.
- Collaborate with Data Analysts, Data Scientists, and Software Engineers to deliver data solutions.
- Monitor, troubleshoot, and maintain data workflows.
- Document data models, pipelines, and technical processes.
Requirements
- Bachelor's degree in Computer Science, Information Technology, Data Science, or a related field.
- Strong proficiency in Python, PySpark, and SQL.
- Hands-on experience with Databricks and Snowflake.
- Good understanding of ETL/ELT pipeline development.
- Familiarity with PyTorch is an added advantage.
- Knowledge of data warehousing concepts and cloud platforms is preferred.
- Strong analytical and problem-solving skills.
- Excellent communication and teamwork abilities.