Data Engineer
Posted Updated 8
views
Summary
Designs, builds, and maintains scalable ETL/ELT data pipelines using Databricks and AWS, writing optimized SQL and Python for data processing. The role also maintains data warehouses/data lakes, monitors data quality, and prepares clean datasets for Power BI reporting and analytics.
We are seeking a Data Engineer to develop and optimize scalable data solutions, transforming raw data into reliable datasets. This role involves collaboration with both business and technical teams to ensure high-quality data is available for reporting and analytics, making a significant impact on our data-driven decision-making processes.
Responsibilities
- Design, develop, and maintain scalable ETL/ELT data pipelines using Databricks and AWS.
- Build and optimize data workflows that ingest, transform, and load data from various internal and external sources.
- Integrate data from databases, APIs, cloud storage, and third-party applications.
- Develop and optimize SQL queries and Python scripts for efficient data processing.
- Maintain and improve data warehouses and data lake environments to ensure performance, reliability, and scalability.
- Monitor data quality, troubleshoot pipeline issues, and implement validation processes.
- Collaborate with business stakeholders and IT teams to understand reporting and analytics requirements.
- Support Power BI reporting by preparing clean and well-structured datasets.
- Document data pipelines, workflows, and technical processes.
- Participate in testing and production deployment to ensure stable and reliable data solutions.
Required Qualifications
- At least 3 years of experience in Data Engineering, ETL Development, or a related role.
- Hands-on experience using Databricks for developing or maintaining data pipelines.
- Experience working with AWS services for data storage, processing, or integration.
- Strong SQL skills, including query optimization and data transformation.
- Proficiency in Python for data engineering and automation.
- Experience building ETL/ELT pipelines and integrating multiple data sources.
- Familiarity with APIs, data modeling, and data quality best practices.
Preferred Qualifications
- Experience with GitHub or other version control tools.
- Knowledge of data warehousing concepts and best practices.
- Familiarity with data visualization tools like Power BI or Tableau.
- Experience in the retail or health & wellness industry.
- Strong problem-solving skills and attention to detail.