Data Engineer
Posted Updated 4
views
Summary
Designs and maintains scalable data pipelines and cloud-based ETL/ELT workflows using Python, SQL, and Azure/AWS to deliver clean datasets for analytics.
Data Engineer
As a Data Engineer, you will be responsible for designing and maintaining robust, scalable data pipelines and architectures.
You will work with various data sources and cloud platforms to ensure efficient data integration, transformation, and storage for analytics and business use.
Key Responsibilities
- Build and manage end-to-end data pipelines and workflows
- Develop and maintain ETL/ELT processes for structured and unstructured data
- Integrate data from APIs, databases, and cloud storage
- Optimize data systems for performance, reliability, and cost-efficiency
- Collaborate with analysts and stakeholders to deliver clean, accessible datasets
- Monitor and troubleshoot data-related issues and bottlenecks
- Manage cloud infrastructure (Azure, AWS) for data storage and processing
- Document technical workflows and provide technical support to the team
Qualifications
- Degree in Computer Engineering, Data Science, Statistics, Physics, or a related IT field
- Proficient in Python (Apache Spark), SQL, and DAX
- Experience with Azure, Databricks, Power BI, and GitHub
- Knowledge of ETL/ELT, APIs, data modeling, data quality, and governance
- Familiarity with file types: parquet, CSV, JSON, etc.
- Knowledge of Agile methodologies (Scrum/Kanban)
- Nice to have: LS Central, Dynamics 365, Tableau, AL, HTML, CSS