Data Engineer
Posted Updated 1
view
Summary
Design and maintain scalable ETL/ELT data pipelines and medallion architecture solutions. You will integrate client source systems (ERP, CRM, SaaS), develop data models, and ensure data quality using technologies like SQL, Python, PySpark, and Microsoft Fabric or Azure Data Factory.
JOB DESCRIPTION
The Data Engineer will design, build, and maintain modern data solutions that support analytics, AI, reporting, and business transformation. The role will focus on developing reliable and scalable data pipelines, integrating client source systems, maintaining data quality, and improving data platform performance.
The Data Engineer will design, build, and maintain modern data solutions that support analytics, AI, reporting, and business transformation. The role will focus on developing reliable and scalable data pipelines, integrating client source systems, maintaining data quality, and improving data platform performance.
The ideal candidate combines strong technical data engineering skills with a consulting mindset, the ability to communicate effectively with both technical and non-technical stakeholders, and a strong commitment to quality, collaboration, and continuous improvement.
JOB RESPONSIBILITIES
Data Pipeline Architecture & Delivery
Data Pipeline Architecture & Delivery
- Design and implement scalable ETL/ELT data pipelines using modern cloud data platforms.
- Build medallion architecture solutions across Bronze, Silver, and Gold data layers.
- Develop data pipelines using Spark notebooks, data pipelines, lakehouses, and related data engineering technologies.
- Build reliable data foundations by connecting source systems, establishing data models, and implementing data quality standards.
- Optimize data pipelines for reliability, scalability, performance, and maintainability.
Source System Integration
- Build and maintain integrations with ERP, CRM, SaaS, and other source systems.
- Develop reliable data workflows using integration platforms, APIs, native connectors, and other appropriate technologies.
- Support both data ingestion and reverse ETL processes where required.
- Monitor integrations and address connector, schema, and data quality issues.
Data Modeling, Governance & Quality
- Develop and maintain effective data models aligned with reporting, analytics, and business requirements.
- Support semantic layer design and ensure data structures are optimized for downstream analytics.
- Implement data quality monitoring, validation, lineage, and access controls.
- Identify data quality risks and recommend practical improvements.
- Ensure solutions follow appropriate standards for security, governance, testing, and documentation.
Monitoring & Continuous Improvement
- Monitor pipeline performance, connector health, schema changes, and data quality.
- Troubleshoot and resolve data integration, transformation, and performance issues.
- Identify opportunities for automation, standardization, and process improvement.
- Develop and maintain reusable pipeline patterns, delivery templates, documentation, and engineering playbooks.
- Support ongoing enhancements, new source integrations, platform expansion, and optimization initiatives.
Collaboration & Client Engagement
- Collaborate with Analytics Engineers to ensure data models and pipelines support business and reporting requirements.
- Work closely with project managers, solution architects, engineers, and other technical teams.
- Participate in discovery sessions, workshops, solution design discussions, and client-facing engagements.
- Translate technical concepts, risks, tradeoffs, and recommendations into clear business terms.
- Proactively communicate project risks, dependencies, assumptions, and potential challenges.
- Contribute to mentoring, knowledge sharing, and continuous improvement across the engineering team.
QUALIFICATIONS
- 3+ years of experience building and supporting production data pipelines using ETL/ELT processes in cloud or modern data environments.
- Strong experience with modern data platforms and data engineering technologies.
- Proficiency in SQL, Python, and PySpark or equivalent data processing technologies.
- Strong data modeling experience, including dimensional modeling and modern data architecture patterns.
- Experience building, maintaining, and troubleshooting ETL/ELT and reverse ETL processes.
- Experience with Microsoft Fabric, Azure Data Factory, or an equivalent modern cloud data platform.
- Experience with data integration platforms such as Fivetran, CData, native connectors, APIs, or equivalent technologies.
- Understanding of data governance fundamentals, including data quality, lineage, access controls, and monitoring.
- Strong problem-solving and analytical skills.
- Strong written and verbal communication skills.
- Ability to collaborate effectively with engineering, analytics, BI, project management, and business teams.
- Strong documentation and organizational skills.
- Ability to work independently while contributing effectively within a collaborative team environment.
- Comfortable receiving and providing constructive feedback.
- Ability to understand client objectives and translate business requirements into practical technical solutions.
- NICE TO HAVE
- Hands-on experience with Microsoft Fabric.
- Consulting or client-facing experience.
- Experience working in collaborative pod or cross-functional team structures.
- Experience with Spark, Lakehouse architectures, and medallion architecture.
- Experience with Fivetran, CData, or similar integration platforms.
- Experience supporting analytics, AI, or business intelligence initiatives.
- Experience developing reusable engineering standards, playbooks, and delivery assets.
JOB REQUIREMENTS