Data Engineer
Posted Updated
Summary
Build, optimize and maintain end-to-end Databricks data pipelines (Medallion Architecture, PySpark, Airflow/ADF orchestration, Azure DevOps CI/CD) for a technology consulting firm serving a major banking and financial-services client. Hybrid role based in Lisbon.
Join a team where Databricks is at the heart of the engineering strategy and where your work powers large-scale financial platforms used across global markets
- Location: Flexible Hybrid in Lisbon
- Contract: Full-time, direct contract with a global technology consulting leader
Our client is a world-renowned technology consulting company partnering with a major multinational in banking and financial services. This long-term program is part of a data and cloud modernization journey centred on Databricks, scalable architectures and high-quality data products. You will join an international engineering environment dedicated to transforming complex data into reliable, governed and business-critical assets.
Responsibilities
- Build, optimize and maintain end-to-end Databricks pipelines, ensuring performance, reliability and scalability.
- Apply Medallion Architecture principles to design structured, maintainable and production-ready data layers.
- Develop distributed processing solutions with PySpark, aligned with best engineering practices.
- Design and orchestrate workflows using ADF and Airflow, improving automation and observability.
- Contribute to the evolution of Databricks Lakehouse standards, performance tuning and cluster optimization.
- Integrate CI/CD practices using Azure DevOps, with automated testing and version control.
- Collaborate closely with data scientists, architects and business stakeholders to operationalize data products and ensure alignment with financial-sector requirements.
- Support governance, security and documentation across the Databricks ecosystem.
Required Skills
- Strong, demonstrable experience with Databricks as the primary engineering platform.
- Solid understanding of Medallion Architecture and modern lakehouse principles.
- Proficiency in PySpark and distributed processing frameworks.
- Experience with Azure Data Factory, Airflow and workflow orchestration.
- Familiarity with Azure DevOps CI/CD, Git workflows and automated deployments.
- Comfortable working in Agile, collaborative and multicultural environments.
?
Nice to Have
- Advanced experience with Azure DevOps for pipeline automation.
- Background in data testing or test framework development.
- Exposure to investment banking or financial services data landscapes.
?
Ready to take the next step?
Apply now and join an international Databricks-driven project shaping the future of digital innovation in financial services.
Candidate-se aqui
- Location: Flexible Hybrid in Lisbon
- Contract: Full-time, direct contract with a global technology consulting leader
Our client is a world-renowned technology consulting company partnering with a major multinational in banking and financial services. This long-term program is part of a data and cloud modernization journey centred on Databricks, scalable architectures and high-quality data products. You will join an international engineering environment dedicated to transforming complex data into reliable, governed and business-critical assets.
Responsibilities
- Build, optimize and maintain end-to-end Databricks pipelines, ensuring performance, reliability and scalability.
- Apply Medallion Architecture principles to design structured, maintainable and production-ready data layers.
- Develop distributed processing solutions with PySpark, aligned with best engineering practices.
- Design and orchestrate workflows using ADF and Airflow, improving automation and observability.
- Contribute to the evolution of Databricks Lakehouse standards, performance tuning and cluster optimization.
- Integrate CI/CD practices using Azure DevOps, with automated testing and version control.
- Collaborate closely with data scientists, architects and business stakeholders to operationalize data products and ensure alignment with financial-sector requirements.
- Support governance, security and documentation across the Databricks ecosystem.
Required Skills
- Strong, demonstrable experience with Databricks as the primary engineering platform.
- Solid understanding of Medallion Architecture and modern lakehouse principles.
- Proficiency in PySpark and distributed processing frameworks.
- Experience with Azure Data Factory, Airflow and workflow orchestration.
- Familiarity with Azure DevOps CI/CD, Git workflows and automated deployments.
- Comfortable working in Agile, collaborative and multicultural environments.
?
Nice to Have
- Advanced experience with Azure DevOps for pipeline automation.
- Background in data testing or test framework development.
- Exposure to investment banking or financial services data landscapes.
?
Ready to take the next step?
Apply now and join an international Databricks-driven project shaping the future of digital innovation in financial services.
Candidate-se aqui