Azure data engineer in analytics platforms

Описание

We are looking for an experienced Data Engineer to join the Data Services ART within the Client Data Analytics Platform (ADAP). The role involves designing, developing and operating scalable, secure and reliable data solutions to support analytics, AI and business-critical applications. The ideal candidate will have strong hands-on experience in Azure cloud-native data engineering, Azure Databricks, large-scale data pipelines, infrastructure as code and automated CI/CD deployments.

Задачи

Design, develop, test and maintain scalable batch and streaming data pipelines, datasets, data models and data products. Build cloud-native data solutions using Azure Databricks, Azure Data Lake Storage (ADLS), Azure Event Hubs, Apache Spark and Delta Lake. Develop and manage lakehouse architectures and high-performance data processing solutions. Provision and manage cloud infrastructure using Terraform and Infrastructure as Code (IaC) practices. Build, maintain and optimize CI/CD pipelines using Azure DevOps. Apply software engineering best practices, including automated testing, version control, code quality, documentation and deployment automation. Implement monitoring, observability and operational controls to improve reliability, scalability, performance and cost efficiency. Ensure data quality, metadata management, governance, security and production readiness. Troubleshoot production issues, perform root-cause analysis and implement preventive measures. Collaborate with architects, product owners, engineers and business stakeholders to translate requirements into scalable technical solutions. Participate in agile ceremonies, code reviews, proof of concepts, knowledge sharing and continuous platform improvement.

Требования

8+ years of experience in data engineering, software engineering or a related technical role. Strong hands-on experience designing, building and operating large-scale data platforms or cloud-based data solutions. Strong proficiency in SQL, Python, data modelling and data pipeline development. Hands-on experience with Azure Databricks, Azure Data Lake Storage (ADLS), Azure Event Hubs, Apache Spark/Spark Streaming and Delta Lake. Strong understanding of lakehouse architectures and cloud-native data platforms. Practical experience with Terraform and Infrastructure as Code (IaC). Experience developing and managing CI/CD pipelines using Azure DevOps. Good understanding of cloud security, observability, reliability, scalability and operational best practices. Experience implementing data quality, governance and production monitoring practices. Bachelor’s or Master’s degree in Computer Science, Software Engineering, Data Engineering or a related discipline. Excellent communication and collaboration skills. Fluent in English, with the ability to work effectively in cross-functional teams. Proactive, ownership-driven and capable of solving complex technical problems. Strong commitment to automation, continuous improvement and engineering excellence. Immediate joiners with a valid work permit for the country of employment will be considered.

Будет плюсом

Experience with Rust; Exposure to Google Cloud Platform (GCP); Knowledge of Kubernetes, containers and containerisation technologies; Experience with data visualisation tools; Understanding of data governance, data management and data quality frameworks

Условия

Employment Type: Contract Work Mode: Onsite Location: Poland or Hungary