Senior Data Engineer

Develocraft is looking for a Senior Data Engineer for our client operating in the financial services sector.

Salary to be agreed. We offer a flexible budget based on experience and expertise.

Start ASAP.

Requirements

Must have:

Technical Knowledge

• Hands-on experience in designing, building, and maintaining data platforms, particularly Microsoft Fabric or similar solutions.
• Experience in Data Engineering, including data pipeline design, data transformation, and orchestration.
• Strong understanding of Lakehouse architecture, Data Warehousing, and data modeling.
• Proficiency in Python, Spark, SQL, and data analytics.
• Experience integrating systems through APIs, file-based interfaces, and event-driven architectures.
• Hands-on experience with Azure DevOps, including repositories, work item management, and deployment pipelines.
• Ability to manage services throughout their entire lifecycle, from design and implementation to maintenance and continuous improvement.
• Strong troubleshooting skills across data platforms, pipelines and integrations.
• Ability to produce clear and well-structured technical documentation.

Security and Collaboration

• Understanding of security best practices, access control, and data governance.
• Self-driven with the ability to collaborate effectively with both business and technical teams.
• Strong communication skills, with the ability to explain technical concepts to non-technical stakeholders.
• Ability to translate business requirements into secure, scalable and maintainable technical solutions.
• Results-oriented mindset with a focus on delivering high-quality, secure, and maintainable solutions.

Experience

• Experience delivering enterprise data platforms and data engineering solutions.
• Experience supporting and maintaining data services in production.
• Experience supporting and maintaining production AI services.
• Experience working in regulated industries, particularly financial services, is advantageous.

Responsibilities

• Design, build, maintain, and enhance the Microsoft Fabric data platform, ensuring its availability, performance, scalability, and reliability.
• Develop and maintain data pipelines for data ingestion, transformation, and delivery.
• Design and manage Lakehouse and Data Warehouse solutions.
• Monitor platform health, troubleshoot incidents, and ensure operational continuity.

Data Engineering and Data Modeling:

• Design, develop, and maintain data models for reporting, analytics, and AI workloads.
• Build optimized datasets and semantic models.
• Ensure data quality through validation, error handling, and adherence to data modeling best practices.
• Design, implement, and maintain AI solutions using Microsoft AI and Azure AI services.
• Develop Generative AI solutions, Retrieval-Augmented Generation (RAG) applications, intelligent search capabilities, and AI agents.
• Prepare and manage data for AI models, including prompt engineering, model lifecycle management, and deployment.
• Collaborate with business stakeholders to identify AI use cases and continuously monitor the quality and effectiveness of AI solutions.

Platform Integration and Data Access:

• Integrate source systems with the Microsoft Fabric platform.
• Design APIs and data access services for reporting, business applications, and AI solutions.
• Integrate AI services with enterprise applications while ensuring compliance with security and architectural standards.
• Leverage Azure DevOps for source code management, work item tracking, and deployment automation.
• Implement and maintain CI/CD and MLOps practices.
• Create and maintain clear, comprehensive technical documentation.

Security and Governance:

• Manage access control, auditing, and data security across the platform.
• Ensure compliance with regulatory requirements and Responsible AI principles, including AI model governance, data lineage, monitoring, and risk management.

Collaboration and Project Delivery:

• Translate business requirements into scalable data and AI solutions using Microsoft Fabric, Azure, and AI technologies.
• Support the onboarding of new data sources, platform automation, and continuous platform enhancement.
• Manage and prioritize the engineering backlog using Azure DevOps.
• Act as a technical expert, providing guidance and resolving complex issues related to data platforms and AI solutions.

Develocraft sp. z o.o. is an employment agency registered in a registry kept by Marshal of the Pomorskie Voivodeship under the number 18819.