Commercial Senior Data Engineer
Posted Updated 1
view
Summary
Senior data engineer who designs, builds, and owns end-to-end Azure-based data solutions — pipelines in Azure Data Factory, Databricks, Delta Lake and Snowflake using Python, SQL and PySpark. Handles production escalations, sets CI/CD and coding standards, mentors engineers, and supports AI/ML and Generative AI workloads.
Key Skills Required
- Strong hands-on experience with Azure Data Platform
- Azure Data Factory (ADF)
- Databricks
- Delta Lake
- Snowflake
- Python, SQL, PySpark
- Azure DevOps Jenkins
- CI/CD, DevOps & DataOps practices
- Infrastructure as Code and Git-based workflows
- Data pipeline design, development, testing and deployment
- Performance optimization, scalability, security and cost optimization
- Strong experience in production data platforms and architectural decision-making
- Knowledge of Azure Machine Learning, Azure OpenAI and Cognitive Services
- Understanding of data engineering for AI/ML and Generative AI workloads
- Experience with Agile/Scrum/Kanban environments
Key Responsibilities
- Design, develop, test and release end-to-end Azure-based data solutions.
- Own and manage complex data pipelines and data platforms.
- Design and implement data engineering solutions using Azure Data Factory, Databricks, Snowflake and related technologies.
- Translate business requirements into scalable, secure and maintainable data solutions.
- Handle complex incidents, defects and production escalations.
- Ensure data solutions meet security, governance, performance and reliability standards.
- Identify and resolve technical debt while driving continuous improvement.
- Establish data engineering standards, coding practices, CI/CD processes and monitoring.
- Mentor other engineers through technical guidance, design reviews and knowledge sharing.
- Support AI/ML and Generative AI use cases from a data engineering and Azure platform perspective.
Candidate Profile
We are looking for candidates with strong senior-level data engineering expertise, hands-on Azure experience, excellent problem-solving abilities and experience owning production-grade data solutions end to end.