Senior Data Platform Engineer
Posted 5
views
Summary
Senior engineer who designs, builds, and maintains scalable, secure, and cost-efficient data pipelines and lake/warehouse architectures on Microsoft Azure, using Databricks, Azure Data Factory, Terraform, and Python/PySpark. Works with stakeholders in Agile delivery, applying CI/CD and software best practices.
- Design, build, and maintain scalable, secure, and cost-efficiency data pipelines and data lake/warehouse architectures on Microsoft Azure.
- Support the adoption and implementation of modern data engineering tools and frameworks, including Databricks and Azure Data Factory.
- Monitor, optimize, and troubleshoot data platform performance, reliability, and operational efficiency.
- Collaborate with product delivery teams and business stakeholders to define data requirements and deliver reliable data solutions.
- Prepare high-level and detailed technical designs for data solutions.
- Apply best practices in software development, testing, deployment, and CI/CD automation.
- Participate in Agile delivery processes, including Scrum and Kanban ceremonies.
- Stay current with emerging data and analytics technologies and recommend continuous improvements.
Relevant Experience and Other Requirements:
- At least five years of experience in data engineering, platform engineering, or software engineering, including the architecture, design, and development of integrated systems.
- Strong analytical and problem-solving skills.
- Proven experience with Microsoft Azure cloud services, including CI/CD pipelines and Azure DevOps.
- Proven experience with Infrastructure as Code (IaC) tools such as Terraform.
- Proven experience developing microservices for data integration using TypeScript or Python.
- Hands-on experience developing data pipelines and modern data warehouse solutions using Databricks, Python, and PySpark.
- Practical experience with relational databases, particularly PostgreSQL.
- Experience ingesting data from SharePoint Online, Excel files, and other enterprise data sources.
- Knowledge of Power BI semantic models, DAX, and REST API integration is an advantage.
- Familiarity with software development lifecycle (SDLC), project management methodologies, and Agile frameworks such as Scrum or Kanban.
- Relevant certifications in Secure Coding, Scrum, DevOps, Azure, or Databricks are an advantage.
- Experience working in collaborative teams of five or more members.
- Ability to engage effectively with senior executives and stakeholders from diverse cultural backgrounds.
- Excellent oral and written communication skills in English.