Data Platform Engineer [up to RM 20k]
Summary
Senior data platform engineer (7+ years) designing and running scalable Azure/Databricks lakehouse pipelines for a Malaysian financial services firm, while liaising with business stakeholders and vendors and mentoring junior engineers. Core stack: Azure (Data Lake, Data Factory, Synapse), Databricks, PySpark, SQL, Python, Scala.
- Public Transport Accessible
- Career Growth
- Great Benefits
about the company
Randstad has recently partnered with an established financial services organisation in Malaysia, with a reputation for providing reliable, secure and user friendly experiences. Your future employers take initiatives within their digital transformation efforts, leveraging on modern technologies and solutions in creating better value.
key responsibilities:
Data Architecture & Engineering: Architect, build, and maintain highly scalable, fault-tolerant data pipelines and data lakehouse solutions utilizing Azure Cloud Services and Databricks.
Stakeholder Management: Act as the primary technical liaison between the data team and business units. Gather requirements, manage expectations, and translate business objectives into scalable data solutions.
Vendor Liaison: Manage relationships with third-party technology vendors and service providers. Evaluate new tools, oversee external deliverables, and ensure vendor alignment with internal architectural standards.
Presentations & Strategy Delivery: Conduct regular presentations for executive leadership and cross-functional teams to demonstrate data capabilities, project roadmaps, and strategic architectural decisions.
Data Processing & Optimization: Write complex, highly optimized queries and data transformations using PySpark, SQL, and Delta Lake within the Databricks environment.
Mentorship & Leadership: Provide technical guidance and best practices to junior engineers, establishing data governance and quality standards across the organization.
requirements
Experience: 7+ years of proven, hands‑on experience in data engineering, data architecture, or a closely related field.
Cloud Expertise: Deep, practical knowledge of the Microsoft Azure ecosystem (e.g., Azure Data Lake Storage, Azure Data Factory, Azure Synapse).
Databricks Mastery: Advanced proficiency in Databricks, including Apache Spark, Delta Lake architecture, and cluster performance tuning.
Communication Skills: Exceptional verbal and written communication skills, with a demonstrated track record of delivering presentations and managing complex stakeholder networks.
Vendor Management: Prior experience negotiating with, evaluating, and managing technology vendors or external consultants.
Programming Languages: Strong proficiency in Python, SQL, and Scala.
nice to haves
DevOps & CI/CD: Experience integrating data pipelines with CI/CD workflows, Infrastructure as Code (Terraform, ARM templates), and Azure DevOps.
Business Intelligence: Familiarity with Power BI, including data modeling, DAX, and integrating Power BI directly with Databricks and Azure data sources.
experience
7 years
skills
databricks, Azure, Azure Data Factory, PowerBI
qualifications
no additional qualifications required
education
Vocational/Professional Qualification
Skills
- ARM Templates
- Azure
- Azure Data Factory
- Azure DevOps
- Azure Synapse
- Business Intelligence
- CI/CD
- Cloud
- Data Engineering
- Data Governance
- Data Lake
- Data Modeling
- Data Pipelines
- Databricks
- Delta Lake
- DevOps
- Infrastructure as Code
- Lakehouse
- Power BI
- PySpark
- Python
- Scala
- Spark
- SQL
- Stakeholder Management
- Terraform
- Vendor Management
Sign in to read and join the discussion.