Skip to content
Search jobs inside your AI agent. Claude, ChatGPT, Claude Code, Codex, Gemini, Cursor and Grok
Connect

Unison Group

Senior Data Engineer (Databricks and Pyspark)

Posted Updated

Description

Unison Group in Malaysia and Singapore seeks a Senior Data Engineer skilled in Databricks, PySpark, Delta Lake and cloud data pipelines. The role combines leadership with hands-on architecture experience to design scalable data platforms.

Fluency in Mandarin is required. This full-time role offers a one-month bonus and opportunities to drive enterprise data initiatives across multi-cloud environments.

Design, build, and optimize data ingestion and transformation pipelines using Databricks, PySpark, and Python. Implement Delta Lake and Medallion architecture for scalable enterprise data platforms. Develop ingestion frameworks for data from SFTP, REST APIs, SharePoint/Graph API, AWS, and Azure sources. Automate workflows using Databricks Workflows, ADF, Azure Functions, and CI/CD pipelines. Optimize Spark jobs for performance, reliability, and cost efficiency. Implement data validation, quality checks, and monitoring with automated alerts and retries. Design secure and governed datasets using Unity Catalog and cloud security best practices. Collaborate with analysts, business users, and cross-functional teams to deliver curated datasets for reporting and analytics. Provide technical leadership and guidance to junior team members. 5–8+ years of experience in Data Engineering. Strong hands-on experience with Databricks, PySpark, Delta Lake, SQL, Python. Experience with Azure Data Lake, ADF, Azure Functions, or AWS equivalents (S3, Lambda). Experience integrating data from APIs, SFTP servers, vendor data providers, and cloud storage. Knowledge of ETL/ELT concepts, Lakehouse/Medallion architecture, and distributed processing. Strong experience with Git, Azure DevOps CI/CD, and YAML pipelines. Ability to optimize Spark workloads (partitioning, caching, Z-ordering, performance tuning). Bonus (1 month)

Skills

What Senior Data Engineering jobs ask for — and how much of it you have →
Apply

See also

Data Engineering jobs by country — openings, pay and top skills →