Senior Data Engineer (AWS & Databricks) (1 Year Contract)

Rhino Partners is looking for an experienced Senior Data Engineer (AWS & Databricks) to join our team and contribute to the design, development and optimisation of enterprise-scale data platforms.

In this role, you will take technical ownership of building reliable, scalable and secure data pipelines and lakehouse solutions using AWS and Databricks. You will work closely with cross-functional teams to deliver high-quality data products that support business intelligence, advanced analytics and AI/ML initiatives.

This is a hands‑on engineering role with opportunities to drive technical architecture, establish engineering best practices and mentor fellow engineers.

Key Responsibilities

• Design, develop and maintain scalable batch and streaming data pipelines using Python, PySpark, SQL, AWS and Databricks.
• Architect and implement modern data lakehouse solutions using Delta Lake and Bronze, Silver and Gold data layers.
• Build robust data ingestion frameworks integrating databases, APIs, files and streaming sources, including incremental loading and Change Data Capture (CDC).
• Develop reusable data models, curated datasets and data products to support analytics, reporting and AI/ML workloads.
• Implement automated data quality checks, schema validation, reconciliation, metadata management and data lineage.
• Develop and manage workflow orchestration using Databricks Workflows, Lakeflow and AWS services.
• Implement CI/CD pipelines, infrastructure as code and automated testing to streamline data platform deployments.
• Monitor and optimise pipeline performance, reliability and cost efficiency across Spark, SQL, AWS and Databricks workloads.
• Establish monitoring, alerting, logging and incident recovery processes to ensure production stability.
• Apply cloud security and data governance best practices, including IAM, encryption, access controls and audit logging.
• Provide technical leadership through architecture reviews, code reviews, mentoring and engineering standards.
• Collaborate with data architects, analysts, data scientists and business stakeholders to translate requirements into scalable data engineering solutions.
• Maintain technical documentation, architecture designs and operational procedures to support knowledge transfer and long-term maintainability.

Requirements

• Degree in Computer Science, Software Engineering, Information Systems or a related discipline.
• 6–10 years of relevant experience in data engineering or software engineering, including hands‑on development and operation of production data pipelines.
• At least 2 years of experience working with cloud‑based data platforms.
• Strong proficiency in Python, PySpark and SQL, including complex transformations, distributed data processing and performance optimisation.
• Hands‑on experience with Databricks, Apache Spark and Delta Lake in production environments.
• Strong AWS data engineering experience, particularly with services such as S3, AWS Glue, IAM, Lambda, Step Functions, CloudWatch and Lake Formation.
• Experience designing and implementing data lakes, lakehouse architectures and dimensional or domain data models.
• Good understanding of incremental ingestion, CDC, data quality, schema evolution, error handling and pipeline recovery.
• Experience with workflow orchestration, Git, CI/CD pipelines and infrastructure as code tools such as Terraform.
• Familiarity with data governance, metadata management, lineage, access controls and cloud security best practices.
• Experience implementing production monitoring, alerting, troubleshooting and performance optimisation.
• Demonstrated ability to take ownership of technical solutions, review code and guide junior engineers.
• Strong analytical, problem‑solving and communication skills, with the ability to collaborate across technical and business teams.

Nice to Have

• Experience with Databricks Unity Catalog, Delta Live Tables, Lakeflow, Auto Loader, Structured Streaming or Delta Sharing.
• Familiarity with Apache Airflow, dbt, Kafka, AWS Kinesis, AWS DMS or similar technologies.
• Experience with data contracts, data products, semantic layers or data mesh architectures.
• Exposure to ML/AI data pipelines, feature engineering, feature stores or MLOps.
• Experience delivering projects within the Singapore public sector, including familiarity with IM8 security requirements and data governance standards.
• Relevant certifications such as Databricks Certified Data Engineer Professional, AWS Certified Data Engineer – Associate or AWS Certified Solutions Architect.

Why Join Rhino Partners?

At Rhino Partners, we connect talented technology professionals with opportunities to work on meaningful digital transformation initiatives. You'll collaborate with experienced engineering teams, work with modern cloud and data technologies, and contribute to solutions that deliver real business impact.

We value technical excellence, continuous learning, collaboration and ownership, providing an environment where engineers can grow their expertise while solving complex challenges.