Databricks Data Engineer

You will drive the implementation and optimization of a scalable Azure cloud-based data platform, enhancing enterprise data ingestion, transformation, and storage. By automating ETL/ELT workflows and enforcing data quality checks, your efforts will directly improve data reliability and reduce manual intervention. Your expertise in near real-time analytics will empower strategic decision-making and business intelligence across the enterprise.

Responsibilities

• Optimize, operate at scale, and enhance Microsoft Azure and Databricks cloud-based data platforms.
• Participate in the full project lifecycle, including design, development, testing, deployment, and documentation alongside external vendors.
• Collaborate with Data Analysts to prepare, cleanse, and build optimized data models and Power BI visualizations.
• Perform proofs of concept for emerging cloud data products.
• Validate data workflows, integration points, pipeline schemas, and performance benchmarks through rigorous testing.
• Create operational playbooks, automation scripts, and technical architecture documentation to streamline deployment and long-term maintainability.

Requirements

• Bachelor’s degree in Computer Science, Information Technology, Data Science, or a related field.
• At least 3 years’ professional experience focusing on data platform implementations or cloud data migrations.
• Technical proficiency in Python, SQL, Spark, Azure Data Factory, Databricks, Unity Catalog, and Power BI.
• Hands-on experience with Event Hub, Kafka, or event messaging systems, alongside Azure Data Lake Storage Gen2.
• Clear understanding of the software development lifecycle (SDLC), data governance principles, data quality, and security.
• Professional certification in a Cloud Data platform or Databricks Certified Data Engineer credential.

Kowloon City, Kowloon City District, HK