Senior Data Engineer
Summary
Senior Data Engineer based in Sydney with 8+ years' experience building data platforms on Azure and cloud data warehouses like Databricks. Day to day involves hands-on SQL and Python, data profiling, pipeline orchestration with Airflow/Control-M, and big data tooling such as Spark, Hive and Hadoop storage.
Minimum Years of Experience: 8Years
Must Have Skills:
· Hands On with advanced SQL, Python etc.
· Hands On in data profiling
· Hands On in working on Cloud like Azure and Cloud DW like Data bricks
· Hands On experience on Scheduling tools like Airflow, Control M etc.
· Knowledgeable on Big Data tools like Spark (Python/Scala), Hive, Impala,
Hue and storage (e.g. HDFS, HBase)
· Knowledgeable in CICD processes – Bit Bucket/GitHub, Jenkins, Nexus etc.
· Knowledgeable managing structured and unstructured data types.
Nice to Have Skills:
· Experience in working with a team of data engineers working from
different locations in different time zones.
· Excellent communication.
· Effective prioritization.
· Pragmatic stakeholder management.Cloud
Platform Familiarity: Understanding of cloud services (AWS, Azure, GCP) to
analyze and optimize cloud-based systems.
· Basic Scripting Knowledge: Ability to write simple scripts in Python,
SQL, or R for data extraction, transformation, and analysis.