Solution Designer / Lead Data Engineer
Summary
Design and lead the build of scalable, event-driven data pipelines on GCP for batch and streaming workloads, mentoring junior engineers.
Solution Designer / Lead Data Engineer at NCS Australia
About the role We are looking for a skilled Solution Designer and Lead Data Engineer to spearhead a vital transformation project aimed at enhancing and modernizing our Data Ingestion framework on Google Cloud Platform (GCP). This position requires a blend of strategic architectural design and extensive data engineering knowledge. You will be responsible for creating scalable, secure, and event-driven data pipeline architectures capable of handling large volumes of both batch and streaming data.
Key facts
Location: Sydney, Sydney Region Engagement: Full-time
What you'll do
- Lead the design and implementation of data ingestion frameworks on GCP.
- Develop and maintain scalable data pipeline architectures to support high-volume data processing.
- Collaborate with cross-functional teams to ensure alignment with business needs and technical requirements.
- Provide guidance and mentorship to junior engineers on best practices in data engineering.
Requirements
- Minimum of 5 years of experience in data engineering or a related field.
- Proficiency in Google Cloud Platform and its data services, including BigQuery, Dataflow, and Pub/Sub.
- Strong background in designing and implementing data pipeline architectures.
- Experience with programming languages such as Python, Java, or Scala.
- Solid understanding of data modeling, ETL processes, and data warehousing concepts.
- Excellent problem-solving skills and ability to work in a fast-paced environment.
Nice to have
- Familiarity with containerization technologies like Docker and Kubernetes.
- Experience with data visualization tools such as Tableau or Looker.
- Knowledge of machine learning concepts and frameworks.
Skills & tools
Google Cloud Platform (GCP) - BigQuery - Dataflow - Pub/Sub - Python, Java, Scala - ETL tools and data modeling techniques
Practical notes
- Applicants must be eligible to work in Australia.
- Competitive salary and benefits package offered.
- Interested candidates should submit their applications by (insert application deadline).