Senior Data Engineer
Summary
Design, build and deploy scalable data solutions on a mature Databricks lakehouse platform for a Queensland utilities organisation, using SQL, Python, and PySpark.
Salary: Contact Usama at Usama@blackroc.co
Senior Data EngineerDuration: 12-month initial contract with strong likelihood of extension
Location: Brisbane
Work Arrangements: 50% work from home
Company Overview
BLACKROC is partnering with a major Queensland utilities organisation to recruit a Senior Data Engineer.
Role Purpose
Join an established data team delivering on a modern Databricks lakehouse platform that's now three years mature and expanding fast. As demand for new data products, reporting solutions and analytics grows, you'll join the Data Delivery team to design, build and deploy scalable data solutions, working across a diverse range of enterprise systems and partnering directly with business stakeholders to turn requirements into trusted, reusable data assets.
Key Responsibilities
- Develop and maintain data ingestion, transformation and orchestration processes using Databricks and associated cloud technologies
- Deliver curated data assets and data products supporting reporting, analytics and business outcomes
- Partner with business stakeholders, project teams and platform teams to translate requirements into scalable solutions
- Contribute to solution design, data modelling and engineering standards
- Undertake unit testing and support quality assurance activities
- Conduct peer reviews and provide constructive feedback to maintain code quality
- Contribute to reusable frameworks, automation and engineering patterns
- Prepare and maintain technical documentation, and support transition of solutions into operational support
- Participate in agile planning, estimation and delivery activities
- At least 5 years' experience in a data engineering, data warehousing or analytics engineering role
- At least 3 years' hands-on experience developing and supporting data solutions using Databricks
- Strong SQL development, optimisation and troubleshooting skills
- Demonstrated experience developing solutions using Python and PySpark
- Experience building data ingestion, transformation and orchestration processes in cloud-based data platforms
- Experience designing and implementing data models supporting reporting, analytics and self-service consumption
- Experience with source control, CI/CD pipelines and automated deployment practices
- Strong problem-solving skills with the ability to work independently and manage competing priorities
- Ability to communicate technical concepts to both technical and non-technical stakeholders
- Experience with Databricks features such as Unity Catalog, Delta Live Tables, Workflows and Asset Bundles
- Experience working within AWS environments and associated cloud services
- Experience using Fivetran or similar SaaS data integration platforms
- Experience implementing data quality, monitoring and observability practices
- Experience within the electricity industry or another utility
- Experience in large or complex enterprise environments
- Experience mentoring or providing technical guidance to team members
At BLACKROC Recruitment, we are committed to fostering diversity, equity, and inclusion. We welcome applications from individuals of diverse backgrounds, including those with lived experiences. If you require any adjustments during the recruitment process, please don't hesitate to reach out to the above-mentioned BLACKROC Consultant.