Data Engineer
Summary
Designs and builds big-data pipelines in AWS/Azure using Python, Spark, SQL and NoSQL to ingest, process, store and analyze data for reporting and advanced analytics.
- Job Description - Data Engineer in Perth
o Minimum 7+ years of experience in design and development of bigdata systems entailing programming skills for external data ingestion, stream processing, storage optimisation, advanced analysis and reporting applications in at least one public cloud
o Knowledge of Platforms & tools: AWS/Azure, Python (Mandatory), Java/Scala, SQL, NoSQL, Spark.
o 3+ years’ experience in programming in Python and at least one other programming language.
o Exposure to Github, CI/CD, Docker and other modern development practices.
o Working with cloud-based infrastructure, especially AWS
o General architecture acumen: Database architecture, ETL, Data warehousing, SOA, Cloud, etc.