Senior Data Engineer: Cloud Pipelines & AI Tools
Posted Updated
Description
Macquarie Group is seeking an experienced data/software engineer in Indonesia with 8+ years of hands-on experience and strong Python skills. You will build scalable ETL pipelines, implement data models, and optimize Spark/PySpark workloads in a cloud environment using AWS, Redshift, S3 and Glue.
The role emphasizes governance, security, and collaboration in an agile setting, with production support and observability responsibilities.
Develop and maintain scalable data pipelines and transformations. Collaborate with stakeholders to resolve data issues and risks. Implement governance, lineage and auditability across pipelines. 8+ years in data or software engineering with Python focus. Strong SQL using joins, CTEs, window functions. Experience building scalable ETL pipelines and data models. Spark or PySpark experience with performance tuning. AWS experience (Redshift, S3, Glue, IAM, Kinesis); familiarity with GCP. Airflow, Argo or dbt with scheduling, retries and backfills. Metadata, lineage, reconciliation, data protection and auditability. Docker, Unix/Linux; Kubernetes is a plus. Git, CI/CD, testing, observability and production support. Awareness of AI-assisted tools like GitHub Copilot; data privacy.Skills
Sign in to read and join the discussion.