Cloud Data Platform Engineer: Airflow, GCP & Terraform
Posted Updated
Description
Confiz Limited LATAM seeks a Data Platform Engineer based in Mexico to design, develop, and maintain scalable data orchestration platforms using Python, GCP, Airflow, and Terraform. You will build self-service capabilities for engineers, data scientists, and analysts while integrating diverse data sources like Kafka, GCS, BigQuery, Postgres, and S3 for batch and real-time processing.
The role emphasizes IaC with Terraform, data governance and lineage, and CI/CD practices.
Design scalable data orchestration platforms for ETL/ELT operations. Build self-service platform capabilities for engineers, data scientists, and analysts. Integrate data sources and destinations (Kafka, GCS, BigQuery, Postgres, S3) for batch and real-time. Provision cloud data infra with Terraform via IaC pipelines. Build pipelines with Airflow and Kubernetes. Develop platform tooling and CLIs using Python. Implement metadata management, data cataloging, governance, and lineage. Design scalable, resilient solutions for large data volumes. Apply CI/CD and Git-based workflows to delivery. Evaluate emerging big data and AI-assisted dev tools. Create and maintain technical and user documentation. Bachelor’s or Master’s degree in Computer Science or Engineering. 5–7 years of professional experience in Data Engineering or related software roles. Strong Python programming for automation and tooling. Strong SQL proficiency in cloud data environments (BigQuery). Hands-on experience designing and maintaining ETL/ELT pipelines and data architectures. Experience with modern data transformation frameworks such as dbt. Strong hands-on experience with GCP data technologies (BigQuery, GCS, Dataproc Serverless). IaC experience with Terraform for provisioning cloud data infra. Hands-on experience building data pipelines with Airflow. Kubernetes experience in data platforms or cloud environments. Experience with Kafka / Confluent Cloud. Familiar with data governance, metadata management, cataloging, and lineage. CI/CD and Git-based workflows (GitHub Actions). Experience with AI-assisted developer tooling. Fortune 500 client exposure Dynamic data platform environment Competitive compensationSkills
Sign in to read and join the discussion.