Mid-Level Data Engineer - up to RM 15k
Summary
Designs and maintains automated ETL/ELT pipelines in Python and Airflow, optimizes cloud data warehouses, and ensures data quality for analytics and product teams.
About the company
Randstad has partnered with a software solutions company, building innovative and reliable solutions for their clientele. Your future employers are building a tech team who are capable of keeping up with the latest technologies and aligned in building great solutions
Key Responsibilities
- Pipeline Development: Design, build, and maintain scalable, automated ETL/ELT data pipelines using Python and Apache Airflow.
- Data Warehousing: Manage and optimize data structures within our Cloud Data Warehouse (e.g., Snowflake, BigQuery, or Redshift) for maximum performance and cost efficiency.
- Data Quality: Implement automated testing, validation checks, and monitoring to ensure data accuracy and pipeline reliability.
- Collaboration: Partner closely with Data Analysts, Data Scientists, and product teams to understand their data needs and deliver clean, well-structured schemas.
- Code Quality: Write clean, documented, and production‑grade code, participating actively in team code reviews.
Requirements
- 4+ years of professional experience in a data engineering role.
- Strong proficiency in Python (writing clean, modular code, handling APIs, and manipulating data).
- Advanced SQL skills (writing complex queries, window functions, and optimizing performance).
- Hands‑on experience building, scheduling, and troubleshooting DAGs in Apache Airflow.
- Production experience working with a modern Cloud Data Warehouse (such as Snowflake, Google BigQuery, or AWS Redshift).
- Familiarity with version control (Git) and standard CI/CD practices.
Nice to Haves
- Experience with data transformation tools like dbt (data build tool).
- Exposure to Docker or Kubernetes.
- Basic understanding of cloud infrastructure (AWS, GCP, or Azure).