Senior Data Engineer (Python, ETL)

Top 3 Reasons To Join Us

The Job

Job Summary:

For one of our clients, we are seeking an experienced Data Engineer to join our dynamic team. The ideal candidate will have a solid background in data engineering, with expertise in Python, ETL tools, Airflow, and Hadoop. You will be responsible for building, maintaining, and optimizing data pipelines that handle high-volume transactions exceeding 50 million per day.

If you're excited about solving challenging problems, delivering polished user experiences, and contributing to a fast-paced and innovative environment, we’d love to meet you.

Key Responsibilities:

• Design, develop, and maintain scalable data pipelines and data models for large transaction systems.
• Utilize Python and ETL tools to extract, transform, and load data efficiently from various sources.
• Implement and manage workflows using Apache Airflow to automate data processing tasks.
• Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and ensure data quality and availability.
• Optimize data processing performance and reliability in a Hadoop environment. ● Monitor and troubleshoot data pipeline issues to ensure operational efficiency and reliability.
• Document processes, data flow, and architecture for future reference and compliance. ● Stay current with industry trends and emerging technologies in data engineering.

Your Skills and Experience

MUST HAVE

• 5+ years of experience in data engineering role.
• Strong proficiency in Python for data processing and manipulation.
• Hands-on experience with ETL tools (e.g., Talend, Informatica, Apache NiFi).
• In-depth knowledge of Apache Airflow for workflow management.
• 3+ years of hands-on experience with Spark
• Strong experience working with Kafka/Flink
• Experience with big data technologies such as Hadoop and its ecosystem (e.g., HDFS,
• Proven experience in building and maintaining big transaction systems (handling over 50 million transactions per day).
• Familiarity with cloud platforms (e.g., AWS, Azure, GCP) is a plus.
• Excellent analytical and problem-solving skills.
• Strong communication skills and the ability to work collaboratively in a team environment.

Why You'll Love Working Here

• A supportive and flexible working environment
• Macbook provided
• Premium health insurance
• SHUI contributions which comply with local laws
• 13th month salary
• 15 paid annual leaves per year
• Work from home 3 days/month
• Company trip, happy hour, snack and coffee