Lead Data Engineer
Posted Updated
Who recruits here Pro
Up to three recruiters at Jobgether with their work email, so you can write to a person.
Part of Pro. How it works
Description
Jobgether is seeking a hands-on Lead Data Engineer based in Brazil to design, build, and maintain scalable data pipelines on AWS. The role combines technical delivery with team leadership in an Agile setting.
You will implement workflows with MWAA/Airflow, develop data solutions with Python, Pandas, and PySpark, and drive data quality and CI/CD practices across the engineering team.
Design, develop, and maintain scalable data pipelines on AWS infrastructure. Build reliable data processing solutions using Python, Pandas, and PySpark. Configure and manage data workflows using MWAA or Apache Airflow. Develop and optimize queries with AWS Athena to support data analysis. Automate infrastructure provisioning and deployment using Terraform and GitHub Actions. Implement testing, monitoring, troubleshooting, and data-quality controls. Participate in code reviews and promote engineering best practices. Collaborate with engineering, product, and business stakeholders in an Agile environment. Provide technical guidance and leadership to the delivery team. At least 5 years of relevant professional experience. At least 1 year of experience leading and managing development teams. Strong Python programming skills applied to data engineering. Practical experience using Pandas for data manipulation and analysis. Advanced working knowledge of PySpark for large-scale distributed data processing. Experience with AWS MWAA or Apache Airflow for workflow orchestration. Familiarity with AWS Athena for querying and analyzing data stored in AWS. Hands-on experience with Terraform for infrastructure automation and management. Working knowledge of GitHub Actions and automated CI/CD pipelines. Good understanding of DevOps principles and their application to data engineering. Upper-intermediate English proficiency (B1 minimum). Healthcare benefits Paid time off Professional certification programs Global career developmentSkills
Sign in to read and join the discussion.