Data Engineer
Summary
Mid-Level Big Data Engineer with 4-6 years of experience building and supporting large-scale Big Data applications and cloud-based data pipelines using Python, AWS EMR, Spark, SQL, and AWS cloud services with focus on EMR performance tuning and cost optimization.
Role: Mid level Big Data Engineer
Position Type: Full-Time Contract (40hrs/week)
Contract Duration: 6+ months
Work Schedule: 8 hours/day (Mon-Fri)
Location: Hybrid 2x days onsite (Wed/Thur) - Hyderabad, India
We are seeking a Mid-Level Data Engineer with 4–6 years of experience to build and support large-scale Big Data applications and cloud-based data pipelines.
The ideal candidate will have strong hands-on experience with Python, AWS EMR, Spark, SQL, and AWS cloud services, with particular expertise in EMR performance tuning and cost optimization.
Key Responsibilities
- Build and maintain scalable data pipelines and Big Data applications.
- Develop data transformation processes using Python, Spark, SQL, Hive, and related technologies.
- Work extensively with AWS services including EMR, EC2, Lambda, S3, DynamoDB, and API Gateway.
- Optimize AWS EMR workloads for performance and cost efficiency.
- Build and maintain Apache Airflow workflows and DAGs.
- Troubleshoot production issues and improve data pipeline reliability.
- Participate in architecture reviews, code reviews, testing, and performance tuning.
- Work with cross-functional Agile teams to deliver scalable data solutions.
Required Skills
- 4–6 years of Data Engineering or Big Data experience.
- Strong Python development experience.
- Hands-on AWS EMR experience — required.
- Experience with EMR performance and cost optimization — required.
- Strong Spark / PySpark experience.
- SQL and data transformation experience.
- AWS experience with EMR, EC2, Lambda, and S3.
- Experience building large-scale batch data pipelines.
- Apache Airflow experience preferred.
- Strong troubleshooting and problem-solving skills.