Urgent Opening for Big Data Engineer for Bangalore/Chennai
Summary
Big Data Engineer at HCLTech (Bangalore/Chennai) building and optimizing large-scale data pipelines and ETL/ELT workflows using Python, PySpark, SQL, Apache Airflow, Kubernetes, S3-compatible object storage, and table formats like Iceberg/Delta Lake/Hudi. Targets candidates with 6-10 years of experience in Agile teams.
About the Company
Hi, We have an urgent opening for Big Data Engineer for Bangalore/Chennai location.
Location- Bangalore/Chennai
About the Role
Experience- 6 to 10 Years
Responsibilities
Mandatory Skills
Core Big Data Platform Skills
- Strong programming experience in Python
- Hands-on expertise with Apache Spark (PySpark preferred)
- Strong SQL development and query optimization skills
- Apache Airflow for workflow orchestration and scheduling
- Kubernetes (AKS/SKE) for container orchestration and deployment. AKS/EKS/GKE also fine.
- MinIO / S3 Compatible Object Storage like AWS S3 or other S3-compatible object storage experience
- Apache Iceberg/Delta Lake / Apache Hudi
Technical Competencies
- Data Pipeline Development
- ETL/ELT Processing
- Data Modelling
- Performance Tuning
- Version Control (Git)
- Agile/Scrum Delivery Model