Data Engineer
Posted Updated 1
view
Summary
Builds and optimizes big-data ETL pipelines using PySpark, Hive, and Spark SQL for Tata Consultancy Services, with senior roles also leading offshore teams and architecting big data solutions. Requires 5-10 years of experience with big data tooling, data warehousing, and Cloudera environments, onsite in major Indian cities.
Job Title- Pyspark (Data Engineer)
Location- Chennai / Kolkata / Hyderabad / Pune/Bangalore
Experience Required- 5-10 years.
Job Description:
We are looking for a skilled and experienced professional to join our team. The ideal candidate should have 5-10 year of relevant experience and to be comfortable working in dynamic, fast-paced environment.
Key Responsibilities:
- Lead the team from offshore by owning project deliverables & provide technical oversight (For Tech Lead)
- Experienced in architecting & Developing Big Data solutions (For Tech Lead)
- Strong understanding of Big Data tools and Apache projects (For Tech Lead, Sr. Developer)
- Good understanding on performance tuning on Big data technologies (For Tech Lead, Sr. Developer)
- Good to have Python and Java background (For Tech Lead, Sr. Developer)
- Experience in relational database and data warehousing concept
- Well versed in Hive QL, Spark QL
- Create ETL transformation using Hive, Spark
- Support development team and testing team in implementing and testing the ETL respectively
- Build data quality rule and orchestrate using in house framework
- Work with Cloudera/Big Data support for root cause analysis and resolutions
- Good team player and collaborate with other team