Data Engineer
Summary
Lead Data Engineer at TCS (Chennai or Pune) building and productionizing big-data ingestion and transformation pipelines on Hadoop using Spark and Scala to support data science and analytics use cases, while leading a squad of 3-5 engineers. Requires 6-8 years of experience, with Scala mandatory.
JOB DESCRIPTION:
Role:Spark,Scala – Data Engineer ;Location: Chennai, Pune ; YOE: 6-8 years
- Big Data/Hadoop Experience particularly in ingesting data and implementing Data ingestion pipelines, SQOOP, HADOOP, HDFS, HIVE, IMPALA, Java, Scala, Spark
- Scala is Mandatory
- Data Engineer with below responsibilities:
- Lead Data Engineer to build data pipelines to support implementation of data science and analytics use cases.
- Candidate needs to be able to develop code in the corresponding language (see technical skills), test it, and follow up with the implementation into Production environment.
- Candidate will also take part in the solution design phase, so experience in analysis requirements is desirable.
- Technical expert to lead a squad of engineers for a Data Product and implement data transformation projects in Hadoop.
- Candidate will lead a team of 3-5 data engineers with minimum supervision and integrate with wider IT and business project teams.
- Good exposure to Unix and HDFS commands
- Experience on Pyspark will be an added advantage
- Experience working on Data Analytics will be an added advantage
- Experience on SAS will be an added advantage