Aws Data Engineer
Summary
Builds and maintains ETL/ELT data pipelines in Databricks on AWS, integrating services like S3, Glue, Lambda, and Redshift; orchestrates workflows (Databricks Workflows/Airflow), models data lakes/warehouses, and tunes performance. Core stack: Databricks, Apache Spark, Python/Scala, advanced SQL; an associate-level AWS/Databricks certification is mandatory.
TCS Hiring | AWS Data Engineer | Bangalore/Noida/Gandhi Nagar/Pune/Chennai
Greetings from Tata Consultancy Services (TCS)!
AWS Data Engg (Databricks):
Exp: 4-8 yrs
Key responsibilities for this role include designing, developing, and maintaining ETL/ELT data pipelines in Databricks using Spark, and integrating Databricks with various AWS services like S3, Glue, Lambda, and Redshift. They contribute to data lake and data warehouse architecture and modeling, orchestrate workflows with tools like Databricks Workflows or Airflow, and optimize performance and troubleshoot issues. Collaboration with stakeholders to define data requirements, implement quality checks, and ensure data governance is also crucial.
Required Skills & Qualifications
Required skills include proven experience in data engineering with Databricks and AWS data services, proficiency in Python or Scala and advanced SQL, and expertise in Apache Spark. In-depth knowledge of core AWS data services and a solid understanding of data warehousing principles, ETL/ELT, and data modeling are essential. Familiarity with DevOps, Git, and orchestration tools is also needed. A bachelor's degree is typical, and relevant certifications like AWS Certified Data Engineer or Databricks Certified Data Engineer are beneficial.
Note: Associate level AWS Databrick certificate is mandatory. /AWS Data Engineer