Spark Big Data Engineer | Scala/Java & ETL Analytics
Posted Updated
Description
Tata Consultancy Services Canada Inc. is seeking a Big Data Engineer to design and maintain robust data pipelines using Apache Spark, Scala, and Java for enterprise analytics.
You will ingest, transform, and optimize large datasets, focusing on performance and cost efficiency while collaborating with data scientists and infrastructure teams. You will also implement testing strategies, maintain documentation, and contribute to code quality through CI/CD practices.
Design and implement end-to-end data pipelines using Apache Spark for ETL and analytics workloads. Develop clean, efficient, and maintainable code using Scala and Java. Build solutions to ingest, transform, and aggregate large-scale financial and operational data. Analyze and optimize Spark applications for performance, memory efficiency, and cost-effectiveness. Collaborate with data scientists, analysts, and infrastructure teams to translate requirements into technical solutions. Troubleshoot and resolve production issues within big data systems. Implement testing strategies and maintain code quality standards. Create and maintain technical documentation and knowledge base articles. Demonstrated professional experience developing production-grade big data applications. Strong expertise in Apache Spark (Core, SQL, Streaming). Advanced programming skills in Scala and/or Java. Strong understanding of distributed computing principles and architecture. Experience with data warehousing and SQL. Knowledge of Git and CI/CD pipelines. Strong analytical and problem-solving skills. Excellent communication and teamwork skills.Skills
Sign in to read and join the discussion.