EY - Scala Data Engineer
Description
Role Overview :
We are seeking a seasoned Scala Data Engineer to join our high-performance engineering team in Pune. In this role, you will architect and maintain robust data pipelines that process massive datasets, ensuring high availability and performance for our global client base.
You will collaborate closely with data scientists, product managers, and infrastructure teams to transform raw data into actionable business intelligence. By building scalable, fault-tolerant systems, you will directly influence our data-driven decision-making capabilities and enhance the overall efficiency of our product ecosystem.
Key Responsibilities :
- Design and implement complex data processing frameworks using Scala and Spark to support large-scale analytical workloads for our enterprise clients.
- Optimize existing Hadoop-based data architectures to improve throughput and reduce latency, ensuring seamless data accessibility across the organization.
- Collaborate with cross-functional teams to translate business requirements into technical specifications, driving the development of high-impact data products.
- Mentor junior engineers by conducting code reviews and promoting best practices in functional programming and distributed systems design.
- Monitor and troubleshoot production data pipelines to maintain data integrity and system reliability, minimizing downtime for end-users.
Required Skillset :
- Demonstrated expertise in building distributed data applications using Scala and Apache Spark, with a deep understanding of functional programming paradigms.
- Proven ability to manage and optimize large-scale data storage and processing environments within the Hadoop ecosystem.
- Strong analytical mindset with the ability to communicate complex technical concepts to non-technical stakeholders effectively.
- Exceptional problem-solving skills and the ability to thrive in a fast-paced, collaborative environment, with a preference for candidates comfortable working in a hybrid setup in Pune.
- A Bachelor's or Master's degree in Computer Science, Information Technology, or a related quantitative field, backed by 5 to 8 years of relevant professional experience in data engineering.