Lead Big Data Engineer

We are seeking a highly skilled Lead Big Data Engineer to join our dynamic team in Karnataka or Tamil Nadu. The ideal candidate will have extensive experience in big data technologies and a strong background in data engineering. As a Lead Big Data Engineer, you will be responsible for designing, implementing, and maintaining scalable data processing systems that handle large volumes of data efficiently. You will work closely with data scientists, analysts, and other stakeholders to ensure that data solutions meet business needs and drive insights. Key Responsibilities: - Lead the design and development of big data solutions using Hadoop and related technologies. - Collaborate with cross-functional teams to gather requirements and translate them into technical specifications. - Develop and optimize data pipelines using PySpark and SQL to ensure high performance and reliability. - Implement data governance and security measures to protect sensitive information. - Mentor and guide junior engineers in best practices for big data engineering. - Troubleshoot and resolve issues related to data processing and system performance. - Stay updated with the latest trends and advancements in big data technologies and methodologies. Required Skills: - Strong expertise in Hadoop ecosystem and big data technologies. - Proficiency in PySpark and SQL for data manipulation and analysis. - Experience with Linux and Unix environments. - Knowledge of shell scripting for automation tasks. - Excellent problem-solving skills and attention to detail. - Strong communication and collaboration skills to work effectively in a team environment. If you are passionate about big data and looking for a challenging role where you can make a significant impact, we encourage you to apply.