Big Data Engineer

We are seeking a skilled Big Data Engineer to join our dynamic team. The ideal candidate will have extensive experience in managing and processing large datasets using various big data technologies. As a Big Data Engineer, you will be responsible for designing, implementing, and maintaining scalable data processing systems that support our data-driven decision-making processes. Key Responsibilities: - Develop and maintain robust data pipelines using technologies such as Apache Spark, Hive, and Flink. - Work with distributed storage systems like HDFS and manage data processing frameworks using YARN. - Collaborate with data scientists and analysts to understand data requirements and deliver high-quality data solutions. - Optimize data processing workflows for performance and efficiency. - Ensure data integrity and security throughout the data lifecycle. - Utilize containerization technologies such as Docker and orchestration tools like Kubernetes to deploy and manage applications in production environments. - Troubleshoot and resolve issues in production systems to ensure high availability and reliability. Required Skills: - Strong experience with big data technologies including Spark, Hive, and Flink. - Proficiency in working with HDFS and YARN. - Solid understanding of Linux operating systems. - Experience with containerization and orchestration tools such as Docker and Kubernetes. - Excellent problem-solving skills and the ability to work in a fast-paced environment. - Strong communication skills to collaborate effectively with cross-functional teams. If you are passionate about big data and eager to work in a challenging environment, we encourage you to apply for this exciting opportunity.