Data Engineer

We are seeking a skilled Data Engineer to join our team in Andhra Pradesh. The ideal candidate will have a strong background in data engineering and be proficient in various technologies and methodologies related to data processing and management. As a Data Engineer, you will be responsible for designing, building, and maintaining scalable data pipelines that support our data lakes and warehousing solutions. Key Responsibilities: - Develop and maintain robust data pipelines using Python, Apache Spark, and PySpark to ensure efficient data processing and transformation. - Collaborate with data scientists and analysts to understand data requirements and deliver high-quality data solutions. - Implement and manage workflows using Apache Airflow to automate data processing tasks and ensure timely data availability. - Design and optimize data lakes and data warehousing solutions to support analytics and reporting needs. - Monitor and troubleshoot data pipeline performance, ensuring data integrity and reliability. - Stay updated with the latest industry trends and technologies in data engineering and apply best practices to improve our data infrastructure. Required Skills: - Proficiency in Python programming and experience with data processing libraries. - Strong knowledge of Apache Spark and PySpark for big data processing. - Experience with data lakes and data warehousing concepts and technologies. - Familiarity with Apache Airflow for workflow management. - Ability to work collaboratively in a team environment and communicate effectively with stakeholders. If you are passionate about data engineering and eager to work in a dynamic environment, we encourage you to apply for this exciting opportunity.