Data Engineer

We are seeking a skilled Data Engineer to join our dynamic team in Andhra Pradesh. The ideal candidate will have a strong background in data engineering, with expertise in AWS, Apache Spark, and Python. As a Data Engineer, you will be responsible for designing, building, and maintaining scalable data pipelines that support our data analytics and business intelligence initiatives. Key Responsibilities: - Develop and maintain robust data pipelines using Apache Spark and PySpark to process large datasets efficiently. - Utilize AWS services to manage and store data, ensuring high availability and security. - Collaborate with data scientists and analysts to understand data requirements and deliver solutions that meet their needs. - Implement data integration solutions using Apache Airflow for workflow management. - Work with messaging systems like Kafka and MQTT to facilitate real-time data processing and streaming. - Write efficient SQL queries to extract and manipulate data from various databases. - Monitor and optimize data pipeline performance, ensuring data quality and reliability. Required Skills: - Proficiency in AWS Data Engineering tools and services. - Strong experience with Apache Spark and PySpark. - Solid programming skills in Python. - Expertise in SQL for data manipulation and querying. - Familiarity with Apache Airflow for workflow orchestration. - Knowledge of Kafka and MQTT for data streaming. - Excellent problem-solving skills and attention to detail. If you are passionate about data engineering and eager to work in a collaborative environment, we encourage you to apply for this exciting opportunity.