Data Engineer

We are looking for a skilled Data Engineer with hands-on experience in Python, PySpark, SQL, and Unix. The candidate will be responsible for developing and maintaining data pipelines, processing large datasets, and ensuring data quality and performance.

Key Responsibilities

• Develop and maintain scalable data pipelines using Python and PySpark.
• Write and optimize SQL queries for data extraction, transformation, and processing.
• Perform data cleansing, validation, and transformation to ensure data quality.
• Use Unix/Linux commands and shell scripting for data processing and job monitoring.
• Troubleshoot data pipeline failures and resolve performance issues.
• Collaborate with cross-functional teams to support data integration and reporting requirements.

Required Skills

• Strong programming experience in Python.
• Hands-on experience with PySpark and large-scale data processing.
• Proficiency in SQL and query optimization.
• Good knowledge of Unix/Linux commands and shell scripting.
• Experience in ETL/ELT processes and data pipeline development.
• Strong analytical, debugging, and problem-solving skills.