PySpark Data Engineer - ETL/ELT Pipelines

We are looking for an experienced PySpark Data Engineer with 58 years of hands-on experience in building scalable data pipelines and modern data platforms.

The ideal candidate should have strong expertise in PySpark, Snowflake, AWS, and core Data Engineering concepts. Experience in Java is a strong advantage.

The candidate will work closely with cross-functional teams to design, develop, and optimize high-performance data solutions that support business analytics and data-driven decision-making.

Key Responsibilities :

- Design, develop, and maintain scalable ETL/ELT pipelines using PySpark.

- Build and optimize data ingestion, transformation, and processing workflows.

- Develop and manage data solutions using Snowflake as the cloud data warehouse.

- Leverage AWS services to build secure, scalable, and reliable data platforms.

- Implement data quality, validation, and monitoring processes.

- Optimize data pipelines for performance, scalability, and cost efficiency.

- Collaborate with business analysts, data architects, and application teams to understand data requirements.

- Troubleshoot production issues and ensure timely resolution.

- Follow best practices for coding, testing, documentation, and deployment.

- Participate in code reviews and contribute to continuous process improvements.

Required Skills :

- 5 - 8 years of experience in Data Engineering.

- Strong hands-on experience with PySpark.

- Good experience with Snowflake (data modeling, SQL optimization, loading/unloading data, performance tuning).

- Strong knowledge of AWS services such as S3, Glue, EMR, Lambda, IAM, Redshift, or related services.

- Excellent SQL skills and experience with relational databases.

- Strong understanding of ETL/ELT frameworks and data warehousing concepts.

- Experience with workflow orchestration tools such as Apache Airflow or similar.

- Familiarity with version control systems like Git.

- Good understanding of CI/CD practices and deployment methodologies.

- Strong analytical, problem-solving, and debugging skills.

Preferred Skills :

- Hands-on experience with Java (strongly preferred).

- Experience with Spark performance tuning and optimization.

- Knowledge of Delta Lake, Iceberg, or other modern data lake technologies.

- Experience working in Agile/Scrum environments.

- Exposure to DevOps practices and Infrastructure as Code is a plus.