Data Engineer

The Job

Data Engineer with 4+ years of experience in building scalable data pipelines, developing data assets, and delivering cloud-based data engineering solutions using modern data platforms and technologies.

Key Responsibilities

• Develop and maintain scalable data pipelines using PySpark, Databricks, and AWS.
• Optimize Spark SQL/PySpark code for performance and scalability.
• Execute data migration and modernization initiatives, including Redshift migrations.
• Refactor legacy codebases to improve maintainability and efficiency.
• Develop unit tests and support TDD practices.
• Troubleshoot complex data and performance issues.
• Manage code through Git, JFrog Artifactory, and CI/CD best practices.

Your Profile

• 4-5 years of Data Engineering experience.
• Strong hands-on experience with Databricks (AWS), PySpark, Apache Spark, and SQL.
• Experience with AWS services such as S3, Redshift, EC2, Lambda, CloudFormation, and Athena.
• Proven experience in data migration projects and large-scale data processing.
• Strong expertise in performance tuning, code optimization, and data validation.
• Experience with source-to-target reconciliation and root cause analysis.
• Understanding of SDLC, version control, and Agile methodologies.

Good to Have

• Power BI or reporting tools exposure.
• Knowledge of data compliance, governance, and audit requirements.
• Experience working in Agile/Scrum environments.