Data Engineer

Job Title: Data Engineer

Company: Jnanasethu IT Labs Private Limited
Location: Bengaluru, Karnataka, India
Experience: 3-5 Years
Employment Type: Full-time

Job Description

We are looking for skilled Data Engineers with strong hands-on expertise in PySpark and SQL to support data engineering initiatives, KPI development, and enterprise reporting solutions.

The ideal candidates will have experience in data processing, transformation, and pipeline development. Familiarity with Palantir Foundry, particularly working with code repositories, is a significant advantage for roles supporting HCQ KPIs. For the Quality Dashboard initiative, strong SQL skills and experience working with Snowflake are essential.

The selected candidates will contribute to building reliable data solutions, developing analytical datasets, and supporting business intelligence and dashboarding requirements.

Key Responsibilities

• Develop, maintain, and optimize data processing workflows using PySpark and SQL.
• Build and manage data transformation pipelines to support business KPIs and analytical reporting.
• Work on HCQ KPI-related data engineering activities using PySpark.
• Utilize Palantir Foundry to develop and manage data workflows, where applicable.
• Work with code repositories to manage, maintain, and collaborate on data engineering projects.
• Develop and optimize SQL queries for data extraction, transformation, validation, and analysis.
• Work with Snowflake to query, process, and prepare datasets for the Quality Dashboard initiative.
• Support the development of a new Quality Dashboard to replace the existing dashboard.
• Ensure data accuracy, consistency, quality, and reliability across reporting workflows.
• Troubleshoot data processing issues and optimize query and pipeline performance.
• Collaborate with data engineers, analysts, BI developers, and business stakeholders to deliver effective data solutions.

Required Skills and Qualifications

1. PySpark and Data Engineering

• Hands-on experience with PySpark for data processing and transformation.
• Understanding of data pipelines, distributed processing, and data engineering concepts.
• Ability to troubleshoot and optimize data processing workflows.

2. SQL and Snowflake

• Strong proficiency in SQL, including complex queries, joins, aggregations, and data transformations.
• Experience querying and processing data using Snowflake.
• Understanding of data validation, query optimization, and performance tuning.
• Ability to prepare reliable datasets for dashboards and analytical reporting.

3. Palantir Foundry Preferred

• Familiarity with Palantir Foundry is a strong advantage, particularly for HCQ KPI-related work.
• Experience working with code repositories in data engineering environments.
• Understanding of data pipelines, transformation workflows, and collaborative development practices.

4. Data Quality and Reporting

• Understanding of data quality, validation, and consistency.
• Ability to transform raw data into structured datasets for business reporting.
• Experience supporting KPI calculations, dashboards, and analytical solutions.

5. Collaboration and Problem-Solving

• Strong analytical and troubleshooting skills.
• Ability to work collaboratively with technical and business teams.
• Good communication skills and attention to detail.

Preferred Skills

• Experience with Palantir Foundry and code repository management.
• Experience working with Snowflake in data engineering or analytics projects.
• Familiarity with Git or similar version control systems.
• Understanding of data warehousing, ETL/ELT processes, and business intelligence workflows.

Ideal Candidate Profile

• Strong hands-on expertise in PySpark and/or SQL, depending on the project assignment.
• Ability to develop efficient, reliable, and scalable data processing solutions.
• Experience supporting KPI calculations and dashboard data requirements.
• Strong problem-solving skills and the ability to troubleshoot data-related issues.
• Ability to work effectively in a collaborative development environment..

Join Jnanasethu IT Labs Private Limited and contribute to building reliable, scalable, and data-driven enterprise solutions.