TEST ENGINEER L3

Job Description

Job Summary

We are looking for an experienced Big Data QA Engineer with strong expertise in validating large-scale data pipelines, ETL processes, and data platforms. The ideal candidate should possess hands-on experience in Big Data testing, data quality validation, SQL, PySpark, Hadoop ecosystem, and automation testing. The role involves ensuring the accuracy, completeness, and reliability of data across enterprise-scale data platforms.

Mandatory Skills

• Big Data Testing
• SQL
• ETL Testing
• PySpark
• Hadoop Ecosystem
• Hive
• HDFS
• Data Validation & Data Quality Testing
• Unix/Linux
• Test Planning & Test Case Design
• Defect Tracking (JIRA)
• Git

Good to Have

• Scala
• Databricks
• Kafka
• Airflow
• Spark Streaming
• Snowflake
• Python Automation
• Selenium
• Cloud Platforms (AWS/Azure/GCP)
• Performance Testing
• GenAI-assisted Testing Tools

Key Responsibilities

• Validate ETL pipelines and large-scale data processing workflows.
• Perform source-to-target reconciliation and data integrity testing.
• Create and execute test cases for data migration and transformation requirements.
• Validate data quality, completeness, and accuracy across multiple systems.
• Execute SQL and PySpark-based validation scripts.
• Test batch and real-time data processing platforms.
• Work closely with data engineers, business analysts, and developers.
• Automate repetitive data validation and regression testing processes.
• Identify, track, and resolve defects throughout the testing lifecycle.

Required Qualifications

• Bachelor's degree in Computer Science, IT, Engineering, or related field.
• 4+ years of experience in Big Data/Data Warehouse Testing.
• Strong expertise in SQL, ETL Testing, and Data Validation.
• Hands-on experience with Hadoop, Hive, Spark, or Databricks.
• Experience working with large datasets and complex data transformations.
• Strong analytical and problem-solving skills.

Preferred Experience

• Data Lake and Data Warehouse testing.
• Real-time data validation using Kafka/Spark Streaming.
• Cloud data platform testing on AWS, Azure, or GCP.
• Automation framework development for data validation.

K͏ey responsibilities

• Validate ETL pipelines and large-scale data processing workflows.
• Perform source-to-target reconciliation and data integrity testing.
• Create and execute test cases for data migration and transformation requirements.
• Validate data quality, completeness, and accuracy across multiple systems.
• Execute SQL and PySpark-based validation scripts.
• Test batch and real-time data processing platforms.
• Work closely with data engineers, business analysts, and developers.
• Automate repetitive data validation and regression testing processes.
• Identify, track, and resolve defects throughout the testing lifecycle.

͏

͏

͏