IT Software Quality Engineer (ETL / Big Data)

IT Software Quality Engineer (ETL / Big Data)

We are working with a leading company seeking an experienced IT Software Quality Engineer to drive quality assurance across complex data platforms and cloud-based ecosystems. The role requires strong expertise in ETL and Big Data testing, data validation, SQL, and test automation, with a focus on ensuring data accuracy, integrity, and reliability across end-to-end data pipelines.

Key Responsibilities

ETL & Data Validation

• Perform end-to-end testing of ETL and data workflows
• Validate data extraction, transformation, and loading processes
• Conduct data reconciliation and cross-system validation
• Ensure data quality, completeness, consistency, and accuracy across source and target systems

Test Planning & Execution

• Analyze business requirements and create detailed test scenarios and test cases
• Develop comprehensive test strategies aligned with business and technical requirements
• Execute functional, integration, system, and regression testing
• Ensure adequate test coverage and quality across releases

SQL & Data Analysis

• Perform advanced SQL-based data validation and analysis
• Validate complex data transformations, aggregations, joins, and business rules
• Investigate and troubleshoot data-related defects and inconsistencies

Automation & Tools (good to have)

• Develop and maintain automated data validation frameworks using Python, PySpark, and Pandas
• Create reusable automation utilities to improve testing efficiency
• Support continuous improvement of test automation capabilities

Cloud & Data Platform Testing

• Validate data pipelines built on Azure Data Factory (ADF)
• Test data processing and storage solutions on Azure Databricks and Azure Data Lake
• Ensure seamless integration between cloud-based data services

API & Integration Testing

• Validate data exchange across integrated applications and services
• Perform API testing using tools such as Postman

Defect Management

• Manage the complete defect lifecycle, from identification through closure
• Collaborate closely with development and integration teams to resolve issues effectively

Reporting & Stakeholder Communication

• Track and report testing progress, quality metrics, risks, dependencies, and blockers
• Provide regular status updates to project stakeholders
• Work closely with developers, business analysts, product owners, and business users

Quality & Risk Management

• Identify quality risks proactively and drive mitigation plans
• Ensure compliance with organizational QA standards, processes, and governance practices

Required Skills & Experience

Must-have: ETL Testing / Big Data Testing, Manual Testing, BI/Reporting Validation, Advanced SQL, Data Validation and Reconciliation, End-to-End Test Lifecycle Management, Azure DevOps, ETL Frameworks and Processes Hands-on experience: Python, PySpark, Pandas, Azure Data Factory (ADF), Azure Databricks, Azure Data Lake Gen2, Postman

Educational Qualification

Bachelor's or Master's Degree in Computer Science, Information Technology, Data Engineering, or a related technical discipline

Key Competencies

Strong analytical and problem-solving skills

Attention to detail and quality focus

Ownership and accountability

Effective stakeholder management

Excellent verbal and written communication skills

Proactive risk identification and mitigation

Preferred Candidate Profile

Strong experience in data-centric testing environments

Exposure to cloud-based data platforms and modern data architectures

Ability to independently lead testing activities and coordinate with stakeholders

Interest in emerging AI-driven testing approaches

#QualityEngineer #ETLTesting #BigData #AzureJobs #SQLJobs #DataValidation #HiringNow #TestAutomation #PySpark #Databricks #TechCareers