QA Engineer - ETL
Posted Updated 3
views
Summary
QA engineer (7-12 yrs, Noida) validating ETL/ELT data pipelines: source-to-target reconciliation, anomaly detection, and automated data-quality frameworks in Python and SQL, integrated into CI/CD, with AI-assisted test generation — in an emergency response / public safety / healthcare operations data domain.
kch-description-color">
Disclaimer: This job posting has been aggregated from external source. Role details, content, and availability are subject to change. Applicants are advised to confirm the latest information directly on the company website before applying.
Role Overview
We are seeking an ETL QA Data Engineer to ensure the accuracy, completeness, and reliability of data pipelines and analytics outputs across a high-volume, event-driven domain (e.g., emergency response, public safety, and healthcare operations). You will validate data ingestion and transformation logic, reconcile records between source and target systems, detect anomalies, and help ensure reporting and dashboards reflect trustworthy data.
Key Responsibilities
Data Validation & Reconciliation
- Perform source-to-target reconciliation for event/transaction-style datasets across ingestion and curated layers.
- Validate completeness, timeliness, and consistency of daily/near-real-time ingestion pipelines.
- Identify and investigate data anomalies (missing records, spikes/drops, duplicates, out-of-range values, schema drift).
- Validate correctness of KPIs and metrics used in dashboards and reports (e.g., volumes, rates, turnaround times, error trends).
- Ensure referential integrity across key entities (e.g., events/encounters, units/resources, locations, agencies/sites).
- Partner with engineering teams to debug discrepancies, confirm root causes, and validate fixes.
Automation & Framework Development
- Build and maintain automated data validation frameworks using Python (PyTest/Unittest) and data quality libraries/tools.
- Develop reusable validation assets:
- reconciliation templates
- data quality rules (nulls, ranges, uniqueness, referential integrity)
- regression suites for critical pipelines
- Integrate validation into CI/CD and orchestration workflows (e.g., validation gates per pipeline run).
- Produce validation artifacts (test reports, defect summaries, data quality scorecards) for ongoing monitoring.
AI-Assisted Validation (with Human Verification)
- Use AI tools (LLMs, Qodo, etc.) to accelerate:
- validation query generation
- reconciliation logic
- test case creation (including edge-case discovery)
- Apply AI-assisted techniques for anomaly detection ideas, boundary scenarios, and coverage expansion.
- Validate all AI-generated outputs against actual data and documented business rules before adoption.
Required Qualifications
- Strong SQL skills for reconciliation and validation (joins, aggregation, window functions, profiling).
- Strong hands-on Python experience for test automation and data validation (not just scripting).
- Experience in ETL/ELT QA, data testing, analytics validation, or data quality engineering.
- Understanding of event-driven / transaction-based systems and common pipeline failure patterns (late-arriving data, corrections, duplicates).
- Experience validating large datasets and reporting layers with high reliability expectations.
Preferred Qualifications
- Experience with data quality tools such as:
- Great Expectations (preferred)
- custom Python validation frameworks
- dbt tests / similar validation frameworks
- Familiarity with cloud data platforms and modern data stacks (any of):
- Snowflake / BigQuery / Redshift
- Databricks / Spark
- Airflow / ADF / Prefect
- Experience with sensitive data handling practices (privacy/security considerations).
- Exposure to interoperability/integration patterns (APIs, HL7/FHIR, EDI, batch feeds)-as applicable to the environment.
What Success Looks Like
- Automated validation coverage for critical pipelines (freshness, completeness, reconciliation).
- Early detection of anomalies and measurable reduction in downstream reporting issues.
- Repeatable, documented validation rules aligned to business definitions and metrics.
- Efficient collaboration with engineering teams to shorten incident triage and fix verification cycles.
Required Skills
QA ETL
Disclaimer: This job posting has been aggregated from external source. Role details, content, and availability are subject to change. Applicants are advised to confirm the latest information directly on the company website before applying.