Automation Test Lead – Data & Platform Engineering @ VIRTUSA
Posted 1
view
Summary
Leads test automation for data & platform engineering: owns a scalable, metadata-driven test framework covering batch/streaming data pipelines, APIs, and end-to-end data product validation. Core stack: Python, advanced SQL, DBT, Airflow, Snowflake, Kafka/Kinesis, AWS, Terraform, Jenkins/GitHub Actions, and Great Expectations/Soda.
Mandatory Skills:
- Strong Python (test frameworks, libraries, CLI tools); advanced SQL
- Hands-on DBT, Airflow, Snowflake or similar; ETL/ELT and data modeling
- Great Expectations/Soda; lineage and catalog systems
- Kafka/Kinesis testing; delivery semantics
- Git workflows; CI/CD (Jenkins, GitHub Actions)
- AWS; IaC (Terraform/CloudFormation)
Desired Skills: API contract testing (PACT); basic UI automation; data mesh/data product exposure; Prometheus/Grafana; regulated domain experience (healthcare, life sciences, finance)
Experience: 9+ years in automation/QE; 5+ years building frameworks for data platforms at scale; track record improving reliability/reducing incidents
,(Test Automation Architecture & Strategy: Own a scalable, modular, metadata-driven framework covering data pipelines (batch/streaming), APIs/backend services, and end-to-end data product validation; enable plug-and-play components, parallel execution, environment isolation, deterministic runs, Data Testing Framework Engineering: SQL-based assertions/reconciliation, schema validation, data contracts, lineage/freshness validation, config-driven test definitions (YAML/JSON), Destructive Testing: Schema drift, backward incompatibility, late-arriving data, partial failures, duplicate/missing/out-of-order events, stress, concurrency, retries, DLQ handling, backpressure, ETL/Streaming Validation at Scale: Row/aggregate/hash-based reconciliation, incremental/backfill validation, delivery semantics validation, window/time-based correctness, Data Quality & Observability: Integrate/extend Great Expectations or Soda; custom validations for accuracy, completeness, uniqueness, timeliness; quality dashboards, CI/CD & DataOps Enforcement: Pre-merge gates, release blockers, selective/parallel test execution, GitHub Actions/Jenkins integration, Test Data Management: Synthetic data generation, masking/anonymization, deterministic datasets, edge case simulation, Performance & Reliability Testing: Pipeline/query benchmarks, concurrency/stress testing, data skew analysis, cost/time optimization, Security & Compliance: PII/PHI exposure checks, encryption/access control, retention, audit requirements; support GxP/SOX/ISO frameworks, Cross-Functional Quality Leadership: Work with data engineers, platform teams, architects; mentor engineers, Incident Analysis & Prevention: Root-cause analysis of production data issues, reduce flaky tests, ) Requirements: Python