Data Engineer – AWS (Data Observability)
Posted 4
views
Summary
Data engineer implementing monitoring and observability for enterprise data pipelines on AWS: building data quality, freshness/volume/schema checks and alerting, setting up tooling like Monte Carlo, Great Expectations, CloudWatch and Datadog, and handling incident triage. Core stack: AWS (Glue, S3, Lambda, Redshift), Python, SQL.
Implement monitoring and observability across enterprise data pipelines to ensure reliability and data trust.
- Build data quality checks, freshness/volume/schema monitoring and alerting
- Set up observability tooling (e.g., Monte Carlo, Great Expectations, CloudWatch, Datadog)
- Perform incident triage and root cause analysis; maintain SLAs and runbooks
Requirements
- Must have: 4+ yrs, AWS (Glue, S3, Lambda, Redshift, CloudWatch), Python, SQL, data quality frameworks