Senior Data Engineer (Data Platform)
Posted Updated
- Vida is looking for a Senior Data Engineer to help shape the data platform that supports much of Vida’s business
- Our team builds and operates the systems that ingest, validate, transform, and serve data across Vida
- We work with data from health plans, employers, clinical programs, and internal systems across member identity, eligibility, claims, billing, reporting, and analytics
- You will own data systems and domains end to end, from ingestion and validation through storage, transformation, and delivery
- You will make architecture decisions, change production systems safely, and help other engineers make good decisions about data infrastructure
- Own production data systems and domains from ingestion through delivery
- Design ingestion patterns that handle incomplete, duplicated, delayed, malformed, and changing partner data
- Define the contracts between source systems, ingestion, transformation, and downstream consumers
- Design data models with clear grain, keys, history, and definitions so the same question doesn’t get two different answers
- Evolve schemas, pipelines, and datasets safely, with retries, backfills, and replay that don’t corrupt or duplicate data
- Improve pipeline performance, observability, and failure handling so you can tell when something is wrong and why
- Work with Engineering, Product, Operations, and Analytics to figure out what the data needs to represent when requirements are incomplete or messy
- Additional responsibilities as needed
- Clear written communication, including documenting technical decisions and explaining changes to people outside of engineering
- Experience designing and operating batch and asynchronous data pipelines, including the failure modes that appear in distributed systems
- Bachelor’s degree at a minimum
- Experience with a cloud data platform and an understanding of storage, query performance, and cost
- Strong SQL and data modeling fundamentals, including grain, keys, history, dimensional modeling, and the tradeoffs between different representations of the same data
- The ability to change production systems in ordered, reversible steps without breaking their consumers
- Working proficiency in Python for pipelines, services, tooling, and tests
- You do not need previous healthcare experience or experience with every technology we use
- Strong instincts around data correctness, including validation, idempotency, late and duplicated records, schema changes, backfills, and reprocessing
- 5+ years building production data systems