Remote Full Time Role_Google Cloud Platform Data Engineer (Platform, ML & Analytics)_Remote, USA
Amaze Systems Inc Remote Full Time Role_Google Cloud Platform Data Engineer (Platform, ML & Analytics)_Remote, USA
Description
Role: Google Cloud Platform Data Engineer (Platform, ML & Analytics)
Location: Remote, USA
Duration: Long term contract
Position Type: Full time
Experience required: 12+ Years
Job Description:
The Senior Data Engineer (Platform, ML & Analytics) at Client will be the only data engineering owner on the Al Insight & Next Best Action (NBA) engine team. They own the whole data lifecycle, from raw ingestion through curated marts to the Feature Store and ML training datasets, and they’re also responsible for HIPAA compliance, disaster recovery (DR) and archival.
Required Qualification in the JD by skill area.
Skill area:
- Data engineering (core): 50%
- Kafka/Dataflow ingestion, Spark ETL/ELT, BigQuery marts, data quality and SLAs, partitioning and cost, Python/SQL, data modeling
- Data engineering (platform governance): 20% HIPAA/PHI controls, disaster recovery (DR) zones and failover, archival and retention, documentation
- ML data engineering: 25% Feature Store population, training dataset pipelines (train/eval splits, versioning), feature engineering on Spark, drift and freshness monitoring, Vertex Al Pipelines and MLOps
- AI: 5% Vertex Al appears only as a pipeline tool or downstream consumer. The JD doesn't ask for model building, GenAl or LLM work
Key asks
- Ingestion: Stream health, benefits, news and click-through files (CV/Parquet) through Kafka and Dataflow into Cloud Storage and BigQuery-
- Transformation: Run ETL/ELT jobs on Dataflow and Datapros Spark to produce clean, normalized BigQuery datasets.
- Curated marts: Build 360° denormalized views that combine prescription (Rx), benefits, behavioral and clinical signals.
- Feature Store and ML datasets: Load the Google Cloud Platform Feature Store, build batch and online feature paths, create train/eval splits and version datasets.
- Monitor data drift and feature freshness with Vertex Al.
- Adobe Analytics: Bring in behavioral event data and line it up with clinical and benefits data.
- Data quality and BigQuery operations: Set up SLAs, alerting, schema validation and anomaly detection. Manage partitioning, schema changes and cost.
- Disaster recovery: Set recovery time and recovery point targets (RTO/RPO), replicate BigQuery across regions, maintain DR buckets and Feature Store snapshots, and test failover runbooks.
- Archival: Set BigQuery table expiry, move older data to cheaper Cloud Storage tiers (Nearline, Coldline, Archive) and meet HIPAA retention
- rules.
- HIPAA and patient health information (PHI): Apply field-level encryption and masking and de-identify data (Safe Harbor or Expert
- Determination) before training. Enforce access controls and audit logging.
- Documentation: Write up data contracts, data lineage, Feature Store definitions, DR runbooks and archival policies.
Must-have skill
- Core data skills: BigQuery (advanced), Kafka, Dataflow/Beam, Spark (PySpark or Scala), Pub/Sub, Composer/Airflow, plus strong SQL and data modeling (Lakehouse).
- ML data work: Feature Stores, Vertex Al Pipelines or similar MLOps tools, and the ML lifecycle.
- Proven experience with: DR design, archival and retention for regulated data, and hands-on HIPAA/PHI controls.
Nice to have
- Adobe Experience Platform, Looker, Google Cloud Platform customer-managed encryption keys (CMEK), HIPAA business associate agreement (BAA) awareness,
- NIST/HITRUST, and healthcare data experience (Rx, clinical or benefits).
Tirath Prajapati
Sr. Talent Acquisition Specialist
Amaze Systems Inc
D: +1
E: |
Skills
Sign in to read and join the discussion.