Senior Data Engineer | Embodied AI
Posted 5
views
Summary
Founding-team Senior Data Engineer for a US-based embodied AI company (hiring via dada consultants, based in Singapore): builds and runs a petabyte-scale AWS data lake/lakehouse for multimodal spatial data — ingest pipelines, distributed CPU/GPU processing, and ML training data delivery. Stack: Python plus Go/Rust/C++/Java/Scala, Spark/Ray/Flink, Iceberg/Delta Lake, Terraform.
We are partnering with a US-based embodied AI company to hire a
Senior Data Engineer .
This is a hands-on, founding-team role suited to someone who has personally built and operated petabyte-scale data platforms in production.
Job Responsibilities • Build and scale a PB-scale data lake / lakehouse for multimodal spatial data — object storage layout, partitioning, open table formats, catalogs, and metadata • Own data versioning and lineage: branches, commits, reproducible dataset snapshots, and provenance from raw capture to customer delivery • Build high-throughput, fault-tolerant ingest pipelines — upload, validation, checksumming, content-addressed deduplication, and promotion through bronze → silver → gold • Build and operate large-scale distributed processing across CPU and GPU fleets — video decode, frame extraction, sync and calibration, large-scale 3D and spatial data processing, and model-based auto-labeling • Build and maintain data pipelines at scale: idempotent jobs, retries, backfills, partitioned reprocessing, and clean failure recovery • Build the dataset curation and query layer — metadata indexing, SQL over the lake, and search so researchers and customers can slice data without copying it • Keep GPUs fed: large-scale ML training data formats, caching, prefetching, and efficient cross-region data movement • Own platform reliability, observability, and cost on AWS — S3 tiering, lifecycle, egress, spot and right-sizing, capacity planning, and alerting • Contribute to infrastructure-as-code and CI/CD; write design docs and runbooks so others can operate the platform without tribal knowledge • Work closely with data capture, labeling, and QA teams to translate workload requirements into platform capabilities
Job Requirements • B.S. or M.S. in Computer Science, Software Engineering, or a related field, or equivalent practical experience • 6+ years in data engineering, data platform, or distributed systems engineering • Hands-on experience building and operating a data platform at petabyte scale in production — able to speak concretely to volumes, throughput, object counts, and failure modes personally handled • Deep object storage experience (S3 or equivalent): key/prefix design, throughput limits, small-file problem, lifecycle tiering, and cost control • Production experience with open table formats — Iceberg, Delta Lake, or Hudi; catalogs such as Glue, Nessie, or similar • Strong experience with distributed processing frameworks — Spark, Ray, Flink, or Dask — tuned for large production jobs • Experience with workflow orchestration (Airflow, Argo, Prefect, or similar) with strong discipline around idempotency, retries, and data contracts • Strong Python plus at least one of Go, Rust, C++, Java, or Scala • Solid AWS infrastructure skills — EC2, EKS/ECS, S3, IAM, VPC — and Terraform or equivalent IaC • Production on-call experience for a data platform; care deeply about uptime, correctness, and cost
www.dadaconsultants.com Licence Number: 18S9037 EA Registration Number: R23112003 Business Registration Number: 201735941W
Senior Data Engineer .
This is a hands-on, founding-team role suited to someone who has personally built and operated petabyte-scale data platforms in production.
Job Responsibilities • Build and scale a PB-scale data lake / lakehouse for multimodal spatial data — object storage layout, partitioning, open table formats, catalogs, and metadata • Own data versioning and lineage: branches, commits, reproducible dataset snapshots, and provenance from raw capture to customer delivery • Build high-throughput, fault-tolerant ingest pipelines — upload, validation, checksumming, content-addressed deduplication, and promotion through bronze → silver → gold • Build and operate large-scale distributed processing across CPU and GPU fleets — video decode, frame extraction, sync and calibration, large-scale 3D and spatial data processing, and model-based auto-labeling • Build and maintain data pipelines at scale: idempotent jobs, retries, backfills, partitioned reprocessing, and clean failure recovery • Build the dataset curation and query layer — metadata indexing, SQL over the lake, and search so researchers and customers can slice data without copying it • Keep GPUs fed: large-scale ML training data formats, caching, prefetching, and efficient cross-region data movement • Own platform reliability, observability, and cost on AWS — S3 tiering, lifecycle, egress, spot and right-sizing, capacity planning, and alerting • Contribute to infrastructure-as-code and CI/CD; write design docs and runbooks so others can operate the platform without tribal knowledge • Work closely with data capture, labeling, and QA teams to translate workload requirements into platform capabilities
Job Requirements • B.S. or M.S. in Computer Science, Software Engineering, or a related field, or equivalent practical experience • 6+ years in data engineering, data platform, or distributed systems engineering • Hands-on experience building and operating a data platform at petabyte scale in production — able to speak concretely to volumes, throughput, object counts, and failure modes personally handled • Deep object storage experience (S3 or equivalent): key/prefix design, throughput limits, small-file problem, lifecycle tiering, and cost control • Production experience with open table formats — Iceberg, Delta Lake, or Hudi; catalogs such as Glue, Nessie, or similar • Strong experience with distributed processing frameworks — Spark, Ray, Flink, or Dask — tuned for large production jobs • Experience with workflow orchestration (Airflow, Argo, Prefect, or similar) with strong discipline around idempotency, retries, and data contracts • Strong Python plus at least one of Go, Rust, C++, Java, or Scala • Solid AWS infrastructure skills — EC2, EKS/ECS, S3, IAM, VPC — and Terraform or equivalent IaC • Production on-call experience for a data platform; care deeply about uptime, correctness, and cost
www.dadaconsultants.com Licence Number: 18S9037 EA Registration Number: R23112003 Business Registration Number: 201735941W