Data Ingestion Engineer (freelance) - Remote - 6 months+
Posted
Description
FREELANCE DATA ENGINEER - REMOTE - 6 MONTHS+
My client working in the automotive industry are currently searching for multiple data engineers to work on reliability, efficiency and throughput of the pipelines of a data platform used for ingesting 500,000 hours of driving data to 100PB.
Responsibilities:
- Debug and fix failing or blocked ingestion pipelines
- Support orchestration across multi-step ingestion workflows
- Optimise spark jobs and data processing pipelines
- Work on high-volume batch processing systems
Required experience:
- Must be experienced in engineering high-volume data processing platforms ingesting multi-millions of records per day
- Strong experience with Spark and Python
- Ideally experience with Airflow, Databricks, Flyte or similar orchestration tooling
- Ideally experience with Java or Scala in Spark-based environment
- Experience working on projects within Vision & Robotics domain
- Must be EU based and hold active freelancer status
If you'd like to be considered for this role please apply directly through the link or send your CV to j.wylie@annapurnarecruitment.com to be considered. Please pass this advert onto anyone else who might be suitable.
Skills
Sign in to read and join the discussion.