Data engineer in clinical trials
Summary
Medidata, a clinical-trials technology company, is hiring a data engineer in London to architect an enterprise semantic data fabric and Iceberg-based data lake, build Kafka/AWS/Snowflake ingestion pipelines and Java/Scala services, and use AI tools for schema mapping, query tuning, and GxP/HIPAA/GDPR-compliant validation.
Описание: Medidata is powering smarter treatments and healthier people through digital solutions to support clinical trials. Celebrating over 25 years of ground-breaking technological innovation across more than 38,000 trials and 12 million patients, Medidata offers industry-leading expertise, analytics-powered insights, and one of the largest clinical trial data sets in the industry.
Задачи:
Architect and evolve the enterprise semantic data fabric, converting multi-stream clinical execution datasets into an object-centric model. Design and execute a modern Data Lake strategy centred on Apache Iceberg as the core storage format, ensuring high-performance querying and seamless interoperability with Snowflake and heterogeneous compute engines. Develop and maintain end-to-end multi-stream ingestion pipelines for complex clinical trial schemas. Use AI-driven schema inference and ontology alignment tools to auto-draft mapping artifacts, dramatically reducing integration timelines across different life science datasets. Build scale, fault-tolerant real-time ingestion pipelines using Kafka, AWS, and Snowflake. Write robust enterprise services in Java or Scala, leveraging AI coding assistants for rapid code generation, refactoring, and performance tuning. Promote technical direction and engineering best practices across teams for Change Data Capture (CDC), clustering, data migration, and aggregation. Use AI query-optimization tools to analyse execution plans, auto-tune complex Snowflake/Iceberg SQL workloads, and eliminate performance bottlenecks. Integrate automated AI inferencing and reasoning layers directly into data pipelines to detect telemetry anomalies in real-time and automatically map safety signals back to operational trial datasets. Lead Test-Driven Development (TDD) and Behavior-Driven Development (BDD) initiatives. Use AI test generators to produce HIPAA/GxP-compliant synthetic clinical trial datasets for automated validation. Leverage AI tools to auto-draft validation artifacts, data lineage manifests, and audit documentation required under GxP, HIPAA, and GDPR standards. Troubleshoot complex production issues across distributed data environments and implement resilient, self-healing pipeline architectures.
Требования:
Bachelor's or Master's degree in Computer Science, Data Science, Software Engineering, or equivalent practical experience, alongside proven years of dedicated professional experience in enterprise data engineering and architecture. Expert-level mastery of SQL (OLAP/OLTP) and enterprise cloud data platforms (specifically Snowflake), combined with practical experience architecting open data lake structures using Apache Iceberg. Demonstrated expertise building production backend services in Java or Scala on AWS, alongside deep familiarity with real-time streaming architectures (e.g., Apache Kafka) and modern data engineering design patterns. Active, practical experience integrating AI developer tools (e.g., GitHub Copilot, Cursor) into daily workflows to accelerate SQL query generation, code refactoring, automated testing, and technical documentation drafting. Proven command of Git revision control, CI/CD pipeline automation, and TDD/BDD practices, augmented by AI-driven test case generation and quality checks. Strong foundational understanding of clinical trial data workflows (e.g., EDC architectures), healthcare data models, and life science compliance standards (GxP, HIPAA, GDPR), with the ability to apply AI/ML tools for schema mapping and zero-loss data integrity verification.
Условия:
Base pay is one part of the Total Rewards that Medidata provides to compensate and recognise employees for their work. Most sales positions are eligible for a commission on the terms of applicable plan documents, and many of Medidata's non-sales positions are eligible for annual bonuses.
Medidata believes that benefits should connect you to the support you need when it matters most and provides benefits, including
- medical, dental, life and disability insurance
- a generous pension
- 25+ paid holidays per year