Skip to main content
TALENT247 // SPECIALIST AI EXECUTIVE SEARCH & TECHNICAL RECRUITMENT
OPS · MLOps & Platform

Senior ML Data Platform Engineer, Petabyte Video Mining

London, UK · Full-time · Hybrid

AI Specialism

MLOps & Platform

Workplace

Hybrid

Location

London, UK

Compensation

£100k – £140k

annual package

Role Dossier & Specification

REF: SENIOR-ML-DATA-PLA

THE CLIENT & MANDATE

Our client is a flagship London physical-AI leader scaling embodied foundation models across global vehicle fleets. Because uniform random road driving is overwhelmingly repetitive, their model breakthroughs depend on an intelligent petabyte-scale data engine that continuously mines rare, high-surprise behavioral interactions from hundreds of millions of driven miles.

They have retained Talent247 to hire a Senior ML Data Platform Engineer for their Petabyte Video Mining team in London. Their group builds multimodal embedding indices, active-learning curation pipelines, and streaming columnar storage engines that turn raw fleet video into high-signal pre-training and post-training mixtures.

KEY RESPONSIBILITIES

  • Architect distributed multimodal ingestion and curation pipelines processing tens of terabytes of daily multi-camera video, kinematics, and audio logs across hybrid cloud and on-prem clusters.
  • Build high-throughput semantic and spatio-temporal retrieval engines over billions of video clips using vision-language embeddings, Lance/Parquet columnar formats, and approximate nearest-neighbor indices.
  • Partner with foundation model researchers to operationalize active-learning and uncertainty-sampling loops that automatically surface high-loss or safety-critical driving episodes from fleet uploads.
  • Engineer zero-copy GPU video transcoding, hardware-accelerated decoding (NVDEC), and deterministic shard shuffling layers feeding multi-thousand-GPU training jobs without IO starvation.
  • Design dataset lineage, deduplication, and mixture-weighting frameworks so scientists can declaratively compose reproducible petabyte training recipes in minutes.
  • Drive storage tiering and compute cost efficiency across object storage, NVMe caches, and spot GPU/CPU fleets.

TECHNICAL ENVIRONMENT & OFFERING

Stack: Python, Rust, Ray Data, Apache Spark, Lance / Apache Arrow / Parquet, PyTorch, NVDEC/FFmpeg, Kubernetes, AWS/GCP & bare-metal object storage. Workplace: Hybrid in London, UK. Compensation: £100,000 – £140,000 base salary plus equity ownership, comprehensive UK benefits, and Skilled Worker visa sponsorship.

Core Requirements

ESSENTIAL QUALIFICATIONS

  • 5+ years of data platform, ML infrastructure, or distributed systems engineering experience operating at multi-petabyte scale.
  • Production mastery of distributed compute frameworks such as Ray Data, Apache Spark, or Dask in Python (plus Rust, Go, or C++ for IO-critical components).
  • Hands-on experience with modern multimodal/columnar storage formats (Lance, Parquet, Arrow, WebDataset, Mosaic Streaming) and cloud object-store throughput tuning.
  • Practical familiarity with running batch GPU inference (CLIP/VLM embeddings, auto-labeling, deduplication) across millions of images or video segments.
  • Strong systems fundamentals in networking, concurrent IO, distributed caching, and Kubernetes job orchestration.

PREFERRED / BONUS EXPERIENCE

  • Prior experience building data curation engines for autonomous driving, robotics, or frontier video/multimodal foundation models.
  • Experience with hardware-accelerated video codecs (H.265/AV1, NVDEC/PyNvVideoCodec) and random-access frame seeking at training time.
  • Contributions to open-source data infrastructure ecosystems (Ray, LanceDB, Arrow, DataFusion, Polars).

Position Telemetry

Location London, UK (Hybrid)
Department MLOps & Platform
Employment type Full-time
Workplace type Hybrid
Salary range
£100k – £140k annual package