Skip to main content
TALENT247 // SPECIALIST AI EXECUTIVE SEARCH & TECHNICAL RECRUITMENT
OPS · MLOps & Platform

Senior MLOps & Enterprise LLM Serving Platform Engineer

Berlin / Cologne, Germany · Full-time · Hybrid

AI Specialism

MLOps & Platform

Workplace

Hybrid

Location

Berlin / Cologne, Germany

Compensation

€92k – €125k

annual package

Role Dossier & Specification

REF: SENIOR-MLOPS-ENTER

THE CLIENT & MANDATE

Our client is a multi-billion-dollar German enterprise AI unicorn with major engineering centers in Berlin and Cologne, trusted by more than 100,000 businesses worldwide for mission-critical neural language intelligence and autonomous cross-lingual enterprise workflows. Unlike application wrappers renting third-party APIs, they train proprietary foundation models on dedicated multi-thousand-GPU supercomputing clusters and serve billions of low-latency enterprise requests daily under strict European data-privacy guarantees.

They have engaged Talent247 to hire a Senior MLOps & Enterprise LLM Serving Platform Engineer in Berlin. Joining their Inference & ML Platform division, you will own the internal model deployment control plane, automated regression/canary pipelines, and dynamic GPU cluster scheduling powering real-time translation, writing, voice, and enterprise agent workloads.

KEY RESPONSIBILITIES

  • Architect, scale, and operate Kubernetes-native LLM and multimodal model serving platforms running custom C++/CUDA, Triton, and vLLM/TensorRT-LLM engines across bare-metal and cloud GPU estates.
  • Build zero-downtime model rollout workflows featuring shadow traffic mirroring, automated quality/latency canary gates, instant rollback, and cryptographic artifact provenance.
  • Design intelligent request routing, prefix-cache-aware load balancing, and multi-tier autoscaling policies that hold sub-200ms interactive SLAs while maximizing fleet GPU utilization.
  • Instrument deep GPU and inference observability (TTFT, TPOT, KV-cache pressure, SM occupancy, NVLink throughput) using Prometheus, OpenTelemetry, DCGM, and Grafana.
  • Partner directly with research scientists and kernel engineers to streamline the path from multi-node training checkpoint to quantized production endpoint.
  • Drive GitOps automation, disaster-recovery readiness, and strict zero-retention enterprise compliance across European and North American serving regions.

TECHNICAL ENVIRONMENT & OFFERING

Their platform stack spans Kubernetes, Go, Python, Rust, Triton Inference Server, TensorRT-LLM, vLLM, Envoy, ArgoCD, and dedicated H100/B200 superclusters. Offered hybrid in Berlin (or Cologne), compensation includes €92,000–€125,000 EUR base salary, equity participation, 30 days annual leave, German relocation/Blue Card visa support, and professional development budgets.

Core Requirements

ESSENTIAL QUALIFICATIONS

  • 5+ years of MLOps, ML platform, or production infrastructure engineering experience operating high-throughput distributed services on Kubernetes.
  • Direct production experience serving large transformer models using vLLM, TensorRT-LLM, Triton Inference Server, SGLang, or custom C++/CUDA runtimes.
  • Strong programming skills in Python and Go (or Rust) for writing Kubernetes operators, deployment controllers, and performance benchmarking tools.
  • Deep operational familiarity with Linux networking, NVIDIA GPU Operator/MIG/DCGM telemetry, Envoy/gRPC load balancing, and GitOps (ArgoCD/Flux, Terraform).
  • Proven ability to diagnose complex tail-latency and throughput regressions across model serving stacks.

PREFERRED / BONUS EXPERIENCE

  • Experience operating bare-metal HPC/Slurm or multi-thousand-GPU data center environments alongside cloud Kubernetes clusters.
  • Working knowledge of model quantization (FP8/INT4), speculative decoding, or disaggregated prefill/decode serving architectures.

Position Telemetry

Location Berlin / Cologne, Germany (Hybrid)
Department MLOps & Platform
Employment type Full-time
Workplace type Hybrid
Salary range
€92k – €125k annual package