Skip to main content
TALENT247 // SPECIALIST AI EXECUTIVE SEARCH & TECHNICAL RECRUITMENT
OPS · MLOps & Platform

Senior Multi-Cloud GPU Orchestration Engineer

New York, NY · Full-time · Hybrid

AI Specialism

MLOps & Platform

Workplace

Hybrid

Location

New York, NY

Compensation

$200k – $280k

annual package

Role Dossier & Specification

REF: SENIOR-MULTI-CLOUD

THE CLIENT & MANDATE

Our client is a leading AI inference cloud headquartered in San Francisco with a major distributed infrastructure hub in New York City. To shield fast-scaling enterprise and AI-native customers from single-cloud GPU capacity crunches and regional outages, their Multi-Cloud Compute Fabric orchestrates tens of thousands of GPUs seamlessly across AWS, GCP, Azure, OCI, and specialized bare-metal neoclouds as a single logical inference mesh.

They have partnered with Talent247 to hire a Senior Multi-Cloud GPU Orchestration Engineer in New York, NY. The successful candidate will design global cross-cloud capacity brokers, sub-second scale-from-zero container/weight loaders, and latency-aware Envoy/Rust traffic routers sustaining 99.995% availability.

KEY RESPONSIBILITIES

  • Architect federated Kubernetes control planes and declarative placement schedulers in Go and Rust brokering real-time H100/H200/B200 GPU capacity across 10+ hyperscale and bare-metal cloud providers.
  • Build global Layer-7 AI traffic routers (Envoy / Rust data planes) balancing geo-proximity, queue depth, prefix-cache locality, and live cross-region GPU health signals with zero-downtime failover.
  • Engineer ultra-fast cross-cloud model weight & container caching layers leveraging NVMe tiering, OCI lazy-pulling, and high-throughput regional object stores to cut replica cold starts to single-digit seconds.
  • Design secure dedicated-VPC customer data planes (PrivateLink, Private Service Connect, WireGuard/Cilium mesh, customer-managed KMS) meeting SOC2 Type II and HIPAA enterprise isolation requirements.
  • Implement predictive autoscaling and spot/on-demand/reserved GPU arbitrage engines optimizing multi-million-dollar monthly compute fleets.
  • Own chaos-engineering game days simulating full cloud-region blackouts and GPU hardware degradations without dropping streaming tokens.

TECHNICAL ENVIRONMENT & OFFERING

Stack: Go, Rust, Python, Kubernetes (custom operators, Karpenter/custom provisioners), Cilium/eBPF, Envoy, gRPC, Terraform/Pulumi, Prometheus/ClickHouse, AWS/GCP/Azure/OCI & bare-metal clouds. Workplace: Hybrid in New York, NY. Compensation: $200,000 – $280,000 base salary plus generous equity grants and full US benefits.

Core Requirements

ESSENTIAL QUALIFICATIONS

  • 5+ years of cloud infrastructure, distributed systems, or multi-cluster Kubernetes platform engineering experience in Go or Rust.
  • Deep production mastery of Kubernetes federation, custom controllers/operators, CNI networking (Cilium/Calico/WireGuard), and high-throughput L7 proxies (Envoy).
  • Hands-on experience managing production GPU fleets across multiple cloud providers, including NVIDIA driver/device-plugin lifecycles, MIG, and GPU health monitoring (DCGM).
  • Strong grounding in cloud security primitives (IAM Federation, PrivateLink/PSC, mTLS service meshes, zero-trust tenant isolation).
  • Track record owning mission-critical data-plane or control-plane infrastructure under strict 99.99%+ uptime SLAs.

PREFERRED / BONUS EXPERIENCE

  • Experience building serverless GPU autoscalers, scale-to-zero cold-start accelerators, or multi-node InfiniBand/RoCE gang scheduling across neoclouds.
  • Contributions to Envoy AI Gateway, Kubernetes Cluster API, Cilium, or OCI image-streaming runtimes.
  • Familiarity with LLM serving patterns (disaggregated prefill/decode routing, KV-cache session affinity).

Position Telemetry

Location New York, NY (Hybrid)
Department MLOps & Platform
Employment type Full-time
Workplace type Hybrid
Salary range
$200k – $280k annual package