Skip to main content
TALENT247 // SPECIALIST AI EXECUTIVE SEARCH & TECHNICAL RECRUITMENT
INF · AI Infrastructure

Staff Model Performance & TensorRT-LLM Engineer

San Francisco, CA · Full-time · Hybrid

AI Specialism

AI Infrastructure

Workplace

Hybrid

Location

San Francisco, CA

Compensation

$240k – $335k

annual package

Role Dossier & Specification

REF: STAFF-MODEL-PERFOR

THE CLIENT & MANDATE

Our client is a multi-billion-dollar AI inference infrastructure platform headquartered in San Francisco, powering mission-critical low-latency model serving for breakout AI-native applications and healthcare, voice, code, and creative-media enterprises. Rather than offering generic shared endpoints, their Model Performance Engineering group squeezes hardware-limit throughput and sub-50ms time-to-first-token out of custom multimodal, audio, diffusion, and 400B+ LLM architectures across dedicated Hopper and Blackwell clusters.

They have retained Talent247 to appoint a Staff Model Performance & TensorRT-LLM Engineer in San Francisco, CA. Operating at the intersection of GPU kernel optimization, compiler runtimes, and speculative decoding research, the successful candidate will own flagship model optimization benchmarks and core runtime primitives across their inference fleet.

KEY RESPONSIBILITIES

  • Architect end-to-end TensorRT-LLM, SGLang, and vLLM runtime optimizations across NVIDIA H100, H200, B200, and GB200 NVL72 clusters—delivering industry-leading latency-per-dollar Pareto frontiers.
  • Design and tune low-bit quantization pipelines (NVFP4, FP8 KV-cache & GEMMs, INT4 AWQ/GPTQ) with custom calibration recipes preserving lossless reasoning, code synthesis, and audio prosody quality.
  • Build online-adaptive speculative decoding engines (EAGLE-style draft heads, n-gram/PLD lookup, multi-token prediction) tailored dynamically to live customer domain traffic.
  • Author custom CUDA, CUTLASS, and Triton kernels for non-standard attention masks, multimodal vision/audio encoders, Diffusion Transformers (DiTs), and disaggregated prefill-decode KV transfers.
  • Diagnose deep GPU runtime bottlenecks across CUDA Graph capture, Python/C++ host launch overhead, NCCL/NVSHMEM collectives, and HBM memory fragmentation using Nsight Compute and Nsight Systems.
  • Publish authoritative technical deep-dives and mentor senior inference engineers across core runtime and forward-deployed performance pods.

TECHNICAL ENVIRONMENT & OFFERING

Stack: C++20, CUDA, CUTLASS/CuTe, Triton, Python, TensorRT-LLM, SGLang, vLLM, FlashInfer, PyTorch, Nsight Compute/Systems, Hopper & Blackwell GPU pods. Workplace: Hybrid in San Francisco, CA. Compensation: $240,000 – $335,000 base salary plus high-upside Series-stage equity, comprehensive healthcare, and 401(k).

Core Requirements

ESSENTIAL QUALIFICATIONS

  • 6+ years of GPU inference performance engineering, deep learning compiler/runtime development, or HPC kernel optimization experience.
  • Deep production mastery of TensorRT-LLM, SGLang, or vLLM internals—including custom C++/CUDA plugins, paged attention, chunked prefill, and multi-GPU tensor/expert parallelism.
  • Proven track record profiling and eliminating kernel-level and host-bound bottlenecks with NVIDIA Nsight Systems and Nsight Compute.
  • Hands-on expertise in post-training quantization (FP8, NVFP4, INT4/INT8) and speculative decoding architectures.
  • Strong communication and technical leadership skills driving cross-team runtime architecture decisions.

PREFERRED / BONUS EXPERIENCE

  • Core upstream contributions to TensorRT-LLM, FlashInfer, SGLang, vLLM, CUTLASS, or xDiT.
  • Experience optimizing non-LLM generative workloads (streaming TTS/ASR audio models, real-time video DiTs, or large cross-encoder rerankers) alongside frontier LLMs.
  • Direct hands-on tuning experience on NVIDIA Blackwell B200 / GB200 NVL72 hardware.

Position Telemetry

Location San Francisco, CA (Hybrid)
Department AI Infrastructure
Employment type Full-time
Workplace type Hybrid
Salary range
$240k – $335k annual package