Skip to main content
TALENT247 // SPECIALIST AI EXECUTIVE SEARCH & TECHNICAL RECRUITMENT
INF · AI Infrastructure

Principal GPU Inference Kernel Engineer

San Francisco, CA · Full-time · On-site

AI Specialism

AI Infrastructure

Workplace

On-site

Location

San Francisco, CA

Compensation

$300k – $395k

annual package

Role Dossier & Specification

REF: PRINCIPAL-GPU-INFE

THE CLIENT & MANDATE

Our client is a premier frontier AI research and deployment company in San Francisco serving trillions of tokens every day across interactive reasoning assistants, real-time voice streams, and developer APIs. At planetary inference scale, shaving microseconds off attention, MoE dispatch, and quantized GEMM kernels translates directly into tens of thousands of reclaimed GPUs and dramatically deeper test-time reasoning budgets for users worldwide.

They have partnered with Talent247 to appoint a Principal GPU Inference Kernel Engineer onsite in San Francisco, CA. Operating at the bare metal of NVIDIA Hopper, Blackwell, and next-generation accelerator architectures, the successful candidate will set the technical bar for custom CUDA/PTX kernels and disaggregated serving runtimes across flagship frontier models.

KEY RESPONSIBILITIES

  • Design and hand-optimize state-of-the-art CUDA, PTX, CUTLASS, and CuTe C++ kernels for paged/sparse FlashAttention, grouped MoE GEMMs, fused activation-quantization, and speculative multi-token verification.
  • Push hardware utilization near theoretical tensor-core and HBM3e/HBM4 bandwidth ceilings using warp specialization, TMA asynchronous copies, persistent kernels, and NVFP4/FP8/MXFP microscaling formats.
  • Co-architect disaggregated prefill-and-decode serving topologies and ultra-low-latency NVLink / InfiniBand KV-cache transfer engines (custom NVSHMEM / GPUDirect RDMA collectives).
  • Partner directly with pre-training model architects to co-design hardware-friendly attention heads, MoE expert granularities, and KV-cache compression schemes 12 months ahead of launch.
  • Build deterministic cycle-level profiling harnesses, numerical accuracy verifiers, and automated autotuning pipelines across heterogeneous GPU SKUs.
  • Serve as an organization-wide technical authority mentoring senior kernel and compiler engineers across inference and RL rollout teams.

TECHNICAL ENVIRONMENT & OFFERING

Stack: C++20, CUDA, PTX/SASS inspection, CUTLASS 3.x / CuTe, Triton, NVSHMEM/NCCL, PyTorch custom ops, Nsight Compute/Systems, Hopper (H100/H200) & Blackwell (B200/GB200 NVL72) superclusters. Workplace: Onsite in San Francisco, CA. Compensation: $300,000 – $395,000 base salary plus exceptional Principal-level equity grants and comprehensive benefits.

Core Requirements

ESSENTIAL QUALIFICATIONS

  • 8+ years of low-level GPU performance engineering, HPC kernel architecture, or deep learning inference runtime development in C++ and CUDA.
  • Mastery of modern NVIDIA GPU microarchitecture (Hopper WGMMA/TMA, Blackwell tcgen05/NVFP4, warp scheduling, register pressure, shared-memory swizzling, L2 persistence).
  • Track record authoring production kernels that outperform vendor library baselines (cuBLAS, cuDNN) on real transformer or MoE workloads.
  • Deep familiarity with multi-GPU communication primitives (NVLink, NVSwitch, NVSHMEM, GPUDirect RDMA) and CUDA Graph execution models.
  • Ability to bridge low-level SASS/PTX hardware realities with high-level model architecture trade-offs.

PREFERRED / BONUS EXPERIENCE

  • Core authorship or major contributions to FlashAttention, FlashInfer, TensorRT-LLM, vLLM, SGLang, DeepEP, FasterTransformer, or CUTLASS.
  • Experience optimizing large-scale disaggregated prefill/decode serving or multi-node NVL72 rack-scale inference.
  • Compiler internals familiarity (Triton backend, MLIR, LLVM NVPTX) for custom code-generation DSLs.

Position Telemetry

Location San Francisco, CA
Department AI Infrastructure
Employment type Full-time
Workplace type On-site
Salary range
$300k – $395k annual package