Skip to main content
TALENT247 // SPECIALIST AI EXECUTIVE SEARCH & TECHNICAL RECRUITMENT
ML · Machine Learning

Senior Machine Learning Engineer, Low-Latency Code Models

San Francisco, CA · Full-time · On-site

AI Specialism

Machine Learning

Workplace

On-site

Location

San Francisco, CA

Compensation

$220k – $330k

annual package

Role Dossier & Specification

REF: SENIOR-MACHINE-LEA

THE CLIENT & MANDATE

Our client is the fastest-growing AI-native software engineering platform in history, headquartered in San Francisco and trusted daily by millions of professional developers and majority-adoption engineering teams across the Fortune 500. Unlike generic chat models, their competitive edge comes from purpose-built, ultra-low-latency code models that predict multi-line edits, cursor jumps, and cross-file refactors faster than human thought (<100ms end-to-end).

They have retained Talent247 to hire a Senior Machine Learning Engineer for Low-Latency Code Models onsite in San Francisco, CA. Joining a high-density, low-ego research and engineering team, the appointed engineer will co-design custom speculative architectures, online preference RL loops, and repository-context rankers powering billions of daily code completions.

KEY RESPONSIBILITIES

  • Train and iterate on specialized dense and sparse Mixture-of-Experts (MoE) code models optimized for multi-hunk next-edit prediction, diff application, and cross-file semantic navigation.
  • Pioneer speculative edit-decoding algorithms—exploiting the high overlap between existing source files and proposed edits—to deliver 1,000+ tok/s effective edit speeds under strict sub-100ms p95 SLAs.
  • Co-design repository-wide context retrieval and AST/LSP-aware token compression models that pack exact type definitions, call-graph neighbors, and recent edit history into ultra-compact prompts.
  • Run continuous online distillation and reinforcement learning (DPO/GRPO) from real-world implicit edit acceptance/rejection signals while strictly honoring enterprise zero-data-retention boundaries.
  • Partner side-by-side with custom inference kernel engineers to co-optimize model depth/width, KV-cache reuse across keystrokes, and FP8/FP4 quantization.
  • Own live A/B experimentation and offline replay harnesses measuring keystroke savings, edit precision, and zero-distraction calibration.

TECHNICAL ENVIRONMENT & OFFERING

Stack: Python, PyTorch, C++/CUDA/Triton, Megatron/FSDP, SGLang/TensorRT-LLM/custom speculative serving engines, tree-sitter/LSP, multi-thousand H100/B200 GPU clusters. Workplace: Onsite in San Francisco, CA. Compensation: $220,000 – $330,000 base salary plus premier hyper-growth equity, daily catered meals, and full medical/401(k) benefits.

Core Requirements

ESSENTIAL QUALIFICATIONS

  • 4+ years of machine learning research engineering experience training or post-training large transformer models, code models, or ultra-low-latency neural systems.
  • Deep hands-on mastery of PyTorch distributed training, model distillation, SFT/RLHF/GRPO post-training, and sequence-level loss engineering.
  • Strong systems intuition for transformer inference bottlenecks—KV-cache prefix sharing, speculative draft-verify decoding, batching dynamics, and quantization trade-offs.
  • Exceptional software engineering taste: comfortable jumping from a PyTorch training loop into Rust/TypeScript editor context assembly or CUDA profiling.
  • Passion for developer tools and building AI models whose quality you feel personally in your own workflow every hour.

PREFERRED / BONUS EXPERIENCE

  • Prior work on code generation models, speculative/medusa/grammatical decoding, diff-representation tokenization, or neural code search.
  • Familiarity with compiler/IDE internals (LSP, tree-sitter, control-flow/type graphs) as structured conditioning signals for LLMs.
  • Publications at NeurIPS, ICML, ICLR, or PLDI/ICSE, or standout open-source ML/devtools contributions.

Position Telemetry

Location San Francisco, CA
Department Machine Learning
Employment type Full-time
Workplace type On-site
Salary range
$220k – $330k annual package