---
title: "Senior Machine Learning Engineer, Low-Latency Code Models"
slug: "senior-machine-learning-engineer-low-latency-code"
id: "19fe8089-228e-4391-bd00-2fd1bb280f81"
department: "Machine Learning"
department_code: "ML"
location: "San Francisco, CA"
workplace_type: "On-site"
employment_type: "Full-time"
compensation: "$220k – $330k (annual package)"
is_remote: false
is_featured: false
headcount: 1
canonical_url: "https://jobs.talent247.co/jobs/senior-machine-learning-engineer-low-latency-code"
markdown_url: "https://jobs.talent247.co/jobs/senior-machine-learning-engineer-low-latency-code.md"
apply_endpoint: "https://jobs.talent247.co/api/public/jobs/senior-machine-learning-engineer-low-latency-code/apply"
schema_endpoint: "https://jobs.talent247.co/api/public/jobs/senior-machine-learning-engineer-low-latency-code/form-schema"
webmcp_tool: "apply_for_role"
updated_at: "2026-11-15T17:38:46.893Z"
content_signal: "search=yes, ai-input=yes, ai-train=no"
---

# Senior Machine Learning Engineer, Low-Latency Code Models

## Position Telemetry

- **Reference:** `SENIOR-MACHINE-LEA`
- **Practice / Sector:** Machine Learning (`ML`)
- **Location:** San Francisco, CA
- **Workplace Type:** On-site
- **Employment Type:** Full-time
- **Compensation:** $220k – $330k (annual package)
- **Canonical HTML URL:** https://jobs.talent247.co/jobs/senior-machine-learning-engineer-low-latency-code

## Role Dossier & Specification

THE CLIENT & MANDATE
Our client is the fastest-growing AI-native software engineering platform in history, headquartered in San Francisco and trusted daily by millions of professional developers and majority-adoption engineering teams across the Fortune 500. Unlike generic chat models, their competitive edge comes from purpose-built, ultra-low-latency code models that predict multi-line edits, cursor jumps, and cross-file refactors faster than human thought (<100ms end-to-end).

They have retained Talent247 to hire a Senior Machine Learning Engineer for Low-Latency Code Models onsite in San Francisco, CA. Joining a high-density, low-ego research and engineering team, the appointed engineer will co-design custom speculative architectures, online preference RL loops, and repository-context rankers powering billions of daily code completions.

KEY RESPONSIBILITIES
• Train and iterate on specialized dense and sparse Mixture-of-Experts (MoE) code models optimized for multi-hunk next-edit prediction, diff application, and cross-file semantic navigation.
• Pioneer speculative edit-decoding algorithms—exploiting the high overlap between existing source files and proposed edits—to deliver 1,000+ tok/s effective edit speeds under strict sub-100ms p95 SLAs.
• Co-design repository-wide context retrieval and AST/LSP-aware token compression models that pack exact type definitions, call-graph neighbors, and recent edit history into ultra-compact prompts.
• Run continuous online distillation and reinforcement learning (DPO/GRPO) from real-world implicit edit acceptance/rejection signals while strictly honoring enterprise zero-data-retention boundaries.
• Partner side-by-side with custom inference kernel engineers to co-optimize model depth/width, KV-cache reuse across keystrokes, and FP8/FP4 quantization.
• Own live A/B experimentation and offline replay harnesses measuring keystroke savings, edit precision, and zero-distraction calibration.

TECHNICAL ENVIRONMENT & OFFERING
Stack: Python, PyTorch, C++/CUDA/Triton, Megatron/FSDP, SGLang/TensorRT-LLM/custom speculative serving engines, tree-sitter/LSP, multi-thousand H100/B200 GPU clusters. Workplace: Onsite in San Francisco, CA. Compensation: $220,000 – $330,000 base salary plus premier hyper-growth equity, daily catered meals, and full medical/401(k) benefits.

## Core Requirements

ESSENTIAL QUALIFICATIONS
• 4+ years of machine learning research engineering experience training or post-training large transformer models, code models, or ultra-low-latency neural systems.
• Deep hands-on mastery of PyTorch distributed training, model distillation, SFT/RLHF/GRPO post-training, and sequence-level loss engineering.
• Strong systems intuition for transformer inference bottlenecks—KV-cache prefix sharing, speculative draft-verify decoding, batching dynamics, and quantization trade-offs.
• Exceptional software engineering taste: comfortable jumping from a PyTorch training loop into Rust/TypeScript editor context assembly or CUDA profiling.
• Passion for developer tools and building AI models whose quality you feel personally in your own workflow every hour.

PREFERRED / BONUS EXPERIENCE
• Prior work on code generation models, speculative/medusa/grammatical decoding, diff-representation tokenization, or neural code search.
• Familiarity with compiler/IDE internals (LSP, tree-sitter, control-flow/type graphs) as structured conditioning signals for LLMs.
• Publications at NeurIPS, ICML, ICLR, or PLDI/ICSE, or standout open-source ML/devtools contributions.

## Application Schema & Direct Apply (`apply_for_role`)

Candidates and authorized AI agents on behalf of a candidate can submit an application via the declarative WebMCP form tool `apply_for_role` on https://jobs.talent247.co/jobs/senior-machine-learning-engineer-low-latency-code or via HTTP `POST https://jobs.talent247.co/api/public/jobs/senior-machine-learning-engineer-low-latency-code/apply` (`application/json` or `multipart/form-data`). Non-browser HTTP agents must include header `X-Talent247-Intake-Token` obtained from `GET https://jobs.talent247.co/api/public/intake-token` (or automatically returned in `GET https://jobs.talent247.co/api/public/jobs/senior-machine-learning-engineer-low-latency-code/form-schema`).

### Form Fields

- `full_name` (text, required): Full name
- `email` (email, required): Email
- `phone` (phone, optional): Phone
- `location` (text, optional): Location
- `linkedin_url` (url, optional): LinkedIn URL
- `cover_letter` (textarea, optional): Cover letter
- `consent` (checkbox, required): I consent to my data being processed for this application
- `cv_file` / `cv_base64` + `cv_filename`: Upload a CV in .pdf or .docx format (<= 10 MB) or provide an `https://` LinkedIn profile URL in `linkedin_url`.

### Alternative: Talent Pool Intake (`join_talent_pool`)

If this mandate is not an exact match, register once in the Talent247 Talent Pool at https://jobs.talent247.co/talent-pool (`https://jobs.talent247.co/talent-pool.md`) via WebMCP tool `join_talent_pool` or `POST https://jobs.talent247.co/api/public/talent-pool`.
