---
title: "Senior Systems Engineer, Ultra-Low-Latency Inference Cloud"
slug: "senior-systems-engineer-ultra-low-latency"
id: "e6f1b328-24f1-41e8-a685-88dd8084a471"
department: "AI Infrastructure"
department_code: "INF"
location: "Austin, TX"
workplace_type: "Hybrid"
employment_type: "Full-time"
compensation: "$185k – $255k (annual package)"
is_remote: false
is_featured: false
headcount: 1
canonical_url: "https://jobs.talent247.co/jobs/senior-systems-engineer-ultra-low-latency"
markdown_url: "https://jobs.talent247.co/jobs/senior-systems-engineer-ultra-low-latency.md"
apply_endpoint: "https://jobs.talent247.co/api/public/jobs/senior-systems-engineer-ultra-low-latency/apply"
schema_endpoint: "https://jobs.talent247.co/api/public/jobs/senior-systems-engineer-ultra-low-latency/form-schema"
webmcp_tool: "apply_for_role"
updated_at: "2026-11-15T17:42:49.733Z"
content_signal: "search=yes, ai-input=yes, ai-train=no"
---

# Senior Systems Engineer, Ultra-Low-Latency Inference Cloud

## Position Telemetry

- **Reference:** `SENIOR-SYSTEMS-ENG`
- **Practice / Sector:** AI Infrastructure (`INF`)
- **Location:** Austin, TX
- **Workplace Type:** Hybrid
- **Employment Type:** Full-time
- **Compensation:** $185k – $255k (annual package)
- **Canonical HTML URL:** https://jobs.talent247.co/jobs/senior-systems-engineer-ultra-low-latency

## Role Dossier & Specification

THE CLIENT & MANDATE
Our client is a breakthrough custom AI silicon and inference-cloud provider operating massive dedicated AI supercomputing clusters across North America, with a flagship systems engineering hub in Austin, TX. Because their custom accelerators generate LLM tokens at 2,000+ tokens per second per stream—an order of magnitude faster than conventional GPU clouds—standard HTTP gateways and Python serving wrappers immediately become the bottleneck.

They have engaged Talent247 to hire a Senior Systems Engineer for their Ultra-Low-Latency Inference Cloud in Austin, TX. The appointed engineer will build ultra-lean Rust/C++/Go streaming data planes, stateful KV-cache-aware request routers, and multi-region failover fabrics capable of sustaining millions of tokens per second with single-digit millisecond gateway overhead.

KEY RESPONSIBILITIES
• Architect high-concurrency, zero-copy Rust and Go token-streaming gateways (HTTP/2, HTTP/3, gRPC, SSE, WebSockets) adding <2ms p99 overhead on 2,000+ tok/s reasoning and coding streams.
• Design distributed prefix-cache-aware and session-affine request schedulers that maximize on-silicon KV-cache hit rates across multi-turn agentic coding and tool-calling workloads.
• Build continuous-batching admission controllers, multi-tier token-bucket rate limiters, and fairness queues balancing interactive low-latency API traffic against massive asynchronous batch/RL rollout jobs.
• Engineer zero-downtime weight-reload orchestration and fast cross-datacenter traffic draining over bare-metal Kubernetes clusters connected directly to custom AI accelerators.
• Instrument nanosecond-to-millisecond distributed tracing (OpenTelemetry, eBPF) tracking time-to-first-token (TTFT), inter-token latency (ITL), and queue wait across every hop.
• Partner closely with compiler and runtime engineers to expose structured outputs, speculative decoding, and reasoning-effort controls cleanly via industry-standard streaming chat-completion APIs.

TECHNICAL ENVIRONMENT & OFFERING
Stack: Rust, Go, C++20, Tokio/io_uring, Envoy, gRPC, Kubernetes (bare-metal), Redis/Dragonfly, ClickHouse, Prometheus/eBPF, Linux high-speed networking. Workplace: Hybrid in Austin, TX. Compensation: $185,000 – $255,000 base salary plus meaningful equity, relocation assistance, and comprehensive US benefits.

## Core Requirements

ESSENTIAL QUALIFICATIONS
• 5+ years of systems, networking, or cloud data-plane engineering experience building ultra-low-latency, high-throughput distributed services in Rust, C++, or Go.
• Deep understanding of Linux networking internals, async event loops (Tokio/epoll/io_uring), HTTP/2 & gRPC framing, backpressure handling, and tail-latency elimination.
• Production experience operating stateful or hardware-attached services on Kubernetes, custom schedulers, or Envoy/service-mesh control planes.
• Familiarity with LLM inference serving mechanics—including continuous batching, prefill vs. decode disaggregation, KV-cache locality, and streaming tokenization.
• Strong operational discipline around load testing, flame-graph profiling, and multi-region high availability.

PREFERRED / BONUS EXPERIENCE
• Direct contributions to LLM serving routers or gateways (vLLM router, SGLang router, Dynamo, Envoy AI Gateway, Pingora).
• Experience with RDMA/RoCEv2 networking, DPDK/XDP kernel-bypass, or custom hardware host-interface drivers.
• Track record scaling developer-facing commercial API clouds under strict 99.99% uptime SLAs.

## Application Schema & Direct Apply (`apply_for_role`)

Candidates and authorized AI agents on behalf of a candidate can submit an application via the declarative WebMCP form tool `apply_for_role` on https://jobs.talent247.co/jobs/senior-systems-engineer-ultra-low-latency or via HTTP `POST https://jobs.talent247.co/api/public/jobs/senior-systems-engineer-ultra-low-latency/apply` (`application/json` or `multipart/form-data`). Non-browser HTTP agents must include header `X-Talent247-Intake-Token` obtained from `GET https://jobs.talent247.co/api/public/intake-token` (or automatically returned in `GET https://jobs.talent247.co/api/public/jobs/senior-systems-engineer-ultra-low-latency/form-schema`).

### Form Fields

- `full_name` (text, required): Full name
- `email` (email, required): Email
- `phone` (phone, optional): Phone
- `location` (text, optional): Location
- `linkedin_url` (url, optional): LinkedIn URL
- `cover_letter` (textarea, optional): Cover letter
- `consent` (checkbox, required): I consent to my data being processed for this application
- `cv_file` / `cv_base64` + `cv_filename`: Upload a CV in .pdf or .docx format (<= 10 MB) or provide an `https://` LinkedIn profile URL in `linkedin_url`.

### Alternative: Talent Pool Intake (`join_talent_pool`)

If this mandate is not an exact match, register once in the Talent247 Talent Pool at https://jobs.talent247.co/talent-pool (`https://jobs.talent247.co/talent-pool.md`) via WebMCP tool `join_talent_pool` or `POST https://jobs.talent247.co/api/public/talent-pool`.
