AI Infrastructure
New York, NY
On-site · Full-time
$225k – $320k
annual package
Redwood City, CA · Full-time · Hybrid
AI Infrastructure
Hybrid
Redwood City, CA
$225k – $325k
annual package
Our client is a multi-billion-dollar generative AI inference and post-training cloud headquartered in Redwood City, CA, founded by core creators of modern deep-learning frameworks. Serving trillions of daily tokens for top AI-native products and enterprises, their inference engine powers ultra-fast open-weight Mixture-of-Experts (MoE) models alongside thousands of concurrently hot-swapped custom LoRA and RL-tuned adapters on shared GPU pods.
They have retained Talent247 to appoint a Senior LLM Inference Engine Engineer (MoE & LoRA) in Redwood City, CA. Working at the core of their proprietary C++/CUDA/Python serving runtime, the successful candidate will push distributed MoE expert parallelism, disaggregated KV-caching, and multi-LoRA batching to industry-leading throughput-per-dollar frontiers.
Stack: C++20, CUDA, CUTLASS, Triton, Python, PyTorch, custom SGLang/vLLM/TensorRT-LLM-class inference runtime, NVSHMEM/RDMA, H100/H200/B200 GPU clusters. Workplace: Hybrid in Redwood City, CA. Compensation: $225,000 – $325,000 base salary plus high-growth unicorn equity and full US benefits.
xgrammar/outlines).