Skip to main content
TALENT247 // SPECIALIST AI EXECUTIVE SEARCH & TECHNICAL RECRUITMENT
APP · Applied AI

Forward-Deployed AI Engineer, Inference & Fine-Tuning

New York, NY · Full-time · Hybrid

AI Specialism

Applied AI

Workplace

Hybrid

Location

New York, NY

Compensation

$195k – $275k

annual package

Role Dossier & Specification

REF: FORWARD-DEPLOYED-A

THE CLIENT & MANDATE

Our client is a high-growth AI inference and customization cloud headquartered on the San Francisco Peninsula with a flagship East Coast Applied Engineering hub in New York City. Fast-scaling AI startups and Fortune 500 engineering organizations rely on them to migrate massive token-volume workloads from slow, rate-limited proprietary APIs onto blazing-fast, custom post-trained open-weight models.

They have engaged Talent247 to recruit a Forward-Deployed AI Engineer specializing in Inference & Fine-Tuning in New York, NY. Embedded directly with top customer ML and platform engineering teams, the successful candidate will own end-to-end model distillation, reinforcement fine-tuning, and sub-second serving optimization across flagship production deployments.

KEY RESPONSIBILITIES

  • Partner hands-on with customer Staff/Principal ML engineers to benchmark, distill, and fine-tune open-weight dense and MoE models (via SFT, DPO, and GRPO reinforcement fine-tuning) that surpass proprietary frontier baselines on their exact domain tasks.
  • Build golden evaluation harnesses, custom programmatic/rubric graders, and synthetic trajectory curation pipelines over real customer production logs (under strict VPC/zero-retention controls).
  • Co-optimize customer prompt topologies for prefix-cache hit rates, grammar-constrained JSON schemas, parallel tool-calling, and custom EAGLE/ngram speculative decoding configurations.
  • Diagnose end-to-end TTFT and tokens-per-second bottlenecks across multi-region dedicated GPU deployments, tuning tensor/expert parallelism and batching SLAs to traffic shapes.
  • Prototype compound multi-model architectures (fast semantic routers + specialized LoRA adapters + verifier models) cutting customer inference spend by 5x–10x while halving latency.
  • Channel field insights directly into core inference engine, fine-tuning API, and developer SDK features alongside Redwood City product engineers.

TECHNICAL ENVIRONMENT & OFFERING

Stack: Python, TypeScript, PyTorch, SFT/DPO/GRPO fine-tuning APIs & custom recipes, industry-standard chat-completion & native gRPC streaming SDKs, vLLM/SGLang/TensorRT-class serving parameters, Kubernetes, AWS/GCP. Workplace: Hybrid in New York, NY. Compensation: $195,000 – $275,000 base salary plus substantial equity grants, performance incentives, and full US benefits.

Core Requirements

ESSENTIAL QUALIFICATIONS

  • 4+ years of applied ML engineering, ML infrastructure, or forward-deployed AI engineering experience shipping high-throughput LLM systems into production.
  • Hands-on track record fine-tuning open-weight language or multimodal models (dense and sparse Mixture-of-Experts transformer architectures) using LoRA/full-weight SFT, DPO, or RLVR/GRPO.
  • Strong command of LLM inference mechanics—KV-cache sizing, prefix caching, quantization (FP8/FP4), tensor vs. pipeline parallelism, and speculative decoding trade-offs.
  • Production coding fluency in Python and TypeScript/Go, comfortable pairing directly inside customer repositories and evaluation harnesses.
  • High autonomy and crisp technical communication with VPs of Engineering and AI research leads.

PREFERRED / BONUS EXPERIENCE

  • Experience optimizing real-time voice-agent LLM backends, code-completion models, or high-QPS search/reranking pipelines.
  • Prior forward-deployed or applied solutions engineering tenure at an AI lab, inference cloud, or MLOps unicorn.
  • Contributions to open-source fine-tuning, evaluation, or structured-generation libraries.

Position Telemetry

Location New York, NY (Hybrid)
Department Applied AI
Employment type Full-time
Workplace type Hybrid
Salary range
$195k – $275k annual package