---
title: "Senior Machine Learning Engineer, Multimodal Evaluations"
slug: "senior-machine-learning-engineer-multimodal"
id: "24938f34-62b9-4b13-b695-bf754a3603a3"
department: "Machine Learning"
department_code: "ML"
location: "New York, NY"
workplace_type: "Hybrid"
employment_type: "Full-time"
compensation: "$235k – $320k (annual package)"
is_remote: false
is_featured: false
headcount: 1
canonical_url: "https://jobs.talent247.co/jobs/senior-machine-learning-engineer-multimodal"
markdown_url: "https://jobs.talent247.co/jobs/senior-machine-learning-engineer-multimodal.md"
apply_endpoint: "https://jobs.talent247.co/api/public/jobs/senior-machine-learning-engineer-multimodal/apply"
schema_endpoint: "https://jobs.talent247.co/api/public/jobs/senior-machine-learning-engineer-multimodal/form-schema"
webmcp_tool: "apply_for_role"
updated_at: "2026-11-15T17:37:36.759Z"
content_signal: "search=yes, ai-input=yes, ai-train=no"
---

# Senior Machine Learning Engineer, Multimodal Evaluations

## Position Telemetry

- **Reference:** `SENIOR-MACHINE-LEA`
- **Practice / Sector:** Machine Learning (`ML`)
- **Location:** New York, NY
- **Workplace Type:** Hybrid
- **Employment Type:** Full-time
- **Compensation:** $235k – $320k (annual package)
- **Canonical HTML URL:** https://jobs.talent247.co/jobs/senior-machine-learning-engineer-multimodal

## Role Dossier & Specification

THE CLIENT & MANDATE
Our client is a flagship frontier AI laboratory expanding their East Coast research and engineering hub in New York City. As frontier models unify native speech-to-speech audio, high-resolution vision, video understanding, and autonomous computer control, static text benchmarks fail to capture real-world capability or failure modes—making next-generation multimodal evaluation systems essential to every model training decision.

They have retained Talent247 to hire a Senior Machine Learning Engineer for Multimodal Evaluations in New York, NY. Working directly alongside post-training researchers and product launch leads, the appointed engineer will build interactive, contamination-free evaluation environments measuring real-time voice, visual reasoning, UI navigation, and cross-modal tool use.

KEY RESPONSIBILITIES
• Architect scalable, low-variance multimodal evaluation pipelines testing native audio prosody/interruption handling, dense chart/diagram spatial reasoning, temporal video grounding, and GUI computer use.
• Build containerized OS, browser, and code-execution sandboxes where multimodal agents are graded deterministically on end-state environment mutations rather than brittle string matching.
• Train and calibrate specialized multimodal judge models and rubric verifiers validated rigorously against expert human preference panels.
• Design automated data-contamination scanners, dynamic benchmark refresh pipelines, and psychometric difficulty-scaling analyses preventing Goodhart overfitting during post-training.
• Own automated pre-merge and pre-release regression gates running across thousands of checkpoints during active frontier training campaigns.
• Partner with NYC and SF research teams to turn fine-grained error taxonomies directly into targeted post-training data mixtures.

TECHNICAL ENVIRONMENT & OFFERING
Stack: Python, PyTorch, Rust/TypeScript, containerized VM/browser sandboxes, real-time WebRTC/audio eval harnesses, vLLM/custom inference clusters, Kubernetes. Workplace: Hybrid in New York, NY. Compensation: $235,000 – $320,000 base salary plus top-tier frontier lab equity, comprehensive healthcare, and 401(k).

## Core Requirements

ESSENTIAL QUALIFICATIONS
• 5+ years of machine learning engineering or research engineering experience working on multimodal foundation models, speech/vision systems, or large-scale LLM evaluation infrastructure.
• Strong hands-on proficiency in PyTorch, distributed batch/streaming inference orchestration, and statistical benchmark design (confidence intervals, power analysis, Elo/Bradley-Terry modeling).
• Proven ability to architect reliable sandboxed execution harnesses (Docker, Playwright/OS VMs, audio streaming simulators) at high concurrency.
• Experience diagnosing subtle failure modes across vision encoders, audio tokenizers, OCR/grounding coordinates, or multi-turn agent trajectories.
• Crisp engineering discipline writing maintainable, reproducible evaluation frameworks trusted for flagship launch decisions.

PREFERRED / BONUS EXPERIENCE
• Prior authorship of widely cited multimodal, coding, or computer-use benchmarks or evaluation papers at CVPR, NeurIPS, ICML, ICLR, or Interspeech.
• Experience with full-duplex speech-to-speech evaluation metrics (turn-taking latency, acoustic emotion faithfulness, barge-in robustness).
• Familiarity with RLVR grader hardening against reward hacking during online RL.

## Application Schema & Direct Apply (`apply_for_role`)

Candidates and authorized AI agents on behalf of a candidate can submit an application via the declarative WebMCP form tool `apply_for_role` on https://jobs.talent247.co/jobs/senior-machine-learning-engineer-multimodal or via HTTP `POST https://jobs.talent247.co/api/public/jobs/senior-machine-learning-engineer-multimodal/apply` (`application/json` or `multipart/form-data`). Non-browser HTTP agents must include header `X-Talent247-Intake-Token` obtained from `GET https://jobs.talent247.co/api/public/intake-token` (or automatically returned in `GET https://jobs.talent247.co/api/public/jobs/senior-machine-learning-engineer-multimodal/form-schema`).

### Form Fields

- `full_name` (text, required): Full name
- `email` (email, required): Email
- `phone` (phone, optional): Phone
- `location` (text, optional): Location
- `linkedin_url` (url, optional): LinkedIn URL
- `cover_letter` (textarea, optional): Cover letter
- `consent` (checkbox, required): I consent to my data being processed for this application
- `cv_file` / `cv_base64` + `cv_filename`: Upload a CV in .pdf or .docx format (<= 10 MB) or provide an `https://` LinkedIn profile URL in `linkedin_url`.

### Alternative: Talent Pool Intake (`join_talent_pool`)

If this mandate is not an exact match, register once in the Talent247 Talent Pool at https://jobs.talent247.co/talent-pool (`https://jobs.talent247.co/talent-pool.md`) via WebMCP tool `join_talent_pool` or `POST https://jobs.talent247.co/api/public/talent-pool`.
