Skip to main content
TALENT247 // SPECIALIST AI EXECUTIVE SEARCH & TECHNICAL RECRUITMENT
APP · Applied AI

Senior Multimodal Document Intelligence Engineer

New York, NY · Full-time · Hybrid

AI Specialism

Applied AI

Workplace

Hybrid

Location

New York, NY

Compensation

$200k – $285k

annual package

Role Dossier & Specification

REF: SENIOR-MULTIMODAL-

THE CLIENT & MANDATE

Our client is a flagship enterprise AI company serving elite global law firms, private equity sponsors, and corporate legal teams from New York and San Francisco. Every downstream reasoning workflow in their platform depends on lossless ingestion of tens of millions of complex real-world documents—ranging from heavily redacted scanned PDFs and multi-page nested cap-table spreadsheets to handwritten margin annotations and cross-referenced deal exhibits.

They have engaged Talent247 to recruit a Senior Multimodal Document Intelligence Engineer in New York, NY. Sitting at the intersection of vision-language modeling and high-throughput ingestion infrastructure, the successful candidate will build layout-aware VLM parsing pipelines and hierarchical document graphs that preserve every footnote, clause boundary, and tabular cell with near-perfect fidelity.

KEY RESPONSIBILITIES

  • Train, fine-tune, and deploy compact Vision-Language Models (VLMs) and layout-detection transformers specialized for complex multi-column agreements, dense financial tables, redlines/track-changes, and degraded scans.
  • Architect hierarchical structural chunking and cross-reference resolution graphs that link defined terms, schedules, signature pages, and embedded exhibits across multi-thousand-page deal rooms.
  • Engineer GPU-accelerated ingestion pipelines (vLLM / TensorRT-LLM + async Rust/Python workers) capable of parsing tens of thousands of uploaded pages within seconds of data-room ingestion.
  • Build multimodal bounding-box grounding pipelines so every generated legal or financial claim highlights the exact pixel coordinates and table cell in the source document viewer.
  • Construct adversarial document-parsing benchmarks covering rotated scans, multi-page spanning tables, watermarks, multilingual appendices, and complex mathematical formulas.
  • Optimize cost-quality routing between ultra-fast classical OCR/PDF-tree heuristics and heavy VLM visual-token passes on ambiguous pages.

TECHNICAL ENVIRONMENT & OFFERING

Stack: Python, Rust, PyTorch, Vision-Language Models (Qwen-VL/Pixtral-class custom fine-tunes), vLLM, PDFium/Poppler, Apache Arrow/Postgres, Kubernetes. Workplace: Hybrid in New York, NY. Compensation: $200,000 – $285,000 base salary plus significant equity ownership and full US benefits.

Core Requirements

ESSENTIAL QUALIFICATIONS

  • 4+ years of ML engineering experience in multimodal vision-language models, document AI, OCR/layout understanding, or visual information extraction.
  • Proven experience fine-tuning and serving VLMs, multimodal transformers, or layout-aware vision encoders in PyTorch and vLLM/TensorRT.
  • Deep familiarity with PDF internals, coordinate bounding-box alignment, complex table-structure recognition (HTML/Markdown/JSON serialization), and reading-order reconstruction.
  • Strong systems engineering skills to scale batch and real-time GPU document pipelines with tight p95 latency and memory budgets.
  • Disciplined evaluation mindset building cell-level TEDS, edit-distance, and downstream retrieval-recall benchmarks.

PREFERRED / BONUS EXPERIENCE

  • Experience distilling frontier multimodal models into sub-7B vision-language parsers running at hundreds of pages per second per GPU.
  • Background processing financial filings (10-K/S-1s), M&A virtual data rooms, clinical trial PDFs, or patent specifications.
  • Proficiency in Rust or C++ for high-speed PDF rasterization and structural pre-parsing.

Position Telemetry

Location New York, NY (Hybrid)
Department Applied AI
Employment type Full-time
Workplace type Hybrid
Salary range
$200k – $285k annual package