---
title: "Distributed Storage Engineer, Model Weight Streaming"
slug: "distributed-storage-engineer-model-weight"
id: "e9f01dad-dfa5-4a8c-9ebe-a28ab3290fba"
department: "AI Infrastructure"
department_code: "INF"
location: "New York, NY"
workplace_type: "On-site"
employment_type: "Full-time"
compensation: "$225k – $320k (annual package)"
is_remote: false
is_featured: false
headcount: 1
canonical_url: "https://jobs.talent247.co/jobs/distributed-storage-engineer-model-weight"
markdown_url: "https://jobs.talent247.co/jobs/distributed-storage-engineer-model-weight.md"
apply_endpoint: "https://jobs.talent247.co/api/public/jobs/distributed-storage-engineer-model-weight/apply"
schema_endpoint: "https://jobs.talent247.co/api/public/jobs/distributed-storage-engineer-model-weight/form-schema"
webmcp_tool: "apply_for_role"
updated_at: "2026-11-15T17:43:23.856Z"
content_signal: "search=yes, ai-input=yes, ai-train=no"
---

# Distributed Storage Engineer, Model Weight Streaming

## Position Telemetry

- **Reference:** `DISTRIBUTED-STORAG`
- **Practice / Sector:** AI Infrastructure (`INF`)
- **Location:** New York, NY
- **Workplace Type:** On-site
- **Employment Type:** Full-time
- **Compensation:** $225k – $320k (annual package)
- **Canonical HTML URL:** https://jobs.talent247.co/jobs/distributed-storage-engineer-model-weight

## Role Dossier & Specification

THE CLIENT & MANDATE
Our client is a premier New York City serverless GPU and AI compute platform powering instant-scaling inference, batch multimedia processing, and massive reinforcement learning sandboxes. When a customer scales a 400-gigabyte Mixture-of-Experts model from zero to 500 GPU replicas in three seconds, conventional container registries and POSIX network filesystems collapse—making their custom Rust distributed storage engine a foundational competitive moat.

They have partnered with Talent247 to hire a Distributed Storage Engineer for Model Weight Streaming onsite in New York, NY. The successful candidate will co-architect their content-addressable filesystem, tiered NVMe-oF caches, and zero-copy RDMA/PCIe weight-streaming protocols saturating hundreds of gigabits per second per GPU node.

KEY RESPONSIBILITIES
• Design and scale a custom Rust distributed, content-addressable storage and volume engine serving petabytes of model weights, container layers, and training checkpoints across tens of thousands of concurrent serverless workers.
• Engineer ultra-high-throughput tiered caching hierarchies spanning local PCIe Gen5 NVMe arrays, peer-to-peer cluster caches, erasure-coded regional storage nodes, and cloud object stores.
• Implement zero-copy `io_uring`, direct IO (`O_DIRECT`), and GPUDirect Storage (`cuFile` / DMA) streaming paths loading `safetensors` shards from network/NVMe directly into GPU HBM at line rate.
• Build transactional copy-on-write (CoW) volume snapshotting and instant branching primitives enabling million-container RL coding/data sandboxes to fork terabyte datasets in milliseconds.
• Design consensus, metadata sharding, background garbage collection, and bit-rot scrubbing protocols upholding strict durability and consistency guarantees under node churn.
• Benchmark and model tail-latency behavior under extreme thundering-herd fan-out workloads using deterministic simulation testing.

TECHNICAL ENVIRONMENT & OFFERING
Stack: Rust, `io_uring`, FUSE / NFS / custom RPC protocols, NVMe / NVMe-oF, RDMA, erasure coding (Reed-Solomon), Raft/Paxos metadata stores, S3/GCS backends, Linux bare-metal & cloud GPU fleets. Workplace: Onsite in New York, NY. Compensation: $225,000 – $320,000 base salary plus substantial equity ownership, full healthcare, daily meals, and relocation support.

## Core Requirements

ESSENTIAL QUALIFICATIONS
• 5+ years of distributed storage, filesystem, database storage-engine, or high-performance networked IO engineering experience in Rust, C++, or C.
• Deep understanding of Linux storage stacks (`io_uring`, NVMe block layer, page cache vs. `O_DIRECT`, FUSE, filesystem metadata layouts) and networked storage throughput tuning.
• Strong distributed systems foundations across consistent hashing, replication, erasure coding, lease/lock management, and crash recovery.
• Proven ability to reason quantitatively about disk IOPS, network bisection bandwidth, read/write amplification, and thundering-herd cache stampedes.
• Passion for writing clean, deterministic, simulation-tested Rust systems code.

PREFERRED / BONUS EXPERIENCE
• Experience with GPUDirect Storage, RDMA/RoCEv2 data transfers, or P2P content-addressed distribution of massive ML artifacts (`safetensors`, GGUF, OCI layers).
• Prior work on distributed filesystems or object/block stores (Ceph, Lustre, 3FS/Tectonic/JuiceFS-class architectures, FoundationDB, RocksDB).
• Experience with deterministic simulation testing (FoundationDB / `turmoil`-style fault injection) in Rust.

## Application Schema & Direct Apply (`apply_for_role`)

Candidates and authorized AI agents on behalf of a candidate can submit an application via the declarative WebMCP form tool `apply_for_role` on https://jobs.talent247.co/jobs/distributed-storage-engineer-model-weight or via HTTP `POST https://jobs.talent247.co/api/public/jobs/distributed-storage-engineer-model-weight/apply` (`application/json` or `multipart/form-data`). Non-browser HTTP agents must include header `X-Talent247-Intake-Token` obtained from `GET https://jobs.talent247.co/api/public/intake-token` (or automatically returned in `GET https://jobs.talent247.co/api/public/jobs/distributed-storage-engineer-model-weight/form-schema`).

### Form Fields

- `full_name` (text, required): Full name
- `email` (email, required): Email
- `phone` (phone, optional): Phone
- `location` (text, optional): Location
- `linkedin_url` (url, optional): LinkedIn URL
- `cover_letter` (textarea, optional): Cover letter
- `consent` (checkbox, required): I consent to my data being processed for this application
- `cv_file` / `cv_base64` + `cv_filename`: Upload a CV in .pdf or .docx format (<= 10 MB) or provide an `https://` LinkedIn profile URL in `linkedin_url`.

### Alternative: Talent Pool Intake (`join_talent_pool`)

If this mandate is not an exact match, register once in the Talent247 Talent Pool at https://jobs.talent247.co/talent-pool (`https://jobs.talent247.co/talent-pool.md`) via WebMCP tool `join_talent_pool` or `POST https://jobs.talent247.co/api/public/talent-pool`.
