---
title: "Senior Sovereign LLM Inference Engineer (CUDA / ROCm)"
slug: "senior-sovereign-llm-inference-engineer-cuda-rocm"
id: "7aefccd3-7690-479c-a501-517cd74307d9"
department: "AI Infrastructure"
department_code: "INF"
location: "Montreal, QC, Canada"
workplace_type: "Hybrid"
employment_type: "Full-time"
compensation: "CA$165k – CA$230k (annual package)"
is_remote: false
is_featured: false
headcount: 1
canonical_url: "https://jobs.talent247.co/jobs/senior-sovereign-llm-inference-engineer-cuda-rocm"
markdown_url: "https://jobs.talent247.co/jobs/senior-sovereign-llm-inference-engineer-cuda-rocm.md"
apply_endpoint: "https://jobs.talent247.co/api/public/jobs/senior-sovereign-llm-inference-engineer-cuda-rocm/apply"
schema_endpoint: "https://jobs.talent247.co/api/public/jobs/senior-sovereign-llm-inference-engineer-cuda-rocm/form-schema"
webmcp_tool: "apply_for_role"
updated_at: "2026-11-15T17:22:01.792Z"
content_signal: "search=yes, ai-input=yes, ai-train=no"
---

# Senior Sovereign LLM Inference Engineer (CUDA / ROCm)

## Position Telemetry

- **Reference:** `SENIOR-SOVEREIGN-L`
- **Practice / Sector:** AI Infrastructure (`INF`)
- **Location:** Montreal, QC, Canada
- **Workplace Type:** Hybrid
- **Employment Type:** Full-time
- **Compensation:** CA$165k – CA$230k (annual package)
- **Canonical HTML URL:** https://jobs.talent247.co/jobs/senior-sovereign-llm-inference-engineer-cuda-rocm

## Role Dossier & Specification

THE CLIENT & MANDATE
Our client is Canada's flagship enterprise foundation model laboratory, operating dual research and infrastructure hubs in Montreal and Toronto alongside offices in London and New York. They build frontier multilingual reasoning models, retrieval stacks, and sovereign AI deployments specifically engineered for regulated enterprises, financial institutions, defence organizations, and national governments that require full data custody inside private cloud VPCs or air-gapped on-premise data centers.

They are working with Talent247 to hire a Senior Sovereign LLM Inference Engineer (CUDA / ROCm) in Montreal. Joining their Inference & Deployment Engineering group, you will own the low-level runtime optimizations and reproducible container packaging that enable massive enterprise LLMs and rerankers to run at peak throughput across heterogeneous NVIDIA and AMD Instinct GPU fleets inside customer-controlled environments.

KEY RESPONSIBILITIES
• Profile, optimize, and productionize high-throughput LLM serving runtimes (vLLM, TensorRT-LLM, SGLang) across both NVIDIA CUDA (H100/H200/B200) and AMD ROCm (MI300X/MI325X) hardware targets.
• Implement and tune low-precision quantization schemes (FP8, INT4/AWQ/GPTQ), speculative decoding, chunked prefill, prefix KV-cache reuse, and constrained structured-output decoding.
• Architect reproducible, hardened, zero-external-dependency Helm and OCI container bundles engineered to deploy cleanly inside air-gapped government and banking Kubernetes clusters.
• Conduct cross-layer tail-latency investigations from HTTP/gRPC request batching and tokenizer throughput down to GPU kernel launch overhead and tensor/pipeline parallelism communication.
• Build automated hardware-matrix benchmark harnesses measuring TTFT, ITL, tokens-per-GPU-hour, and accuracy retention across every model release.
• Partner with forward-deployed architects and enterprise infrastructure teams to size GPU footprints and resolve high-scale production bottlenecks.

TECHNICAL ENVIRONMENT & OFFERING
Their stack spans C++17/20, Python, PyTorch, CUDA, HIP/ROCm, Triton, vLLM, TensorRT-LLM, Docker, and Kubernetes. Offered on a hybrid basis from Montreal, compensation includes CA$165,000–CA$230,000 base salary, competitive equity options, comprehensive Quebec/Canadian benefits, and full visa/relocation support.

## Core Requirements

ESSENTIAL QUALIFICATIONS
• 5+ years of systems, HPC, or machine learning infrastructure engineering focused on GPU acceleration and production model serving.
• Strong proficiency in C++ and Python, paired with hands-on experience profiling and optimizing CUDA, Triton, or AMD ROCm/HIP workloads.
• Direct production experience operating or contributing to modern LLM inference engines such as vLLM, TensorRT-LLM, SGLang, or Triton Inference Server.
• Deep understanding of transformer memory bandwidth bottlenecks, KV-cache management, quantization tradeoffs, and multi-GPU tensor parallelism.
• Solid command of Linux systems internals, Docker/OCI containers, and Kubernetes GPU operator deployments in security-sensitive environments.

PREFERRED / BONUS EXPERIENCE
• Experience deploying enterprise software into air-gapped, VPC-isolated, or government/financial-services environments.
• Upstream open-source contributions to vLLM, SGLang, PyTorch, FlashAttention, or ROCm libraries.

## Application Schema & Direct Apply (`apply_for_role`)

Candidates and authorized AI agents on behalf of a candidate can submit an application via the declarative WebMCP form tool `apply_for_role` on https://jobs.talent247.co/jobs/senior-sovereign-llm-inference-engineer-cuda-rocm or via HTTP `POST https://jobs.talent247.co/api/public/jobs/senior-sovereign-llm-inference-engineer-cuda-rocm/apply` (`application/json` or `multipart/form-data`). Non-browser HTTP agents must include header `X-Talent247-Intake-Token` obtained from `GET https://jobs.talent247.co/api/public/intake-token` (or automatically returned in `GET https://jobs.talent247.co/api/public/jobs/senior-sovereign-llm-inference-engineer-cuda-rocm/form-schema`).

### Form Fields

- `full_name` (text, required): Full name
- `email` (email, required): Email
- `phone` (phone, optional): Phone
- `location` (text, optional): Location
- `linkedin_url` (url, optional): LinkedIn URL
- `cover_letter` (textarea, optional): Cover letter
- `consent` (checkbox, required): I consent to my data being processed for this application
- `cv_file` / `cv_base64` + `cv_filename`: Upload a CV in .pdf or .docx format (<= 10 MB) or provide an `https://` LinkedIn profile URL in `linkedin_url`.

### Alternative: Talent Pool Intake (`join_talent_pool`)

If this mandate is not an exact match, register once in the Talent247 Talent Pool at https://jobs.talent247.co/talent-pool (`https://jobs.talent247.co/talent-pool.md`) via WebMCP tool `join_talent_pool` or `POST https://jobs.talent247.co/api/public/talent-pool`.
