Skip to main content
TALENT247 // SPECIALIST AI EXECUTIVE SEARCH & TECHNICAL RECRUITMENT
INF · AI Infrastructure

Senior Sovereign LLM Inference Engineer (CUDA / ROCm)

Montreal, QC, Canada · Full-time · Hybrid

AI Specialism

AI Infrastructure

Workplace

Hybrid

Location

Montreal, QC, Canada

Compensation

CA$165k – CA$230k

annual package

Role Dossier & Specification

REF: SENIOR-SOVEREIGN-L

THE CLIENT & MANDATE

Our client is Canada's flagship enterprise foundation model laboratory, operating dual research and infrastructure hubs in Montreal and Toronto alongside offices in London and New York. They build frontier multilingual reasoning models, retrieval stacks, and sovereign AI deployments specifically engineered for regulated enterprises, financial institutions, defence organizations, and national governments that require full data custody inside private cloud VPCs or air-gapped on-premise data centers.

They are working with Talent247 to hire a Senior Sovereign LLM Inference Engineer (CUDA / ROCm) in Montreal. Joining their Inference & Deployment Engineering group, you will own the low-level runtime optimizations and reproducible container packaging that enable massive enterprise LLMs and rerankers to run at peak throughput across heterogeneous NVIDIA and AMD Instinct GPU fleets inside customer-controlled environments.

KEY RESPONSIBILITIES

  • Profile, optimize, and productionize high-throughput LLM serving runtimes (vLLM, TensorRT-LLM, SGLang) across both NVIDIA CUDA (H100/H200/B200) and AMD ROCm (MI300X/MI325X) hardware targets.
  • Implement and tune low-precision quantization schemes (FP8, INT4/AWQ/GPTQ), speculative decoding, chunked prefill, prefix KV-cache reuse, and constrained structured-output decoding.
  • Architect reproducible, hardened, zero-external-dependency Helm and OCI container bundles engineered to deploy cleanly inside air-gapped government and banking Kubernetes clusters.
  • Conduct cross-layer tail-latency investigations from HTTP/gRPC request batching and tokenizer throughput down to GPU kernel launch overhead and tensor/pipeline parallelism communication.
  • Build automated hardware-matrix benchmark harnesses measuring TTFT, ITL, tokens-per-GPU-hour, and accuracy retention across every model release.
  • Partner with forward-deployed architects and enterprise infrastructure teams to size GPU footprints and resolve high-scale production bottlenecks.

TECHNICAL ENVIRONMENT & OFFERING

Their stack spans C++17/20, Python, PyTorch, CUDA, HIP/ROCm, Triton, vLLM, TensorRT-LLM, Docker, and Kubernetes. Offered on a hybrid basis from Montreal, compensation includes CA$165,000–CA$230,000 base salary, competitive equity options, comprehensive Quebec/Canadian benefits, and full visa/relocation support.

Core Requirements

ESSENTIAL QUALIFICATIONS

  • 5+ years of systems, HPC, or machine learning infrastructure engineering focused on GPU acceleration and production model serving.
  • Strong proficiency in C++ and Python, paired with hands-on experience profiling and optimizing CUDA, Triton, or AMD ROCm/HIP workloads.
  • Direct production experience operating or contributing to modern LLM inference engines such as vLLM, TensorRT-LLM, SGLang, or Triton Inference Server.
  • Deep understanding of transformer memory bandwidth bottlenecks, KV-cache management, quantization tradeoffs, and multi-GPU tensor parallelism.
  • Solid command of Linux systems internals, Docker/OCI containers, and Kubernetes GPU operator deployments in security-sensitive environments.

PREFERRED / BONUS EXPERIENCE

  • Experience deploying enterprise software into air-gapped, VPC-isolated, or government/financial-services environments.
  • Upstream open-source contributions to vLLM, SGLang, PyTorch, FlashAttention, or ROCm libraries.

Position Telemetry

Location Montreal, QC, Canada (Hybrid)
Department AI Infrastructure
Employment type Full-time
Workplace type Hybrid
Salary range
CA$165k – CA$230k annual package