Skip to main content
NEUN
Back to Careers

TRM Labs

Machine Learning Infrastructure Engineer

San Francisco, CAFull-timeGlobal
šŸ“Š MidšŸ  Hybrid

Job Description

[AI-summarized by JobStash]

You will design, build, and operate GPU-backed infrastructure to run production ML and LLM workloads. You will optimize inference systems for throughput and cost, implement model optimization and compilation workflows, and support distributed inference patterns such as model and tensor parallelism. You will schedule heterogeneous workloads across accelerators, instrument systems for GPU load, memory, batching, and token throughput, and work with engineering and ML teams to transition models from experimentation to reliable production services.

Requirements

  • ā—Bachelor's degree or equivalent in Computer Science or related field
  • ā—5+ years of experience building and operating distributed systems or infrastructure in production
  • ā—Experience deploying and operating ML/LLM inference workloads on GPU clusters in cloud environments (AWS and/or GCP)
  • ā—Deep understanding of high-throughput inference systems including batching strategies and token throughput optimization
  • ā—Experience with ML serving frameworks such as Triton Inference Server, vLLM, Ray Serve, ONNX Runtime, or HuggingFace Optimum
  • ā—Experience optimizing GPU load, memory efficiency, and production performance bottlenecks
  • ā—Familiarity with distributed inference strategies including model parallelism and tensor parallelism
  • ā—Experience working with Kubernetes or equivalent orchestration systems
  • ā—Familiarity with heterogeneous accelerators (e.g., Inferentia) is a plus
  • ā—CUDA familiarity and experience debugging GPU-related issues is a plus
  • ā—Adaptable and autonomous with excellent communication and collaboration skills

Responsibilities

  • ā—Design and operate GPU cluster infrastructure
  • ā—Optimize high-throughput inference
  • ā—Enable distributed inference strategies
  • ā—Implement model optimization and compilation workflows
  • ā—Schedule heterogeneous workloads
  • ā—Build observability into ML infrastructure
  • ā—Partner across engineering teams to transition models to production

Tech Stack

autoscalingCUDAparallel computingAWSobservabilityInferenceFlashAttentiontensor parallelismtoken throughputperformance engineering
Expired
Search