Jobiglo

No results.

Distributed Training & Inference Optimization Engineer (LLM)

rakuten · Tokyo

New
Senior 🇬🇧 English
PyTorch DeepSpeed FSDP Megatron-LM vLLM TensorRT-LLM Triton SGLang CUDA NCCL Kubernetes FlashAttention LoRA

Job description

About the role

Rakuten's AI & Data Division (AIDD) runs the GPU Optimization Department (GPUOD), which manages and optimizes the company-wide AI infrastructure across thousands of accelerators, including Hopper and upcoming Blackwell architectures. As a GPU Training & Inference Optimization Engineer, you will maximize the performance, efficiency and scalability of LLM training and inference workloads on Rakuten's GPU clusters, optimizing training frameworks and inference engines so that AI models run at peak efficiency.

Key responsibilities

  • Optimize LLM training frameworks (PyTorch, DeepSpeed, Megatron-LM, FSDP) to maximize GPU utilization and reduce training time.
  • Profile and fix distributed training bottlenecks such as NCCL issues, CUDA kernel efficiency and communication overhead.
  • Implement and tune inference optimizations (quantization, dynamic batching, KV caching) for low-latency, high-throughput LLM serving (vLLM, TensorRT-LLM, Triton, SGLang).
  • Work with infrastructure teams to improve GPU cluster scheduling, resource allocation and fault tolerance for large-scale training jobs.
  • Develop benchmarking tools to measure training throughput, memory efficiency and inference latency.
  • Research and apply advanced techniques such as mixture-of-experts and speculative decoding.

Required profile

  • 3+ years of hands-on experience in GPU-accelerated ML training and inference optimization, preferably for LLMs or large-scale deep learning models.
  • Bachelor's degree or higher in Computer Science, Engineering or a related field.
  • English at an advanced level.

Required skills

  • Deep expertise in PyTorch, DeepSpeed, FSDP or Megatron-LM, with distributed training optimization experience.
  • Strong knowledge of LLM inference optimizations (quantization, pruning, KV caching, continuous batching).
  • Desired: CUDA, Triton kernels, NVIDIA tools (Nsight, NCCL), performance profiling (PyTorch Profiler, TensorBoard), Kubernetes for GPU workloads, and open-source framework contributions.

Questions fréquentes

Le salaire n'est pas communiqué publiquement par le recruteur. Vous pouvez postuler et négocier directement avec rakuten.
Cliquez sur "Postuler maintenant" en haut de la page. Vous pouvez importer votre CV en 1 clic — Jobiglo extrait automatiquement vos informations et postule pour vous.
Source : ats:workday

Why are you reporting this job?

Thank you for your report. We will review this job.

Apply in 30 seconds

Enter your email to apply. An account will be created automatically.

Apply now →

By continuing, you accept our terms of use.

Already have an account? Login

A question about this job?

Ask it here: you will get the full job summary by e-mail, right away.

Published 2 hours ago

Expires 1 month from now

1 views · 0 interested

Boost your chances

Upload your CV — we will match you with relevant openings.

Analyzing your CV...

rakuten

Tokyo