Distributed Training & Inference Optimization Engineer (LLM)
rakuten · Tokyo
Job description
About the role
Rakuten's AI & Data Division (AIDD) runs the GPU Optimization Department (GPUOD), which manages and optimizes the company-wide AI infrastructure across thousands of accelerators, including Hopper and upcoming Blackwell architectures. As a GPU Training & Inference Optimization Engineer, you will maximize the performance, efficiency and scalability of LLM training and inference workloads on Rakuten's GPU clusters, optimizing training frameworks and inference engines so that AI models run at peak efficiency.
Key responsibilities
- Optimize LLM training frameworks (PyTorch, DeepSpeed, Megatron-LM, FSDP) to maximize GPU utilization and reduce training time.
- Profile and fix distributed training bottlenecks such as NCCL issues, CUDA kernel efficiency and communication overhead.
- Implement and tune inference optimizations (quantization, dynamic batching, KV caching) for low-latency, high-throughput LLM serving (vLLM, TensorRT-LLM, Triton, SGLang).
- Work with infrastructure teams to improve GPU cluster scheduling, resource allocation and fault tolerance for large-scale training jobs.
- Develop benchmarking tools to measure training throughput, memory efficiency and inference latency.
- Research and apply advanced techniques such as mixture-of-experts and speculative decoding.
Required profile
- 3+ years of hands-on experience in GPU-accelerated ML training and inference optimization, preferably for LLMs or large-scale deep learning models.
- Bachelor's degree or higher in Computer Science, Engineering or a related field.
- English at an advanced level.
Required skills
- Deep expertise in PyTorch, DeepSpeed, FSDP or Megatron-LM, with distributed training optimization experience.
- Strong knowledge of LLM inference optimizations (quantization, pruning, KV caching, continuous batching).
- Desired: CUDA, Triton kernels, NVIDIA tools (Nsight, NCCL), performance profiling (PyTorch Profiler, TensorBoard), Kubernetes for GPU workloads, and open-source framework contributions.
Questions fréquentes
Why are you reporting this job?
Explore further
Salaries, guides and searches in 日本.
Salaries by job title
- Manager III, Customer Engineer 17
- Enterprise Account Manager (Public Sales Headquarters) 12
- Construction Surveillance Technician - Okinawa 10
- Customer Engineer II – Field Service 9
- Customer Engineer I – Field Service 9
- Telco/SP Account Sales 9
- Customer Engineer I - Field Service Engineer 9
- Japan Customs Operation Compliance and Logistics Manager (IC) 6
Apply in 30 seconds
Enter your email to apply. An account will be created automatically.
By continuing, you accept our terms of use.
Already have an account? Login
Une question sur cette offre ?
Posez-la ici : vous recevrez le récapitulatif de l'offre par e-mail, tout de suite.
Boost your chances
Upload your CV — we will match you with relevant openings.
Analyzing your CV...
rakuten
Tokyo
Related job offers
-
Supplier Manager for Field Services
amadeus Tokyo -
Systems & Validation Engineer (Vehicle OS)
applied Tokyo -
Technical Project Manager - Automotive Programs
applied Tokyo -
Programmeur·euse compétitif·ve – Benchmarks IA (Remote)
Vettedlance -
Enregistreur de flux de travail pour outils de développement (éditeurs de code, IDE, clients API)
Vettedlance