Senior AI Engineer - LLM Adoption Team
rakuten · Tokyo
Job description
About the role
Rakuten's AI & Data Division builds search, recommendation, data science, advertising, marketing, pricing and inventory solutions powered by AI. Its LLM Adoption team drives adoption of Rakuten AI and open-source LLMs hosted in its own environment, embedding AI engineers inside business units to replace third-party alternatives with in-house large language models. As a Senior AI Engineer you own a business unit engagement end to end, working inside their engineering team.
Key responsibilities
- Embed in a business unit's engineering team, design the agent (task decomposition, tool and function calling, multi-step orchestration, state, retrieval, guardrails) and write production code in their codebase.
- Take agents from prototype to production, owning latency, cost, failure modes and monitoring, and deciding early when an agent is the wrong answer.
- Select the right in-house or open-source model per task, proving it with task-level evaluation and building eval sets and harnesses with the business unit.
- Migrate from third-party APIs to in-house models, with parity testing, prompt porting and staged cutover, and feed gaps back to model and platform teams.
- Own prompt and context engineering: structured output, tool schemas, retrieval strategy and caching, versioned and measured against evals.
- Turn work into cookbooks, recipes and reference implementations, and raise the bar through code review, mentorship and hands-on enablement.
Required profile
- Over 6 years of professional software engineering experience, with recent hands-on focus on building LLM applications.
- Proven agentic engineering experience: building, shipping, evaluating and debugging agents in production.
- Prompt and context engineering as a discipline, with structured outputs and systematic iteration against evals.
- Able to work embedded in another engineering team, communicate with business stakeholders and handle undefined requirements.
- Willingness to write documentation, examples and recipes others can use.
Required skills
- Practical model knowledge: strengths and weaknesses of open-weight and commercial models, and when to fine-tune, prompt or retrieve.
- Solid production fundamentals: APIs, backend services, testing, observability, and self-hosted or cloud inference (e.g. vLLM).
- Desired: Japanese, migration from third-party LLM APIs to self-hosted models, retrieval-augmented generation at production scale, exposure to fine-tuning, and experience with distributed teams in Japan or Asia.
Questions fréquentes
Why are you reporting this job?
Explore further
Salaries, guides and searches in Japan.
Salaries by job title
- Manager III, Customer Engineer 17
- Enterprise Account Manager (Public Sales Headquarters) 12
- Construction Surveillance Technician - Okinawa 10
- Customer Engineer II – Field Service 9
- Customer Engineer I – Field Service 9
- Telco/SP Account Sales 9
- Customer Engineer I - Field Service Engineer 9
- Japan Customs Operation Compliance and Logistics Manager (IC) 6
Apply in 30 seconds
Enter your email to apply. An account will be created automatically.
By continuing, you accept our terms of use.
Already have an account? Login
A question about this job?
Ask it here: you will get the full job summary by e-mail, right away.
Published 2 hours ago
Expires 1 month from now
1 views · 0 interested
Boost your chances
Upload your CV — we will match you with relevant openings.
Analyzing your CV...
rakuten
Tokyo
Related job offers
-
Supplier Manager for Field Services
amadeus Tokyo -
Systems & Validation Engineer (Vehicle OS)
applied Tokyo -
Technical Project Manager - Automotive Programs
applied Tokyo -
Programmeur·euse compétitif·ve – Benchmarks IA (Remote)
Vettedlance -
Enregistreur de flux de travail pour outils de développement (éditeurs de code, IDE, clients API)
Vettedlance