Kimi K3: Open Frontier Intelligence

Kimi Team's technical report presents Kimi K3, a 2.8-trillion-parameter native multimodal mixture-of-experts model, covering its hybrid attention architecture, training and agentic post-training recipes, systems infrastructure, evaluations, deployment tradeoffs, and open-weight release.

6 sections ยท 16 lessons

Course outline

Frontier Thesis

  1. Two Scaling Axes
  2. Reading K3 Numbers
  3. Evidence Map

Architecture Mechanics

  1. Hybrid KDA and MLA
  2. Bounded Delta Attention
  3. Attention Residuals
  4. Stable LatentMoE
  5. Choose the Bottleneck Fix

Pre-Training Recipe

  1. Curated Multimodal Data
  2. Scaling Laws and Schedules
  3. Million-Token Curriculum
  4. Audit a Long-Context Recipe

Agentic Post-Training

  1. SFT to RL to MOPD
  2. White-Box Agent Harnesses
  3. Verifiable Task Synthesis
  4. Design an Agent RL Suite

Systems Infrastructure

  1. KDA Systems Co-Design
  2. MoE, RL, and Serving State
  3. Plan a 1M-Context Deployment

Evaluation Judgment

  1. Benchmark Reading
  2. Cost, Cases, and Openness
  3. Make a Deployment Recommendation

Start learning with Wondering