Yi-Lightning
01.AI · 2024-10-16
Yi-Lightning is 01.AI's proprietary, API-hosted mixture-of-experts language model, released on October 16, 2024 and documented in a technical report that December. It uses fine-grained expert segmentation and routing, a 3:1 sliding-window/full-attention layer pattern, and cross-layer KV-cache reuse; 01.AI gives only the rounded 100B-parameter size, so no exact total or active count is known. The report specifies a 100,352-token SentencePiece BPE vocabulary and a 64K-token long-context extension, but does not disclose an unambiguous exact integer context limit.