LFM2-8B-A1B
Liquid AI · 2025-10-07 · 8.3B parameters
LFM2-8B-A1B is Liquid AI's early sparse Mixture-of-Experts checkpoint with 8.3 billion reported total parameters and 1.5 billion reported active parameters per token, designed for on-device deployment and tool use. It uses the LFM2 hybrid backbone with 18 gated short-convolution blocks and six GQA blocks; all but the first two layers replace dense feed-forwards with 32-expert, top-4 SwiGLU MoE blocks routed by normalized-sigmoid gating with adaptive biases. Liquid AI reports a 12T-token initial pre-training stage followed by 1T-token long-context mid-training at a 32,768-token context length.