Vicuna 13B v1.1
LMSYS · 2023-04-12 · 13B parameters
Vicuna 13B v1.1 is an autoregressive chat assistant developed by LMSYS through supervised instruction fine-tuning of LLaMA 1 on about 70,000 ShareGPT conversations. Version 1.1 changed the conversation separator from ### to the EOS token and corrected the supervised fine-tuning loss computation; LMSYS released the weights on April 12, 2023 under a non-commercial license.
Benchmark scores
| Benchmark | Score |
|---|---|
| ARC AI2 | 43.2 |
| BBH (BIG-Bench Hard) | 43.0 |
| BoolQ | 83.5 |
| Capability | 90.2 |
| Epoch Capabilities Index (ECI) | 93.2 |
| GSM8K | 28.1 |
| HellaSwag (Unspecified Scoring Protocol) | 57.8 |
| OpenBookQA | 33.0 |
| PIQA | 77.4 |
| WinoGrande | 70.8 |