Mirrors/llama.cpp
1
0
Fork 0
mirror of https://github.com/ggerganov/llama.cpp.git synced 2024-11-24 09:06:52 +01:00
Code Issues Packages Projects Releases Wiki Activity

Updated Feature matrix (markdown)

Romain D 2024-03-21 17:22:48 +00:00
parent e4f77876f9
commit 3532deb5dc
1 changed files with 4 additions and 3 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

5
Feature-matrix.md

@ -3,7 +3,7 @@
| **K-quants** | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ 🐢⁵ | ✅ 🐢⁵ | 🚫 | | **K-quants** | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ 🐢⁵ | ✅ 🐢⁵ | 🚫 |
| **I-quants** | ✅ 🐢⁴ | ✅ 🐢⁴ | ✅ 🐢⁴ | ✅ | ✅ | Partial¹ | 🚫 | 🚫 | 🚫 | | **I-quants** | ✅ 🐢⁴ | ✅ 🐢⁴ | ✅ 🐢⁴ | ✅ | ✅ | Partial¹ | 🚫 | 🚫 | 🚫 |
| **Multi-GPU** | N/A | N/A | N/A | ✅ | ❓ | 🚫 | ❓ | ✅ | ❓ | | **Multi-GPU** | N/A | N/A | N/A | ✅ | ❓ | 🚫 | ❓ | ✅ | ❓ |
| **K cache quants** | ✅ | ❓ | ❓ | ✅ | Partial³ 🐢 | ❓ | ✅ | 🚫 | 🚫 | | **K cache quants** | ✅ | ❓ | ❓ | ✅ 🐢³ | Partial⁶ 🐢³ | ❓ | ✅ | 🚫 | 🚫 |
| **MoE architecture** | ✅ | ❓ | ✅ | ✅ | ✅ | ❓ | Partial² | 🚫 | 🚫 | | **MoE architecture** | ✅ | ❓ | ✅ | ✅ | ✅ | ❓ | Partial² | 🚫 | 🚫 |
* ✅: feature works * ✅: feature works
@ -12,6 +12,7 @@
* 🐢: feature is slow * 🐢: feature is slow
* ¹: IQ3_S and IQ1_S, see #5886 * ¹: IQ3_S and IQ1_S, see #5886
* ²: Only with `-ngl 0` * ²: Only with `-ngl 0`
* ³: Only `-ctk q8_0`, inference is 50% slower * ³: Inference is 50% slower
* ⁴: Slower than K-quants of comparable size * ⁴: Slower than K-quants of comparable size
* ⁵: Slower than cuBLAS/rocBLAS on similar cards * ⁵: Slower than cuBLAS/rocBLAS on similar cards
* ⁶: Only q8_0 and iq4_nl