From b1c6785367b8469e28f2ccab7cf14c2d81800fb8 Mon Sep 17 00:00:00 2001 From: Romain D <90720+Artefact2@users.noreply.github.com> Date: Tue, 5 Mar 2024 12:45:31 +0000 Subject: [PATCH] Updated Feature matrix (markdown) --- Feature-matrix.md | 14 ++++++++++---- 1 file changed, 10 insertions(+), 4 deletions(-) diff --git a/Feature-matrix.md b/Feature-matrix.md index ba9b732..29e6fd6 100644 --- a/Feature-matrix.md +++ b/Feature-matrix.md @@ -1,11 +1,17 @@ | | **CPU (AVX2)** | **CPU (ARM NEON)** | **Metal** | **cuBLAS** | **rocBLAS** | **SYCL** | **CLBlast** | **Vulkan** | **Kompute** | |:--------------------:|:--------------:|:------------------:|:---------:|:----------:|:----------------:|:--------:|:-----------:|:----------:|:-----------:| -| **K-quants** | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | 🚫 | -| **I-quants** | ✅ (SLOW) | ✅ (SLOW) | ✅ (SLOW) | ✅ | ✅ | Partial¹ | 🚫 | 🚫 | 🚫 | +| **K-quants** | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ 🐢⁵ | ✅ 🐢⁵ | 🚫 | +| **I-quants** | ✅ 🐢⁴ | ✅ 🐢⁴ | ✅ 🐢⁴ | ✅ | ✅ | Partial¹ | 🚫 | 🚫 | 🚫 | | **Multi-GPU** | N/A | N/A | N/A | ✅ | ❓ | 🚫 | ❓ | ✅ | ❓ | -| **K cache quants** | ✅ | ❓ | ❓ | ✅ | Partial³ (SLOW) | ❓ | ✅ | 🚫 | 🚫 | +| **K cache quants** | ✅ | ❓ | ❓ | ✅ | Partial³ 🐢 | ❓ | ✅ | 🚫 | 🚫 | | **MoE architecture** | ✅ | ❓ | ✅ | ✅ | ✅ | ❓ | Partial² | 🚫 | 🚫 | +* ✅: feature works +* 🚫: feature does not work +* ❓: unknown, please contribute if you can test it youself +* 🐢: feature is slow * ¹: IQ3_S and IQ1_S, see #5886 * ²: Only with `-ngl 0` -* ³: Only `-ctk q8_0` \ No newline at end of file +* ³: Only `-ctk q8_0`, inference is 50% slower +* ⁴: Slower than K-quants of comparable size +* ⁵: Slower than hipBLAS/cuBLAS on similar cards \ No newline at end of file