From 27130d440e9dfc2eae5fd2067bff5bb9b33bc217 Mon Sep 17 00:00:00 2001 From: Romain D <90720+Artefact2@users.noreply.github.com> Date: Tue, 5 Mar 2024 12:36:14 +0000 Subject: [PATCH] Created Feature matrix (markdown) --- Feature-matrix.md | 11 +++++++++++ 1 file changed, 11 insertions(+) create mode 100644 Feature-matrix.md diff --git a/Feature-matrix.md b/Feature-matrix.md new file mode 100644 index 0000000..7e6eb66 --- /dev/null +++ b/Feature-matrix.md @@ -0,0 +1,11 @@ +# llama.cpp feature matrix + +| | **CPU (AVX2)** | **CPU (ARM NEON)** | **Metal** | **cuBLAS** | **rocBLAS** | **SYCL** | **CLBlast** | **Vulkan** | **Kompute** | +|:--------------------:|:--------------:|:------------------:|:---------:|:----------:|:----------------:|:--------:|:-----------:|:----------:|:-----------:| +| **K-quants** | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | 🚫 | +| **I-quants** | ✅ (SLOW) | ✅ (SLOW) | ✅ (SLOW) | ✅ | ✅ | Partial¹ | 🚫 | 🚫 | 🚫 | +| **Multi-GPU** | N/A | N/A | N/A | ✅ | ❓ | 🚫 | ❓ | ✅ | ❓ | +| **K cache quants** | ✅ | ❓ | ❓ | ✅ | Only q8_0 (SLOW) | ❓ | ✅ | 🚫 | 🚫 | +| **MoE architecture** | ✅ | ❓ | ✅ | ✅ | ✅ | ❓ | Only -ngl 0 | 🚫 | 🚫 | + +* ¹: IQ3_S and IQ1_S, see #5886 \ No newline at end of file