1
0
mirror of https://github.com/ggerganov/llama.cpp.git synced 2025-01-15 14:50:51 +01:00
llama.cpp/ggml/src
Markus Tavenrath 7c5bfd57f8
Optimize Vulkan backend for better CPU performance and less GPU synchronization overhead. ()
* Optimize Vulkan backend for better CPU performance and less GPU synchronization overhead.

- Allocation overhead for the temporary std::vectors was easily detectable with a sampling profiler and simple to remove.
- ggml_vk_sync_buffer introduce a full pipeline sync which has a significant cost on the GPU side, sometimes larger than the actual kernel execution. Adding only barriers for shader read/writes and transfers seems to be sufficient looking at the code which either launches compute kernels or copies tensors.

* Fix small typo

---------

Co-authored-by: 0cc4m <picard12@live.de>
2024-08-11 10:09:09 +02:00
..
ggml-cann ggml : add epsilon as a parameter for group_norm () 2024-08-06 10:26:46 +03:00
ggml-cuda ggml : add epsilon as a parameter for group_norm () 2024-08-06 10:26:46 +03:00
ggml-sycl [SYCL] Updated SYCL device filtering () 2024-08-07 11:25:36 +01:00
kompute@4565194ed7 llama : reorganize source code + improve CMake () 2024-06-26 18:33:02 +03:00
kompute-shaders llama : reorganize source code + improve CMake () 2024-06-26 18:33:02 +03:00
llamafile ggml : move sgemm sources to llamafile subfolder () 2024-07-10 15:23:29 +03:00
vulkan-shaders cmake : Link vulkan-shaders-gen with pthreads () 2024-08-06 15:21:47 +02:00
CMakeLists.txt cann: update cmake () 2024-07-30 12:37:35 +02:00
ggml-aarch64.c ggml : ignore more msvc warnings (ggml/906) 2024-08-08 13:19:31 +03:00
ggml-aarch64.h ggml : minor naming changes () 2024-07-12 10:46:02 +03:00
ggml-alloc.c ggml : reduce hash table reset cost () 2024-07-27 04:41:55 +02:00
ggml-backend-impl.h llama : reorganize source code + improve CMake () 2024-06-26 18:33:02 +03:00
ggml-backend.c ggml-backend : fix async copy from CPU () 2024-08-07 13:29:02 +02:00
ggml-blas.cpp ggml : reduce hash table reset cost () 2024-07-27 04:41:55 +02:00
ggml-cann.cpp [CANN]: Fix ggml_backend_cann_buffer_get_tensor () 2024-08-06 12:42:42 +08:00
ggml-common.h feat: Support Moore Threads GPU () 2024-07-28 01:41:25 +02:00
ggml-cuda.cu ggml-backend : fix async copy from CPU () 2024-08-07 13:29:02 +02:00
ggml-impl.h ggml : reading the runtime sve config of the cpu () 2024-08-03 18:34:41 +02:00
ggml-kompute.cpp ggml : reduce hash table reset cost () 2024-07-27 04:41:55 +02:00
ggml-metal.m metal : fix uninitialized abort_callback () 2024-08-10 15:42:10 +02:00
ggml-metal.metal ggml : fix quant dot product with odd number of blocks () 2024-07-19 17:17:27 +02:00
ggml-quants.c ggml : reading the runtime sve config of the cpu () 2024-08-03 18:34:41 +02:00
ggml-quants.h ggml : reading the runtime sve config of the cpu () 2024-08-03 18:34:41 +02:00
ggml-rpc.cpp Merge commit from fork 2024-08-09 23:03:21 +03:00
ggml-sycl.cpp [SYCL] Add TIMESTEP_EMBEDDING OP () 2024-07-30 14:56:51 +08:00
ggml-vulkan.cpp Optimize Vulkan backend for better CPU performance and less GPU synchronization overhead. () 2024-08-11 10:09:09 +02:00
ggml.c Merge commit from fork 2024-08-09 23:03:21 +03:00