mirror of
https://github.com/ggerganov/llama.cpp.git
synced 2025-01-28 04:47:04 +01:00
0226613853
* threadpool: skip polling for unused threads Currently all threads do N polling rounds even if only 1 thread is active (n_threads_cur == 1). This commit adds a check to skip the polling for unused threads (ith >= n_threads_cur). n_threads_cur is now an atomic_int to explicitly tell thread sanitizer that it is written from one thread and read from other threads (not a race conditions). * threadpool: further simplify and improve ggml_barrier Avoid using strict memory order while polling, yet make sure that all threads go through full memory barrier (memory fence) on ggml_barrier entrace and exit. * threads: add simple barrier test This test does lots of small, parallel matmul ops where the barriers in between dominate the overhead. * threadpool: improve thread sync for new-graphs Using the same tricks as ggml_barrier. All the polling is done with relaxed memory order to keep it efficient, once the new graph is detected we do full fence using read-modify-write with strict memory order. * threadpool: improve abort handling Do not use threadpool->ec (exit code) to decide whether to exit the compute loop. threadpool->ec is not atomic which makes thread-sanitizer rightfully unhappy about it. Instead introduce atomic threadpool->abort flag used for this. This is consistent with how we handle threadpool->stop or pause. While at it add an explicit atomic_load for n_threads_cur for consistency. * test-barrier: release threadpool before releasing the context fixes use-after-free detected by gcc thread-sanitizer on x86-64 for some reason llvm sanitizer is not detecting this issue. |
||
|---|---|---|
| .. | ||
| ggml-cann | ||
| ggml-cuda | ||
| ggml-sycl | ||
| kompute@4565194ed7 | ||
| kompute-shaders | ||
| llamafile | ||
| vulkan-shaders | ||
| CMakeLists.txt | ||
| ggml-aarch64.c | ||
| ggml-aarch64.h | ||
| ggml-alloc.c | ||
| ggml-backend-impl.h | ||
| ggml-backend.c | ||
| ggml-blas.cpp | ||
| ggml-cann.cpp | ||
| ggml-common.h | ||
| ggml-cpu-impl.h | ||
| ggml-cuda.cu | ||
| ggml-impl.h | ||
| ggml-kompute.cpp | ||
| ggml-metal.m | ||
| ggml-metal.metal | ||
| ggml-quants.c | ||
| ggml-quants.h | ||
| ggml-rpc.cpp | ||
| ggml-sycl.cpp | ||
| ggml-vulkan.cpp | ||
| ggml.c | ||