Mirrors/llama.cpp
1
0
Fork 0
mirror of https://github.com/ggerganov/llama.cpp.git synced 2024-12-26 14:20:31 +01:00
Code Issues Packages Projects Releases Wiki Activity
235f6e14bf
llama.cpp/examples/sycl
History
Neo Zhang Jianyu faf67b3de4
[SYCL]set context default value to avoid memory issue, update guide (#9476) 
* set context default to avoid memory issue, update guide

* Update docs/backend/SYCL.md

Co-authored-by: Meng, Hengyu <hengyu.meng@intel.com>

---------

Co-authored-by: arthw <14088817+arthw@users.noreply.github.com>
Co-authored-by: Meng, Hengyu <hengyu.meng@intel.com>
2024-09-18 08:30:31 +08:00
..
build.sh llama : reorganize source code + improve CMake (#8006) 2024-06-26 18:33:02 +03:00
CMakeLists.txt build: rename main → llama-cli, server → llama-server, llava-cli → llama-llava-cli, etc... (#7809) 2024-06-13 00:41:52 +01:00
ls-sycl-device.cpp Support multiple GPUs (split mode) on SYCL backend (#5806) 2024-03-02 19:49:30 +08:00
README.md [SYCL] Updated SYCL device filtering (#8901) 2024-08-07 11:25:36 +01:00
run-llama2.sh [SYCL]set context default value to avoid memory issue, update guide (#9476) 2024-09-18 08:30:31 +08:00
win-build-sycl.bat Removes multiple newlines at the end of files that is breaking the editorconfig step of CI. (#8258) 2024-07-02 12:18:10 -04:00
win-run-llama2.bat [SYCL] correct cmd name (#8877) 2024-08-06 09:09:12 +08:00

README.md

llama.cpp/example/sycl

This example program provides the tools for llama.cpp for SYCL on Intel GPU.

Tool

Tool Name Function Status
llama-ls-sycl-device List all SYCL devices with ID, compute capability, max work group size, ect. Support

llama-ls-sycl-device

List all SYCL devices with ID, compute capability, max work group size, ect.

  1. Build the llama.cpp for SYCL for the specified target (using GGML_SYCL_TARGET).

  2. Enable oneAPI running environment (if GGML_SYCL_TARGET is set to INTEL -default-)

source /opt/intel/oneapi/setvars.sh
  1. Execute
./build/bin/llama-ls-sycl-device

Check the ID in startup log, like:

found 2 SYCL devices:
|  |                   |                                       |       |Max    |        |Max  |Global |                     |
|  |                   |                                       |       |compute|Max work|sub  |mem    |                     |
|ID|        Device Type|                                   Name|Version|units  |group   |group|size   |       Driver version|
|--|-------------------|---------------------------------------|-------|-------|--------|-----|-------|---------------------|
| 0| [level_zero:gpu:0]|                Intel Arc A770 Graphics|    1.3|    512|    1024|   32| 16225M|            1.3.29138|
| 1| [level_zero:gpu:1]|                 Intel UHD Graphics 750|    1.3|     32|     512|   32| 62631M|            1.3.29138|