Radoslav Gerganov 5e31828d3e

ggml : add RPC backend (#6829) ...

* ggml : add RPC backend

The RPC backend proxies all operations to a remote server which runs a
regular backend (CPU, CUDA, Metal, etc).

* set TCP_NODELAY

* add CI workflows

* Address review comments

* fix warning

* implement llama_max_devices() for RPC

* Address review comments

* Address review comments

* wrap sockfd into a struct

* implement get_alignment and get_max_size

* add get_device_memory

* fix warning

* win32 support

* add README

* readme : trim trailing whitespace

* Address review comments

* win32 fix

* Address review comments

* fix compile warnings on macos

2024-05-14 14:27:19 +03:00

..

base64.hpp

llava : expose as a shared library for downstream projects (#3613)

2023-11-07 00:36:23 +03:00

build-info.cpp.in

build : link against build info instead of compiling against it (#3879)

2023-11-02 08:50:16 +02:00

CMakeLists.txt

main: add --json-schema / -j flag (#6659)

2024-04-15 18:35:21 +01:00

common.cpp

ggml : add RPC backend (#6829)

2024-05-14 14:27:19 +03:00

common.h

ggml : add RPC backend (#6829)

2024-05-14 14:27:19 +03:00

console.cpp

check C++ code with -Wmissing-declarations (#3184)

2023-09-15 15:38:27 -04:00

console.h

gguf : new file format with flexible meta data (beta) (#2398)

2023-08-21 23:07:43 +03:00

grammar-parser.cpp

Fix memory bug in grammar parser (#7194)

2024-05-10 21:01:08 +10:00

grammar-parser.h

gguf : new file format with flexible meta data (beta) (#2398)

2023-08-21 23:07:43 +03:00

json-schema-to-grammar.cpp

JSON schema conversion: ⚡️ faster repetitions, min/maxLength for strings, cap number length (#6555)

2024-04-12 19:43:38 +01:00

json-schema-to-grammar.h

JSON: [key] -> .at(key), assert() -> GGML_ASSERT (#7143)

2024-05-08 21:53:08 +02:00

json.hpp

json-schema-to-grammar improvements (+ added to server) (#5978)

2024-03-21 11:50:43 +00:00

log.h

Update LOG_IMPL and LOG_TEE_IMPL (#7029)

2024-05-01 23:31:30 +02:00

ngram-cache.cpp

Fixed lookup compilation issues on Windows (#6273)

2024-03-24 14:21:17 +01:00

ngram-cache.h

lookup: complement data from context with general text statistics (#5479)

2024-03-23 01:24:36 +01:00

sampling.cpp

server: fix reported top tokens for temperature 0 (#7203)

2024-05-11 10:11:28 +02:00

sampling.h

server: fix reported top tokens for temperature 0 (#7203)

2024-05-11 10:11:28 +02:00

stb_image.h

examples: support LLaVA v1.5 (multimodal model) (#3436)

2023-10-12 18:23:18 +03:00

train.cpp

code : normalize enum names (#5697)

2024-02-25 12:09:09 +02:00

train.h

sync : ggml (backend v2) (#3912)

2023-11-13 14:16:23 +02:00