2023-03-10 19:40:58 +01:00
|
|
|
*.o
|
|
|
|
*.a
|
2023-06-28 17:53:37 +02:00
|
|
|
*.so
|
2023-08-21 22:07:43 +02:00
|
|
|
*.gguf
|
2023-08-12 01:17:25 +02:00
|
|
|
*.bin
|
2023-08-23 16:31:34 +02:00
|
|
|
*.exe
|
|
|
|
*.dll
|
main : log file (#2748)
* initial, base LOG macro
* add *.log to .gitignore
* added basic log file handler
* reverted log auto endline to better mimic printf
* remove atomics and add dynamic log target
* log_enable/disable, LOG_TEE, basic usage doc
* update .gitignore
* mv include to common, params, help msg
* log tostring helpers, token vectors pretty prints
* main: replaced fprintf/LOG_TEE, some trace logging
* LOG_DISABLE_LOGS compile flag, wrapped f in macros
* fix LOG_TEELN and configchecker
* stub LOG_DUMP_CMDLINE for WIN32 for now
* fix msvc
* cleanup main.cpp:273
* fix stray whitespace after master sync
* log : fix compile warnings
- do not use C++20 stuff
- use PRIu64 to print uint64_t
- avoid string copies by using const ref
- fix ", ##__VA_ARGS__" warnings
- compare strings with == and !=
* log : do not append to existing log + disable file line func by default
* log : try to fix Windows build
* main : wip logs
* main : add trace log
* review: macro f lowercase, str append to sstream
* review: simplify ifs and str comparisons
* fix MSVC, formatting, FMT/VAL placeholders
* review: if/else cleanup
* review: if/else cleanup (2)
* replace _ prefix with _impl suffix
---------
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
2023-08-30 08:29:32 +02:00
|
|
|
*.log
|
2023-09-03 10:48:49 +02:00
|
|
|
*.gcov
|
|
|
|
*.gcno
|
|
|
|
*.gcda
|
|
|
|
*.dot
|
2023-04-19 19:10:08 +02:00
|
|
|
.DS_Store
|
|
|
|
.build/
|
2023-03-10 19:40:58 +01:00
|
|
|
.cache/
|
2023-04-19 19:10:08 +02:00
|
|
|
.direnv/
|
|
|
|
.envrc
|
|
|
|
.swiftpm
|
|
|
|
.venv
|
2023-06-06 08:55:10 +02:00
|
|
|
.clang-tidy
|
2023-03-10 19:40:58 +01:00
|
|
|
.vs/
|
|
|
|
.vscode/
|
|
|
|
|
2023-09-03 10:48:49 +02:00
|
|
|
lcov-report/
|
|
|
|
gcovr-report/
|
|
|
|
|
2023-08-25 11:09:42 +02:00
|
|
|
build*/
|
2023-06-16 08:58:11 +02:00
|
|
|
out/
|
2023-07-18 13:24:43 +02:00
|
|
|
tmp/
|
2023-03-10 19:40:58 +01:00
|
|
|
|
2023-03-11 11:26:46 +01:00
|
|
|
models/*
|
2023-07-18 13:24:43 +02:00
|
|
|
models-mnt
|
2023-03-11 11:26:46 +01:00
|
|
|
|
2023-03-10 19:40:58 +01:00
|
|
|
/main
|
|
|
|
/quantize
|
2023-04-08 00:09:18 +02:00
|
|
|
/quantize-stats
|
2023-03-17 23:03:48 +01:00
|
|
|
/result
|
2023-03-25 19:26:40 +01:00
|
|
|
/perplexity
|
2023-03-28 17:34:35 +02:00
|
|
|
/embedding
|
2023-06-15 19:42:48 +02:00
|
|
|
/train-text-from-scratch
|
2023-08-12 01:17:25 +02:00
|
|
|
/convert-llama2c-to-ggml
|
2023-06-17 18:30:22 +02:00
|
|
|
/simple
|
2023-04-30 14:32:37 +02:00
|
|
|
/benchmark-matmult
|
2023-04-18 22:00:08 +02:00
|
|
|
/vdot
|
Server Example Refactor and Improvements (#1570)
A major rewrite for the server example.
Note that if you have built something on the previous server API, it will probably be incompatible.
Check out the examples for how a typical chat app could work.
This took a lot of effort, there are 24 PR's closed in the submitter's repo alone, over 160 commits and a lot of comments and testing.
Summary of the changes:
- adds missing generation parameters: tfs_z, typical_p, repeat_last_n, repeat_penalty, presence_penalty, frequency_penalty, mirostat, penalize_nl, seed, ignore_eos
- applies missing top k sampler
- removes interactive mode/terminal-like behavior, removes exclude parameter
- moves threads and batch size to server command-line parameters
- adds LoRA loading and matches command line parameters with main example
- fixes stopping on EOS token and with the specified token amount with n_predict
- adds server timeouts, host, and port settings
- adds expanded generation complete response; adds generation settings, stop reason, prompt truncated, model used, and final text
- sets defaults for unspecified parameters between requests
- removes /next-token endpoint and as_loop parameter, adds stream parameter and server-sent events for streaming
- adds CORS headers to responses
- adds request logging, exception printing and optional verbose logging
- adds better stopping words handling when matching multiple tokens and while streaming, or when it finishes on a partial stop string
- adds printing an error when it can't bind to the host/port specified
- fixes multi-byte character handling and replaces invalid UTF-8 characters on responses
- prints timing and build info on startup
- adds logit bias to request parameters
- removes embedding mode
- updates documentation; adds streaming Node.js and Bash examples
- fixes code formatting
- sets server threads to 1 since the current global state doesn't work well with simultaneous requests
- adds truncation of the input prompt and better context reset
- removes token limit from the input prompt
- significantly simplified the logic and removed a lot of variables
---------
Co-authored-by: anon998 <131767832+anon998@users.noreply.github.com>
Co-authored-by: Henri Vasserman <henv@hot.ee>
Co-authored-by: Felix Hellmann <privat@cirk2.de>
Co-authored-by: Johannes Gäßler <johannesg@5d6.de>
Co-authored-by: Lesaun Harvey <Lesaun@gmail.com>
2023-06-17 13:53:04 +02:00
|
|
|
/server
|
2023-03-29 22:51:37 +02:00
|
|
|
/Pipfile
|
2023-06-28 17:53:37 +02:00
|
|
|
/embd-input-test
|
2023-08-21 22:07:43 +02:00
|
|
|
/gguf
|
|
|
|
/gguf-llama-simple
|
2023-06-06 09:18:03 +02:00
|
|
|
/libllama.so
|
2023-08-18 12:44:58 +02:00
|
|
|
/llama-bench
|
2023-08-30 11:52:46 +02:00
|
|
|
/baby-llama
|
|
|
|
/beam-search
|
|
|
|
/save-load-state
|
2023-09-04 12:39:57 +02:00
|
|
|
/speculative
|
2023-05-01 18:23:47 +02:00
|
|
|
build-info.h
|
2023-03-10 19:40:58 +01:00
|
|
|
arm_neon.h
|
|
|
|
compile_commands.json
|
2023-06-16 08:58:11 +02:00
|
|
|
CMakeSettings.json
|
2023-03-17 23:03:48 +01:00
|
|
|
|
2023-03-28 18:39:01 +02:00
|
|
|
__pycache__
|
2023-08-25 08:26:05 +02:00
|
|
|
dist
|
2023-04-05 17:06:02 +02:00
|
|
|
|
|
|
|
zig-out/
|
|
|
|
zig-cache/
|
2023-04-19 19:10:08 +02:00
|
|
|
|
|
|
|
ppl-*.txt
|
2023-05-08 16:41:54 +02:00
|
|
|
qnt-*.txt
|
2023-05-11 23:23:08 +02:00
|
|
|
perf-*.txt
|
2023-04-28 18:13:33 +02:00
|
|
|
|
|
|
|
examples/jeopardy/results.txt
|
2023-07-21 12:09:16 +02:00
|
|
|
|
2023-07-21 12:53:27 +02:00
|
|
|
poetry.lock
|
|
|
|
poetry.toml
|
|
|
|
|
2023-07-21 12:09:16 +02:00
|
|
|
# Test binaries
|
2023-08-13 16:00:48 +02:00
|
|
|
tests/test-grammar-parser
|
2023-08-28 17:38:35 +02:00
|
|
|
tests/test-llama-grammar
|
2023-07-21 12:09:16 +02:00
|
|
|
tests/test-double-float
|
|
|
|
tests/test-grad0
|
|
|
|
tests/test-opt
|
|
|
|
tests/test-quantize-fns
|
|
|
|
tests/test-quantize-perf
|
|
|
|
tests/test-sampling
|
2023-08-28 17:38:35 +02:00
|
|
|
tests/test-tokenizer-0-llama
|
|
|
|
tests/test-tokenizer-0-falcon
|
|
|
|
tests/test-tokenizer-1
|