1
0
mirror of https://github.com/ggerganov/llama.cpp.git synced 2025-01-22 17:49:10 +01:00
llama.cpp/models
Galunid daab3d7f45
Add more tokenizer tests ()
* Add more tokenizer tests

* Add starcoder

* Update test vocab files

* Restrict bpe tokenizer tests to unicode planes

* Update comment

* Comment cosmetics

* Remove bloom vocab/test
2023-10-24 09:17:17 +02:00
..
.editorconfig gguf : new file format with flexible meta data (beta) () 2023-08-21 23:07:43 +03:00
ggml-vocab-aquila.gguf Work on the BPE tokenizer () 2023-10-03 09:16:26 +02:00
ggml-vocab-baichuan.gguf Add more tokenizer tests () 2023-10-24 09:17:17 +02:00
ggml-vocab-falcon.gguf Work on the BPE tokenizer () 2023-10-03 09:16:26 +02:00
ggml-vocab-gpt-neox.gguf Add more tokenizer tests () 2023-10-24 09:17:17 +02:00
ggml-vocab-llama.gguf gguf : new file format with flexible meta data (beta) () 2023-08-21 23:07:43 +03:00
ggml-vocab-mpt.gguf Add test for MPT tokenization () 2023-10-22 21:21:42 +02:00
ggml-vocab-refact.gguf Add more tokenizer tests () 2023-10-24 09:17:17 +02:00
ggml-vocab-starcoder.gguf Add more tokenizer tests () 2023-10-24 09:17:17 +02:00