Kerfuffle 91f6499393
Respect tokenizer.ggml.add_bos_token value when tokenizing (#4040)
* gguf-py: gguf-dump: Respect --no-tensor flag in JSON mode.

* Respect add_bos_token GGUF metadata value

* gguf-py: Try to fix SpecialVocab giving up too easily for the Nth time
2023-11-16 19:14:37 -07:00
..
2023-11-13 14:16:23 +02:00
2023-11-13 14:16:23 +02:00
2023-03-29 20:21:09 +03:00
2023-08-30 09:29:32 +03:00
2023-08-08 14:44:48 +03:00