Mirrors/llama.cpp
1
0
Fork 0
mirror of https://github.com/ggerganov/llama.cpp.git synced 2024-12-25 22:08:46 +01:00
Code Issues Packages Projects Releases Wiki Activity
1,416 Commits 359 Branches 2,868 Tags 343 MiB
5be6c803fa
Commit Graph

2 Commits

Author SHA1 Message Date
Kerfuffle
a5e7dbd614
llama : validate special token ids are in range when loading GGUF model (#3635) 
* Add validation for special token ids to llama.cpp

Small optimization for llama_byte_to_token SPM mode

* Fix BPE newline check, only I could break something so simple

* Killll meeeeee

* Account for GGUF_KEY_KEY only setting when the key exists

* Minor code cleanups.

* Fix convert.py error msg when added tokens are out of range

* Make gguf SpecialVocab vocab size-aware

Update conversion scripts accordingly

* Avoid a string copy

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
 2023-10-22 21:14:56 +03:00
Xingchen Song(宋星辰)
02d2875def
llm : add bloom models (#3553) 
* feat: Support bloom models

* fix(bloom): fix model size

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
 2023-10-10 17:48:21 +03:00