Mirrors/llama.cpp
1
0
Fork 0
mirror of https://github.com/ggerganov/llama.cpp.git synced 2024-12-27 22:59:24 +01:00
Code Issues Packages Projects Releases Wiki Activity
3,255 Commits 359 Branches 2,870 Tags 350 MiB
17bb0eaec3
Commit Graph

2 Commits

Author SHA1 Message Date
Georgi Gerganov
9eb5d5617d
convert : add t5 tokenizer tests 2024-07-02 10:39:49 +03:00
DAN™
889bdd7686
command-r : add BPE pre-tokenization (#7063) 
* Add BPE pre-tokenization for Command-R/R+.

* Bump transformers convert requirement.

* command-r : add individual digits regex

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
 2024-05-05 08:19:30 +03:00