1
0
mirror of https://github.com/ggerganov/llama.cpp.git synced 2025-01-22 17:49:10 +01:00
Commit Graph

1 Commits

Author SHA1 Message Date
DAN™
889bdd7686
command-r : add BPE pre-tokenization ()
* Add BPE pre-tokenization for Command-R/R+.

* Bump transformers convert requirement.

* command-r : add individual digits regex

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
2024-05-05 08:19:30 +03:00