Mirrors/llama.cpp
1
0
Fork 0
mirror of https://github.com/ggerganov/llama.cpp.git synced 2025-01-15 06:40:45 +01:00
Code Issues Packages Projects Releases Wiki Activity
4,420 Commits 365 Branches 2,926 Tags 322 MiB
5047dd3546
Commit Graph

3 Commits

Author SHA1 Message Date
Xuan Son Nguyen
0da5d86026
server : allow using LoRA adapters per-request (#10994) 
* slot.can_batch_with

* lora per request

* test: force disable cache prompt

* move can_batch_with check

* fix condition

* add slow test with llama 8b

* update docs

* move lora change task to queue

* Apply suggestions from code review

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>

* lora_base

* remove redundant check

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
 2025-01-02 15:05:18 +01:00
Georgi Gerganov
1da7b76569
server : fix speculative decoding with context shift (#10641) 
* server : fix speculative decoding with context shift

ggml-ci

* server : take into account speculative limits

ggml-ci

* server : add tests
 2024-12-04 22:38:20 +02:00
Xuan Son Nguyen
b782e5c7d4
server : add more test cases (#10569) 
* server : add split model test

* add test speculative

* add invalid cases
 2024-11-29 21:48:56 +01:00