4 Commits

Author	SHA1	Message	Date
Johannes Gäßler	c8771ab5f8	CUDA: fix misaligned shared memory read (#8123)	2024-06-26 08:28:02 +02:00
Johannes Gäßler	9a590c8226	CUDA: optimize MMQ int8 tensor core performance (#8062) ... * CUDA: optimize MMQ int8 tensor core performance * only a single get_mma_tile_x_k function * simplify code, make functions constexpr	2024-06-24 12:41:23 +02:00
Johannes Gäßler	bdcb8f4222	CUDA: int8 tensor cores for MMQ (q4_K, q5_K, q6_K) (#7860)	2024-06-11 08:26:07 +02:00
Johannes Gäßler	1f0dabda8d	CUDA: use tensor cores for MMQ (#7676) ... * CUDA: int8 tensor cores for MMQ (legacy quants) * fix out-of-bounds writes * __builtin_assume -> GGML_CUDA_ASSUME * fix writeback returning too early	2024-06-10 11:45:13 +02:00