From 333b2970fae2c02234b4048255f8689d3e7ad80e Mon Sep 17 00:00:00 2001 From: brian khuu Date: Wed, 12 Jun 2024 00:28:26 +1000 Subject: [PATCH] dev-note: more compact table --- dev-notes.md | 50 ++++++++++++++++++++++++++------------------------ 1 file changed, 26 insertions(+), 24 deletions(-) diff --git a/dev-notes.md b/dev-notes.md index 62c732a..ec95796 100644 --- a/dev-notes.md +++ b/dev-notes.md @@ -3,10 +3,10 @@ These are general free form note with pointers to good jumping to point to under stand the llama.cpp codebase. -(`@` is a vscode jump to symbol code for your convenience. [Also making a feature request to vscode to be able to jump to file and symbol](https://github.com/microsoft/vscode/issues/214870)) +(`@` is a vscode jump to symbol code for your convenience. [Also making a feature request to vscode to be able to jump to file and symbol via `:@`](https://github.com/microsoft/vscode/issues/214870)) -## Where are the definitions? +## Where are the definitions for GGUF in llama.cpp? [GGUF file structure spec (WARN: As of 2024-06-11 the llama.cpp implementation is the canonical source for now)](https://github.com/ggerganov/ggml/blob/master/docs/gguf.md#file-structure) @@ -26,28 +26,30 @@ Please use this as an index not as canonical reference. The purpose of this table is to allow you to quickly locate major elements of the gguf file standard. -| GGUF Elements Of Interest | c name | c type | gguf.c reference | vscode search line | -|-------------------------------------------------------|-------------------------|---------------------------|---------------------------|---------------------| -| Magic | magic | `uint8_t[4]` | `struct gguf_header` | `@gguf_header` | -| Version | version | `uint32_t` | `struct gguf_header` | `@gguf_header` | -| Tensor Count | n_tensors | `uint64_t` | `struct gguf_header` | `@gguf_header` | -| Key Value Count | n_kv | `uint64_t` | `struct gguf_header` | `@gguf_header` | -| Key Value Linked List | kv | `gguf_kv *` | `struct gguf_context` | `@gguf_context` | -| Tensor Info Linked List | infos | `gguf_tensor_info *` | `struct gguf_context` | `@gguf_context` | -| Key Value Entry - Key | gguf_kv.key | `gguf_str` | `struct gguf_kv` | `@gguf_kv` | -| Key Value Entry - Type | gguf_kv.type | `gguf_type` | `struct gguf_kv` | `@gguf_kv` | -| Key Value Entry - Type | gguf_kv.value | `gguf_value` | `struct gguf_kv` | `@gguf_kv` | -| Tensor Info Entry - Name | gguf_tensor_info.name | `gguf_str` | `struct gguf_tensor_info` | `@gguf_tensor_info` | -| Tensor Info Entry - Tensor shape dimension count | gguf_tensor_info.n_dim | `uint32_t` | `struct gguf_tensor_info` | `@gguf_tensor_info` | -| Tensor Info Entry - Tensor shape sizing array | gguf_tensor_info.ne | `uint64_t[GGML_MAX_DIMS]` | `struct gguf_tensor_info` | `@gguf_tensor_info` | -| Tensor Info Entry - Tensor Encoding Scheme / Strategy | gguf_tensor_info.type | `ggml_type` | `struct gguf_tensor_info` | `@gguf_tensor_info` | -| Tensor Info Entry - Offset from start of 'data' | gguf_tensor_info.offset | `uint64_t` | `struct gguf_tensor_info` | `@gguf_tensor_info` | +| Header Name | GGUF Elements Of Interest | c name | c type | vscode search line | +|-------------------|-----------------------------------|-------------------------|---------------------------|----------------------------| +| GGUF Header | Magic | magic | `uint8_t[4]` | gguf.c:`@gguf_header` | +| GGUF Header | Version | version | `uint32_t` | gguf.c:`@gguf_header` | +| GGUF Header | Tensor Count | n_tensors | `uint64_t` | gguf.c:`@gguf_header` | +| GGUF Header | Key Value Count | n_kv | `uint64_t` | gguf.c:`@gguf_header` | +| GGUF Context | Key Value Linked List | kv | `gguf_kv *` | gguf.c:`@gguf_context` | +| GGUF Context | Tensor Info Linked List | infos | `gguf_tensor_info *` | gguf.c:`@gguf_context` | +| Key Value Entry | Key | gguf_kv.key | `gguf_str` | gguf.c:`@gguf_kv` | +| Key Value Entry | Type | gguf_kv.type | `gguf_type` | gguf.c:`@gguf_kv` | +| Key Value Entry | Type | gguf_kv.value | `gguf_value` | gguf.c:`@gguf_kv` | +| Tensor Info Entry | Name | gguf_tensor_info.name | `gguf_str` | gguf.c:`@gguf_tensor_info` | +| Tensor Info Entry | Tensor shape dimension count | gguf_tensor_info.n_dim | `uint32_t` | gguf.c:`@gguf_tensor_info` | +| Tensor Info Entry | Tensor shape sizing array | gguf_tensor_info.ne | `uint64_t[GGML_MAX_DIMS]` | gguf.c:`@gguf_tensor_info` | +| Tensor Info Entry | Tensor Encoding Scheme / Strategy | gguf_tensor_info.type | `ggml_type` | gguf.c:`@gguf_tensor_info` | +| Tensor Info Entry | Offset from start of 'data' | gguf_tensor_info.offset | `uint64_t` | gguf.c:`@gguf_tensor_info` | Also just note that these values are not actually part of gguf but is there for internal usage and is calculated during model loading. -Aka it's for the writing/reading api +Aka it's for the writing/reading api. -| GGML Elements Of Interest | c name | c type | gguf.c reference | vscode search line | -|-------------------------------------------------------|-------------------------|---------------------------|---------------------------|---------------------| -| Alignment | alignment | `size_t` | `struct gguf_context` | `@gguf_context` | -| Offset Of 'Data' From Beginning Of File | offset | `size_t` | `struct gguf_context` | `@gguf_context` | -| Size Of 'Data' In Bytes | size | `size_t` | `struct gguf_context` | `@gguf_context` | +| Header Name | GGML Elements Of Interest | c name | c type | vscode search line | +|-------------------|-----------------------------------------------------|-----------|----------|----------------------------| +| GGUF Context | Alignment | alignment | `size_t` | gguf.c:`@gguf_context` | +| GGUF Context | Offset Of 'Data' From Beginning Of File | offset | `size_t` | gguf.c:`@gguf_context` | +| GGUF Context | Size Of 'Data' In Bytes | size | `size_t` | gguf.c:`@gguf_context` | +| Tensor Info Entry | Tensor memory mapped pointer location in computer | data | `void *` | gguf.c:`@gguf_tensor_info` | +| Tensor Info Entry | Tensor memory mapped size of layer data in computer | size | `size_t` | gguf.c:`@gguf_tensor_info` |