Mirrors/llama.cpp
1
0
Fork 0
mirror of https://github.com/ggerganov/llama.cpp.git synced 2024-11-22 08:17:58 +01:00
Code Issues Packages Projects Releases Wiki Activity

dev-note: more compact table

brian khuu 2024-06-12 00:28:26 +10:00
parent 315efe7c92
commit 333b2970fa
1 changed files with 26 additions and 24 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

50
dev-notes.md

@ -3,10 +3,10 @@
These are general free form note with pointers to good jumping to point to under
stand the llama.cpp codebase.
(`@<symbol>` is a vscode jump to symbol code for your convenience. [Also making a feature request to vscode to be able to jump to file and symbol](https://github.com/microsoft/vscode/issues/214870))
(`@<symbol>` is a vscode jump to symbol code for your convenience. [Also making a feature request to vscode to be able to jump to file and symbol via `<file>:@<symbol>`](https://github.com/microsoft/vscode/issues/214870))
## Where are the definitions?
## Where are the definitions for GGUF in llama.cpp?
[GGUF file structure spec (WARN: As of 2024-06-11 the llama.cpp implementation is the canonical source for now)](https://github.com/ggerganov/ggml/blob/master/docs/gguf.md#file-structure)
@ -26,28 +26,30 @@ Please use this as an index not as canonical reference.
The purpose of this table is to allow you to quickly locate major elements of
the gguf file standard.
| GGUF Elements Of Interest | c name | c type | gguf.c reference | vscode search line |
|-------------------------------------------------------|-------------------------|---------------------------|---------------------------|---------------------|
| Magic | magic | `uint8_t[4]` | `struct gguf_header` | `@gguf_header` |
| Version | version | `uint32_t` | `struct gguf_header` | `@gguf_header` |
| Tensor Count | n_tensors | `uint64_t` | `struct gguf_header` | `@gguf_header` |
| Key Value Count | n_kv | `uint64_t` | `struct gguf_header` | `@gguf_header` |
| Key Value Linked List | kv | `gguf_kv *` | `struct gguf_context` | `@gguf_context` |
| Tensor Info Linked List | infos | `gguf_tensor_info *` | `struct gguf_context` | `@gguf_context` |
| Key Value Entry - Key | gguf_kv.key | `gguf_str` | `struct gguf_kv` | `@gguf_kv` |
| Key Value Entry - Type | gguf_kv.type | `gguf_type` | `struct gguf_kv` | `@gguf_kv` |
| Key Value Entry - Type | gguf_kv.value | `gguf_value` | `struct gguf_kv` | `@gguf_kv` |
| Tensor Info Entry - Name | gguf_tensor_info.name | `gguf_str` | `struct gguf_tensor_info` | `@gguf_tensor_info` |
| Tensor Info Entry - Tensor shape dimension count | gguf_tensor_info.n_dim | `uint32_t` | `struct gguf_tensor_info` | `@gguf_tensor_info` |
| Tensor Info Entry - Tensor shape sizing array | gguf_tensor_info.ne | `uint64_t[GGML_MAX_DIMS]` | `struct gguf_tensor_info` | `@gguf_tensor_info` |
| Tensor Info Entry - Tensor Encoding Scheme / Strategy | gguf_tensor_info.type | `ggml_type` | `struct gguf_tensor_info` | `@gguf_tensor_info` |
| Tensor Info Entry - Offset from start of 'data' | gguf_tensor_info.offset | `uint64_t` | `struct gguf_tensor_info` | `@gguf_tensor_info` |
| Header Name | GGUF Elements Of Interest | c name | c type | vscode search line |
|-------------------|-----------------------------------|-------------------------|---------------------------|----------------------------|
| GGUF Header | Magic | magic | `uint8_t[4]` | gguf.c:`@gguf_header` |
| GGUF Header | Version | version | `uint32_t` | gguf.c:`@gguf_header` |
| GGUF Header | Tensor Count | n_tensors | `uint64_t` | gguf.c:`@gguf_header` |
| GGUF Header | Key Value Count | n_kv | `uint64_t` | gguf.c:`@gguf_header` |
| GGUF Context | Key Value Linked List | kv | `gguf_kv *` | gguf.c:`@gguf_context` |
| GGUF Context | Tensor Info Linked List | infos | `gguf_tensor_info *` | gguf.c:`@gguf_context` |
| Key Value Entry | Key | gguf_kv.key | `gguf_str` | gguf.c:`@gguf_kv` |
| Key Value Entry | Type | gguf_kv.type | `gguf_type` | gguf.c:`@gguf_kv` |
| Key Value Entry | Type | gguf_kv.value | `gguf_value` | gguf.c:`@gguf_kv` |
| Tensor Info Entry | Name | gguf_tensor_info.name | `gguf_str` | gguf.c:`@gguf_tensor_info` |
| Tensor Info Entry | Tensor shape dimension count | gguf_tensor_info.n_dim | `uint32_t` | gguf.c:`@gguf_tensor_info` |
| Tensor Info Entry | Tensor shape sizing array | gguf_tensor_info.ne | `uint64_t[GGML_MAX_DIMS]` | gguf.c:`@gguf_tensor_info` |
| Tensor Info Entry | Tensor Encoding Scheme / Strategy | gguf_tensor_info.type | `ggml_type` | gguf.c:`@gguf_tensor_info` |
| Tensor Info Entry | Offset from start of 'data' | gguf_tensor_info.offset | `uint64_t` | gguf.c:`@gguf_tensor_info` |
Also just note that these values are not actually part of gguf but is there for internal usage and is calculated during model loading.
Aka it's for the writing/reading api
Aka it's for the writing/reading api.
| GGML Elements Of Interest | c name | c type | gguf.c reference | vscode search line |
|-------------------------------------------------------|-------------------------|---------------------------|---------------------------|---------------------|
| Alignment | alignment | `size_t` | `struct gguf_context` | `@gguf_context` |
| Offset Of 'Data' From Beginning Of File | offset | `size_t` | `struct gguf_context` | `@gguf_context` |
| Size Of 'Data' In Bytes | size | `size_t` | `struct gguf_context` | `@gguf_context` |
| Header Name | GGML Elements Of Interest | c name | c type | vscode search line |
|-------------------|-----------------------------------------------------|-----------|----------|----------------------------|
| GGUF Context | Alignment | alignment | `size_t` | gguf.c:`@gguf_context` |
| GGUF Context | Offset Of 'Data' From Beginning Of File | offset | `size_t` | gguf.c:`@gguf_context` |
| GGUF Context | Size Of 'Data' In Bytes | size | `size_t` | gguf.c:`@gguf_context` |
| Tensor Info Entry | Tensor memory mapped pointer location in computer | data | `void *` | gguf.c:`@gguf_tensor_info` |
| Tensor Info Entry | Tensor memory mapped size of layer data in computer | size | `size_t` | gguf.c:`@gguf_tensor_info` |