4 个版本

0.1.3	2024年2月8日
0.1.2	2024年1月4日
0.1.1	2024年1月4日
0.1.0	2024年1月4日

#199 在机器学习

每月下载量 53 次
在 2 个 crate 中使用 (通过 llm_utils)

MIT 许可证

37KB
628 行

GGUF

这是一个 Rust 项目，提供了解码和使用 GGUF 文件的功能。GGUF 文件是包含键值元数据和张量的二进制文件。

特性

解码 GGUF 文件
从 GGUF 文件中访问键值元数据和张量
支持不同版本的 GGUF 文件：v1、v2、v3

用法

use gguf_rs::get_gguf_container;

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let mut container = get_gguf_container("path_to_your_file")?;
    let model = container.decode()?;
    println!("Model Family: {}", model.model_family());
    println!("Number of Parameters: {}", model.model_parameters());
    println!("File Type: {}", model.file_type());
    println!("Number of Tensors: {}", model.num_tensor());
    Ok(())
}

您还可以将 gguf-rs 作为命令行工具安装。

cargo install gguf-rs

显示模型文件信息

gguf path_to_your_file

Metadata:
+----+-----------------------------------+---------------+
| #  | Key                               | Value         |
+========================================================+
| 1  | general.architecture              | phi2          |
|----+-----------------------------------+---------------|
| 2  | general.file_type                 | 2             |
|----+-----------------------------------+---------------|
| 3  | general.name                      | Phi2          |
|----+-----------------------------------+---------------|
| 4  | general.quantization_version      | 2             |
|----+-----------------------------------+---------------|
| 5  | phi2.attention.head_count         | 32            |
|----+-----------------------------------+---------------|
| 6  | phi2.attention.head_count_kv      | 32            |
|----+-----------------------------------+---------------|
| 7  | phi2.attention.layer_norm_epsilon | 0.00001       |
|----+-----------------------------------+---------------|
| 8  | phi2.block_count                  | 32            |
|----+-----------------------------------+---------------|
| 9  | phi2.context_length               | 2048          |
|----+-----------------------------------+---------------|
| 10 | phi2.embedding_length             | 2560          |
|----+-----------------------------------+---------------|
| 11 | phi2.feed_forward_length          | 10240         |
|----+-----------------------------------+---------------|
| 12 | phi2.rope.dimension_count         | 32            |
|----+-----------------------------------+---------------|
| 13 | tokenizer.ggml.add_bos_token      | false         |
|----+-----------------------------------+---------------|
| 14 | tokenizer.ggml.bos_token_id       | 50256         |
|----+-----------------------------------+---------------|
| 15 | tokenizer.ggml.eos_token_id       | 50256         |
|----+-----------------------------------+---------------|
| 16 | tokenizer.ggml.merges             | [Ġ t,Ġ a,h e] |
|----+-----------------------------------+---------------|
| 17 | tokenizer.ggml.model              | gpt2          |
|----+-----------------------------------+---------------|
| 18 | tokenizer.ggml.token_type         | [1,1,1]       |
|----+-----------------------------------+---------------|
| 19 | tokenizer.ggml.tokens             | [!,",#]       |
|----+-----------------------------------+---------------|
| 20 | tokenizer.ggml.unknown_token_id   | 50256         |
+----+-----------------------------------+---------------+

显示张量

gguf path_to_your_file --tensors

Metadata:
+----+-----------------------------------+---------------+
| #  | Key                               | Value         |
+========================================================+
| 1  | general.architecture              | phi2          |
|----+-----------------------------------+---------------|
| 2  | general.file_type                 | 2             |
|----+-----------------------------------+---------------|
| 3  | general.name                      | Phi2          |
|----+-----------------------------------+---------------|
| 4  | general.quantization_version      | 2             |
|----+-----------------------------------+---------------|
| 5  | phi2.attention.head_count         | 32            |
|----+-----------------------------------+---------------|
| 6  | phi2.attention.head_count_kv      | 32            |
|----+-----------------------------------+---------------|
| 7  | phi2.attention.layer_norm_epsilon | 0.00001       |
|----+-----------------------------------+---------------|
| 8  | phi2.block_count                  | 32            |
|----+-----------------------------------+---------------|
| 9  | phi2.context_length               | 2048          |
|----+-----------------------------------+---------------|
| 10 | phi2.embedding_length             | 2560          |
|----+-----------------------------------+---------------|
| 11 | phi2.feed_forward_length          | 10240         |
|----+-----------------------------------+---------------|
| 12 | phi2.rope.dimension_count         | 32            |
|----+-----------------------------------+---------------|
| 13 | tokenizer.ggml.add_bos_token      | false         |
|----+-----------------------------------+---------------|
| 14 | tokenizer.ggml.bos_token_id       | 50256         |
|----+-----------------------------------+---------------|
| 15 | tokenizer.ggml.eos_token_id       | 50256         |
|----+-----------------------------------+---------------|
| 16 | tokenizer.ggml.merges             | [Ġ t,Ġ a,h e] |
|----+-----------------------------------+---------------|
| 17 | tokenizer.ggml.model              | gpt2          |
|----+-----------------------------------+---------------|
| 18 | tokenizer.ggml.token_type         | [1,1,1]       |
|----+-----------------------------------+---------------|
| 19 | tokenizer.ggml.tokens             | [!,",#]       |
|----+-----------------------------------+---------------|
| 20 | tokenizer.ggml.unknown_token_id   | 50256         |
+----+-----------------------------------+---------------+
Tensors:
+-----+---------------------------+------+----------------+------------+
| #   | Name                      | Type | Dimension      | Offset     |
+======================================================================+
| 1   | token_embd.weight         | Q4_0 | 2560,51200,1,1 | 0          |
|-----+---------------------------+------+----------------+------------|
| 2   | blk.0.attn_norm.bias      | F32  | 2560,1,1,1     | 73728000   |
|-----+---------------------------+------+----------------+------------|
| 3   | blk.0.attn_norm.weight    | F32  | 2560,1,1,1     | 73738240   |
|-----+---------------------------+------+----------------+------------|
| 4   | blk.0.attn_qkv.bias       | F32  | 7680,1,1,1     | 73748480   |
|-----+---------------------------+------+----------------+------------|
| 5   | blk.0.attn_qkv.weight     | Q4_0 | 2560,7680,1,1  | 73779200   |
|-----+---------------------------+------+----------------+------------|
| 6   | blk.0.attn_output.bias    | F32  | 2560,1,1,1     | 84838400   |

测试

该项目包含单元测试。使用 cargo test 运行它们。

贡献

欢迎提交拉取请求。对于重大更改，请先打开一个 issue 来讨论您想要更改的内容。

请确保根据需要更新测试。

GGUF 规范

许可证

MIT

依赖项

~2–13MB
~109K SLoC