Rust GPT4All

本包包含了一组围绕C-API的llmodel Rust绑定。

Crates上的包: - Crates.io: gpt4all

目前仅在 MacOS、Linux (ubuntu) 上进行了测试

先决条件

在Windows和Linux上，构建GPT4All需要完整的Vulkan SDK。您可以从这里下载： https://vulkan.lunarg.com/sdk/home

macOS用户不需要Vulkan，因为GPT4All将使用Metal代替。

安装

安装GPT4All的Rust绑定最简单的方法是使用cargo

cargo add gpt4all

这将从Crates下载最新版本的 gpt4all 包。

本地构建

作为通过cargo下载的替代方案，您也可以从源代码构建Rust绑定。

构建Rust绑定

1. 克隆GPT4All并切换目录

git clone --recurse-submodules https://github.com/nomic-ai/gpt4all.git
cd gpt4all/gpt4all-backend

2. 将本地GPT4All Rust crate添加到项目的Cargo.toml中

[dependencies]
gpt4all = { path = "..path_to_gpt4all../gpt4all/gpt4all-bindings/rust" }

用法

完成

测试一下！在一个Rust脚本中

fn main() {
    // use default model loader
    let model_loader = ModelLoaderBuilder::new().build();

    // load completion model (should already be downloaded and located at default directory 'HOME_DIR/.cache/gpt4all)
    let model = model_loader
        .load_completion_model(NOUS_HERMES_MODEL_FILE, CompletionModelConfig {
            // configure default prompt template for loaded model
            default_prompt_template: Some("<|im_start|>user\n%1<|im_end|>\n<|im_start|>assistant\n%2<|im_end|>\n".to_string()),
        })
        .expect("Failed to load a model");

    // use stateless prompting use case
    let stateless_prompting_builder = StatelessPromptingBuilder::new(&model)
        .system_description("<|im_start|>system\nYou are helpful and kind math teacher.\n<|im_end|>\n")
        .add_reply_expectation("5 + 8, explain for me", "5 + 8 = 13 :)")
        .add_reply_expectation("10 + 8, explain for me", "10 + 8 = 18 :)")
        .add_reply_expectation("1 + 8, explain for me", "1 + 8 = 9 :)");
    
    let stateless_prompting = stateless_prompting_builder.build();

    // ask question
    let answer = stateless_prompting.ask("What is 5 + 5?\n");
    
    println!("{}", answer);
}

嵌入

fn main() {
    // use default model loader
    let model_loader = ModelLoaderBuilder::new().build();

    // load embedding model (should already be downloaded and located at default directory 'HOME_DIR/.cache/gpt4all)
    let embedding_model = model_loader
        .load_embedding_model("nomic-embed-text-v1.f16.gguf")
        .expect("Failed to load a model" );
    
    // configure embedding options
    let file_content = vec!["SOME FILE CONTENT".to_string()];
    let embedding_options = EmbeddingOptionsBuilder::new()
        .do_mean(true)
        .texts(&file_content)
        .build();

    // create embedding from configured options
    let embedding = embedding_model
        .create_embedding(embedding_options)
        .expect("failed to create embedding");
}

开发

如何生成绑定

# Install bindgen if you haven't already
cargo install bindgen

# Generate Rust bindings for llmodel_c.h
bindgen llmodel_c.h -o bindings.rs

发布到crates

• 位于'./scripts'文件夹中的脚本用于为发布到Crates.io准备项目。