perlin

A lazy, zero-allocation and data-agnostic Information Retrieval library

功能

• 支持任意类型的布尔检索和
  • 带有过滤器并以懒加载方式评估且无分配的嵌套短语查询
  • 持久化在磁盘上
  • 在RAM中快速

依赖

std

使用方法

extern crate perlin;

use perlin::language::basic_analyzer;
use perlin::storage::RamStorage;
use perlin::index::boolean_index::{IndexBuilder, QueryBuilder};
use perlin::index::Index;

fn main() {
    // The keeper database.
    // Source: "Inverted Files for Text Search Engines" by Justin Zobel and Alistair Moffat, July 2006
    let collection = vec!["The old night keeper keeps the keep in the town",
                          "In the big old house in the big old gown.",
                          "The house in the town had the big old keep",
                          "Where the old night keeper never did sleep.",
                          "The night keeper keeps the keep in the night",
                          "And keeps in the dark and sleeps in the light."];

    // Create the index in RAM
    let index = IndexBuilder::<_, RamStorage<_>>::new()
        .create(collection.iter().map(|doc| basic_analyzer(doc).into_iter()))
        .unwrap();

    // Build simple query for "keeper"
    let keeper_query = QueryBuilder::atom("keeper".to_string()).build();
    assert_eq!(index.execute_query(&keeper_query).collect::<Vec<_>>(),
               vec![0, 3, 4]);
}

更多内容请参阅文档或示例。

当前状态

版本0.1标志着这个库可能对某人有用的第一个状态。尽管如此，当前实现仍存在一些问题

• 索引速度非常慢
• 从损坏的数据加载索引不会产生良好或有用的错误
• RAM中的数据未压缩
• 索引不可变。一旦创建，就无法添加或删除文档

路线图

从长远来看，这个库有望成为一个功能齐全的信息检索库，支持现代排名检索、自然语言处理工具、分面搜索和分类法。