TinKV 是一个用 Rust 编写的简单快速键值存储引擎。受到 basho/bitcask 的启发，在参加 Talent Plan 课程 后编写。

注意:

• 请勿在生产环境中使用。
• 当前的 set/remove/compact 等操作不是线程安全的。

祝您编码愉快~

特性

• 可嵌入（使用 tinkv 作为库）；
• 内置 CLI（《tt class="src-rs">tinkv）；
• 内置兼容 Redis 的服务器；
• 可预测的读写性能。

使用方法

作为库

$ cargo add tinkv

完整的示例用法可以在 examples/basic.rs 中找到。

use tinkv::{self, Store};

fn main() -> tinkv::Result<()> {
    pretty_env_logger::init();
    let mut store = Store::open("/path/to/tinkv")?;
    store.set("hello".as_bytes(), "tinkv".as_bytes())?;

    let value = store.get("hello".as_bytes())?;
    assert_eq!(value, Some("tinkv".as_bytes().to_vec()));

    store.remove("hello".as_bytes())?;

    let value_not_found = store.get("hello".as_bytes())?;
    assert_eq!(value_not_found, None);

    Ok(())
}

使用自定义选项打开

use tinkv::{self, Store};

fn main() -> tinkv::Result<()> {
    let mut store = tinkv::OpenOptions::new()
        .max_data_file_size(1024 * 1024)
        .max_key_size(128)
        .max_value_size(128)
        .sync(true)
        .open(".tinkv")?;
    store.set("hello".as_bytes(), "world".as_bytes())?;
    Ok(())
}

API

tinkv 存储的公共 API 非常易于使用

运行示例

$ RUST_LOG=trace cargo run --example basic

RUST_LOG 级别可以是以下之一：[trace、debug、info、error]。

$ RUST_LOG=info cargo run --example basic

 2020-06-18T10:20:03.497Z INFO  tinkv::store > open store path: .tinkv
 2020-06-18T10:20:04.853Z INFO  tinkv::store > build keydir done, got 100001 keys. current stats: Stats { size_of_stale_entries: 0, total_stale_entries: 0, total_active_entries: 100001, total_data_files: 1, size_of_all_data_files: 10578168 }
200000 keys written in 9.98773 secs, 20024.57 keys/s
initial: Stats { size_of_stale_entries: 21155900, total_stale_entries: 200000, total_active_entries: 100001, total_data_files: 2, size_of_all_data_files: 31733728 }
key_1 => "value_1_1592475604853568000_hello_world"
after set 1: Stats { size_of_stale_entries: 21155900, total_stale_entries: 200000, total_active_entries: 100002, total_data_files: 2, size_of_all_data_files: 31733774 }
after set 2: Stats { size_of_stale_entries: 21155946, total_stale_entries: 200001, total_active_entries: 100002, total_data_files: 2, size_of_all_data_files: 31733822 }
after set 3: Stats { size_of_stale_entries: 21155994, total_stale_entries: 200002, total_active_entries: 100002, total_data_files: 2, size_of_all_data_files: 31733870 }
after remove: Stats { size_of_stale_entries: 21156107, total_stale_entries: 200003, total_active_entries: 100001, total_data_files: 2, size_of_all_data_files: 31733935 }
 2020-06-18T10:20:14.841Z INFO  tinkv::store > compact 2 data files
after compaction: Stats { size_of_stale_entries: 0, total_stale_entries: 0, total_active_entries: 100001, total_data_files: 2, size_of_all_data_files: 10577828 }
key_1 => "value_1_1592475604853568000_hello_world"

命令行界面（CLI）

安装 tinkv 可执行二进制文件。

$ cargo install tinkv

$ tinkv --help
...
USAGE:
    tinkv [FLAGS] <path> <SUBCOMMAND>

FLAGS:
    -h, --help       Prints help information
    -q, --quiet      Pass many times for less log output
    -V, --version    Prints version information
    -v, --verbose    Pass many times for more log output

ARGS:
    <path>    Path to tinkv datastore

SUBCOMMANDS:
    compact    Compact data files in datastore and reclaim disk space
    del        Delete a key value pair from datastore
    get        Retrive value of a key, and display the value
    help       Prints this message or the help of the given subcommand(s)
    keys       List all keys in datastore
    scan       Perform a prefix scanning for keys
    set        Store a key value pair into datastore
    stats      Display statistics of the datastore

示例用法

$ tinkv /tmp/db set hello world
$ tinkv /tmp/db get hello
world

# Change verbosity level (info).
$ tinkv /tmp/db -vvv compact
2020-06-20T10:32:45.582Z INFO  tinkv::store > open store path: tmp/db
2020-06-20T10:32:45.582Z INFO  tinkv::store > build keydir from data file /tmp/db/000000000001.tinkv.data
2020-06-20T10:32:45.583Z INFO  tinkv::store > build keydir from data file /tmp/db/000000000002.tinkv.data
2020-06-20T10:32:45.583Z INFO  tinkv::store > build keydir done, got 1 keys. current stats: Stats { size_of_stale_entries:0, total_stale_entries: 0, total_active_entries: 1,total_data_files: 2, size_of_all_data_files: 60 }
2020-06-20T10:32:45.583Z INFO  tinkv::store > there are 3 datafiles need to be compacted

客户端 & 服务器

tinkv-server 是一个与 Redis 兼容的键/值存储服务器。然而，并非所有 Redis 命令都受支持。可用的命令有

• get<key>
• mget<key> [<key>...]
• set<key> <value>
• mset<key> <value> [<key> <value>]
• del<key>
• keys<pattern>
• ping[<message>]
• exists<key>
• info[<section>]
• command
• dbsize
• compact：手动触发压缩的扩展命令。

键/值对持久保存在目录 /usr/local/var/tinkv 下的日志文件中。服务器的默认监听地址为 127.0.0.1:7379，您可以使用 Redis 客户端连接到它。

快速入门

安装 tinkv-server 非常简单

$ cargo install tinkv

使用默认配置启动服务器（将日志级别设置为 info 模式）

$ tinkv-server -vv
2020-06-24T13:46:49.341Z INFO  tinkv::store > open store path: /usr/local/var/tinkv
2020-06-24T13:46:49.343Z INFO  tinkv::store > build keydir from data file /usr/local/var/tinkv/000000000001.tinkv.data
2020-06-24T13:46:49.343Z INFO  tinkv::store > build keydir from data file /usr/local/var/tinkv/000000000002.tinkv.data
2020-06-24T13:46:49.343Z INFO  tinkv::store > build keydir done, got 0 keys. current stats: Stats { size_of_stale_entries: 0,total_stale_entries: 0, total_active_entries: 0, total_data_files: 2, size_of_all_data_files: 0 }
2020-06-24T13:46:49.343Z INFO  tinkv::server > TinKV server is listening at '127.0.0.1:7379'

使用 redis-cli 与 tinkv-server 通信

$ redis-cli -p 7379
127.0.0.1:7379> ping
PONG
127.0.0.1:7379> ping "hello, tinkv"
"hello, tinkv"
127.0.0.1:7379> set name tinkv
OK
127.0.0.1:7379> exists name
(integer) 1
127.0.0.1:7379> get name tinkv
(error) ERR wrong number of arguments for 'get' command
127.0.0.1:7379> get name
"tinkv"
127.0.0.1:7379> command
1) "ping"
2) "get"
3) "set"
4) "del"
5) "dbsize"
6) "exists"
7) "compact"
8) "info"
9) "command"
...and more
127.0.0.1:7379> info
# Server
tinkv_version: 0.9.0
os: Mac OS, 10.15.4, 64-bit

# Stats
size_of_stale_entries: 143
size_of_stale_entries_human: 143 B
total_stale_entries: 3
total_active_entries: 1109
total_data_files: 5
size_of_all_data_files: 46813
size_of_all_data_files_human: 46.81 KB
127.0.0.1:7379> notfound
(error) ERR unknown command `notfound`
127.0.0.1:7379>

关于压缩

在每次调用 set 或 remove 后，如果 size_of_stale_entries >= config::COMPACTION_THRESHOLD，将触发压缩过程。压缩步骤非常简单且易于理解

use tinkv::{self, Store};

fn main() -> tinkv::Result<()> {
    pretty_env_logger::init();
    let mut store = Store::open("/path/to/tinkv")?;
    store.compact()?;

    Ok(())
}

.tinkv
├── 000000000001.tinkv.hint -- related index/hint file, for fast startup
├── 000000000001.tinkv.data -- immutable data file
└── 000000000002.tinkv.data -- active data file