该crate提供了Tokenizer，它按SQLite的方式遍历SQL字符串中的标记。

示例

extern crate sqlite3_tokenizer;

use sqlite3_tokenizer::Tokenizer;

fn main() {
    for token in Tokenizer::new("SELECT * FROM t") {
        println!("Token of kind {:?} is written {:?}", token.kind, token.text);
    }
}

输出

Token of kind Select is written "SELECT"
Token of kind Space is written " "
Token of kind Star is written "*"
Token of kind Space is written " "
Token of kind From is written "FROM"
Token of kind Space is written " "
Token of kind Id is written "t"

lib.rs:

以SQLite的方式对SQL字符串进行标记化。

此功能旨在为SQLite库的实现者提供。

示例

for token in Tokenizer::new("SELECT * FROM t") {
    println!("Token of kind {:?} is written {:?}", token.kind, token.text);
}

输出

Token of kind Select is written "SELECT"
Token of kind Space is written " "
Token of kind Star is written "*"
Token of kind Space is written " "
Token of kind From is written "FROM"
Token of kind Space is written " "
Token of kind Id is written "t"