CREATURE 特性（化）

一个用于多态 ML & NLP 特征化的 crate，利用零成本抽象。它提供易于使用且裸机快速的可组合 n-gram 组合器。尽管它是为 NLP 而创建的，但它非常通用，可以应用于计算机视觉等多个领域。

有很多 n-gram crate，但大多数都强制进行堆分配或锁定到不适合您的用例或性能需求的具体类型。在大多数基准测试中，creature_feature 的速度比其他任何东西快 4x - 60x。

查看实时演示

这里 是 creature_feature 使用 WASM 的实时演示

特征化的瑞士军刀

use creature_feature::traits::Ftzr;
use creature_feature::ftzrs::{bigram, bislice, for_each, whole};
use creature_feature::HashedAs;
use creature_feature::convert::Bag;
use std::collections::{HashMap, HashSet, BTreeMap, LinkedList, BTreeSet, BinaryHeap, VecDeque};

let int_data = &[1, 2, 3, 4, 5];
let str_data = "one fish two fish red fish blue fish";

// notice how the left-hand side remains almost unchanged.

// we're using 'bislice' right now (which is a 2-gram of referenced data), 'ftzrs::bigram' would yield owned data instead of references

let ref_feats: Vec<&str>                  = bislice().featurize(str_data);
let ref_feats: LinkedList<&[u8]>          = bislice().featurize(str_data);
let ref_bag:   Bag<HashMap<&[usize], u8>> = bislice().featurize(int_data);
let ref_trigram_bag:   Bag<BTreeMap<&str, i16>>   = for_each(whole()).featurize(str_data.split_ascii_whitespace());
let hashed_trigrams: BTreeSet<HashedAs<u64>> = trislice().featurize(int_data);

上面的五个将分别具有以下值。

["on", "ne", "e ", " f", "fi", "is", "sh", "h ", " t", "tw", "wo", "o ", " f", "fi", "is", "sh", "h ", " r", "re", "ed", "d ", " f", "fi", "is", "sh", "h ", " b", "bl", "lu", "ue", "e ", " f", "fi", "is", "sh"]

[[111, 110], [110, 101], [101, 32], [32, 102], [102, 105], [105, 115], [115, 104], [104, 32], [32, 116], [116, 119], [119, 111], [111, 32], [32, 102], [102, 105], [105, 115], [115, 104], [104, 32], [32, 114], [114, 101], [101, 100], [100, 32], [32, 102], [102, 105], [105, 115], [115, 104], [104, 32], [32, 98], [98, 108], [108, 117], [117, 101], [101, 32], [32, 102], [102, 105], [105, 115], [115, 104]]

Bag({[2, 3, 4]: 1, [3, 4, 5]: 1, [1, 2, 3]: 1})

Bag({"blue": 1, "fish": 4, "one": 1, "red": 1, "two": 1})

{HashedAs(3939941806544028562), HashedAs(7191405660579021101), HashedAs(16403185381100005216)}

以下是更多可能的示例

 // let's now switch to 'bigram'
let owned_feats: BTreeSet<[u8; 2]>        = bigram().featurize(str_data);
let owned_feats: Vec<String>              = bigram().featurize(str_data);
let owned_feats: HashSet<Vec<usize>>      = bigram().featurize(int_data);
let owned_bag:   Bag<HashMap<Vec<usize>, u16>>      = bigram().featurize(int_data);

let hashed_feats: BinaryHeap<HashedAs<u32>> = bislice().featurize(str_data);
let hashed_feats: VecDeque<HashedAs<u64>>   =  bigram().featurize(int_data);

let sentence = str_data.split_ascii_whitespace();
let bag_of_words: Bag<HashMap<String, u128>> = for_each(bigram()) .featurize(sentence.clone());
let bag_of_words: Bag<HashMap<&str, u8>>     = for_each(bislice()).featurize(sentence.clone());

 // and many, MANY more posibilities

我们甚至可以在只特征化输入一次的同时产生多个输出

let (set, list): (BTreeSet<HashedAs<u64>>, Vec<&str>) = bislice().featurize_x2(str_data);

creature_feature 提供了三种通用的特征化器类型

1. NGram<const N: usize> 提供了复制的数据上的 n-gram，并生成所有者数据或多个 [T;N]。示例包括 ftzrs::n_gram、ftzrs::bigram 和 ftzrs::trigram。

2. SliceGram 提供了引用数据上的 n-gram，并生成所有者数据或多个 &[T]。示例包括 ftzrs::n_slice、ftzrs::bislice 和 ftzrs::trislice。

3. 由一个或多个特征化器组成并返回具有不同行为的新的特征化器的组合器。例如包括 ftzrs::for_each，ftzrs::gap_gram，featurizers! 和 ftzrs::bookends。

let sentence = "It is a truth universally acknowledged that Jane Austin must be used in nlp examples";

let one:   Vec<String> = trigram().featurize(sentence);
let two:   Vec<[u8;3]> = trigram().featurize(sentence);
let three: Vec<&str>   = trislice().featurize(sentence); // same performance as &[u8]
let four:  Vec<HashedAs<u64>> = trislice().featurize(sentence); // could have used trigram

struct Book {
   author: String,
   genre: Genre,
   sub_genre: SubGenre,
   year: u16,
}

#[derive(Hash)]
enum Genre {
   Fiction,
   NonFiction,
   Religion,
}

#[derive(Hash)]
enum SubGenre {
   Romance,
   History,
   DataScience,
}

impl Book {
   fn decade(&self) -> u8 {
       unimplemented!()
   }
}

use creature_feature::ftzrs::whole;
use creature_feature::traits::*;
use creature_feature::HashedAs;

struct BookFtzr;

impl<'a> Ftzr<&'a Book> for BookFtzr {
   type TokenGroup = HashedAs<u64>;
   fn push_tokens<Push: FnMut(Self::TokenGroup)>(&self, book: &'a Book, push: &mut Push) {
       whole().push_tokens_from(&book.author, push);
       push(FeatureFrom::from(&book.genre));
       push(FeatureFrom::from(&book.sub_genre));
       push(FeatureFrom::from(book.year));
       push(FeatureFrom::from(book.decade()));
   }
}