关于

这是一个用Rust编写的朴素贝叶斯垃圾邮件过滤器。也许我会进一步完善它，但它主要是为了自学目的。

资源

研究来源

• https://en.wikipedia.org/wiki/Naive_Bayes_spam_filtering
• https://en.wikipedia.org/wiki/Bag-of-words_model

数据来源

• http://www2.aueb.gr/users/ion/data/enron-spam/

lib.rs:

拉默，是对Rust和垃圾邮件被分类为垃圾邮件或非垃圾邮件的幽默说法，是一个垃圾邮件/非垃圾邮件分类库。

以下是一个示例程序，它训练并保存了一个新的模型以供以后使用。

use rammer::{ HSModel, BagOfWords };

fn main() {
    let spam_bow = BagOfWords::from_folder("data/train/spam").expect("Folder not found");
    let ham_bow = BagOfWords::from_folder("data/train/ham").expect("Folder not found");
    let model = HSModel::from_bows(ham_bow, spam_bow);
    model.write_to_json("out/models/enron1_model.json");
}

以下是一个使用现有模型的示例程序。

use rammer::HSModel;
use std::fs;
use rayon::prelude::*;
fn main() {
   let model = HSModel::read_from_json("out/models/enron1_model.json").unwrap();
   let spam_answers = validate(&model, "data/validate/spam", "spam", |p| p > 0.8);
   let ham_answers = validate(&model, "data/validate/ham", "ham", |p| p < 0.2);

   println!("Spam Correctly Classified: {}/{} = {:.4}", spam_answers.0, spam_answers.1, spam_answers.2);
   println!("Ham Correctly Classified: {}/{} = {:.4}", ham_answers.0, ham_answers.1, ham_answers.2);
}

fn validate<F>(model: &HSModel, dir: &str, class: &str, is_correct: F) -> (u32, usize, f64)
    where F: Fn(f64) -> bool + Sync
{
    let ps: Vec<bool> = fs::read_dir(dir)
        .expect("folder exists")
        .par_bridge()
        .filter_map(|maybe_entry| {
            maybe_entry.ok().and_then(|entry| {
                fs::read_to_string(entry.path())
                    .ok()
                    .and_then(|text| Some(model.text_spam_probability(&text[..])))
            })
        })
        .map(|p| { println!("Probability: {:.8}\t\t({})", p, class); is_correct(p) })
        .collect();

    let num_classified_correctly: u32 = ps
        .iter()
        .filter_map(|&b| if b { Some(1) } else { None })
        .sum();

    (
        num_classified_correctly,
        ps.len(),
        num_classified_correctly as f64 / ps.len() as f64
    )

}