Disco Quick

Disco Quick 是一个库，通过结构体的迭代器处理 Discogs 每月 数据转储。它使用 quick-xml 的流式 API，与 discogs-load 解耦，并显著扩展以处理转储中的所有数据。

示例

use disco_quick::DiscogsReader;
use std::env;

fn main() {
    for arg in env::args().skip(1) {
        let reader = match DiscogsReader::from_path(arg.as_ref()) {
            Ok(reader) => reader,
            Err(e) => {
                eprintln!("Error reading {arg}. {e}");
                continue;
            }
        };
        match reader {
            DiscogsReader::Artists(artists) => {
                for artist in artists.take(100) {
                    println!("Artist ID {} is {}", artist.id, artist);
                }
            }
            DiscogsReader::Labels(labels) => {
                for label in labels.take(100) {
                    println!("Label ID {} is {}", label.id, label);
                }
            }
            DiscogsReader::Masters(masters) => {
                for master in masters.take(100) {
                    println!("Master ID {} is {}", master.id, master);
                }
            }
            DiscogsReader::Releases(releases) => {
                for release in releases.take(100) {
                    println!("Release ID {} is {}", release.id, release);
                }
            }
        };
    }
}

性能

在 Ryzen 3900x 处理器上运行 2023-10-01 转储的 examples/count.rs，产生了以下结果

Parsed 8823813 artists in 20.476s (430914.2/s)
Parsed 2044219 labels in 03.186s (641475.25/s)
Parsed 2247215 masters in 23.626s (95113.26/s)
Parsed 16653652 releases in 12m30.206s (22198.756/s)

峰值内存使用量为 5.2MB。