#async #tracing #metrics #log-file

fast_log

Rust异步日志高性能异步日志

19个稳定版本

1.7.3 2024年7月27日
1.6.16 2024年3月9日
1.6.12 2023年12月28日
1.6.10 2023年10月31日
1.0.3 2020年3月8日

#38调试

Download history 611/week @ 2024-05-01 397/week @ 2024-05-08 334/week @ 2024-05-15 601/week @ 2024-05-22 637/week @ 2024-05-29 738/week @ 2024-06-05 793/week @ 2024-06-12 643/week @ 2024-06-19 689/week @ 2024-06-26 725/week @ 2024-07-03 579/week @ 2024-07-10 502/week @ 2024-07-17 1364/week @ 2024-07-24 865/week @ 2024-07-31 683/week @ 2024-08-07 649/week @ 2024-08-14

3,673 每月下载次数
9 crates 中使用

MIT 协议

63KB
1.5K SLoC

fast_log

Build Status GitHub release

一个用于极高速的日志实现,使用Crossbeam/channel,一次批量写入日志,快速日志日期,Appender架构,每个线程一个Appender

  • 高性能,低开销,日志自动合并,完全APPEND模式文件写入
  • 内置ZIPLZ4压缩
  • 支持使用log::logger().flush()方法等待磁盘刷新
  • 支持自定义文件(impl Trait)
  • 支持滚动日志(ByDateBySizeByDuration)
  • 支持保留日志(AllKeepTimeKeepNum) 删除旧日志,防止日志占用磁盘空间
  • 使用#![forbid(unsafe_code)] 100%安全的Rust。
              -----------------
log data->    | main channel(crossbeam)  |   ->          
              ----------------- 
                                        ----------------                                    ----------------------
                                  ->    |thread channel)|  -> background thread  |    appender1  |
                                        ----------------                                    ----------------------

                                        ----------------                                    ----------------------
                                  ->    |thread channel)|  -> background thread  |    appender2  |
                                        ----------------                                    ----------------------

                                        ----------------                                    ----------------------
                                  ->    |thread channel)|  -> background thread  |    appender3  |
                                        ----------------                                    ----------------------

                                        ----------------                                    ----------------------
                                  ->    |thread channel)|  -> background thread  |    appender4  |
                                        ----------------                                    ----------------------


  • 有多快?

  • 无刷新(chan_len=1000000) benches/log.rs

//MACOS(Apple M1MAX-32GB)
test bench_log ... bench:          85 ns/iter (+/- 1,800)
  • 所有日志刷新到文件(chan_len=1000000) example/bench_test_file.rs
//MACOS(Apple M1MAX-32GB)
test bench_log ... bench:          323 ns/iter (+/- 0)
  • 如何使用?
log = "0.4"
fast_log = {version = "1.7"}

或启用zip/lz4/gzip压缩库

log = "0.4"
# "lz4","zip","gzip"
fast_log = {version = "1.7" , features = ["lz4","zip","gzip"]}

性能优化(重要)

  • 使用chan_len(Some(100000)))预分配通道内存可以减少内存分配的开销,例如
use log::{error, info, warn};
fn  main(){
    fast_log::init(Config::new().file("target/test.log").chan_len(Some(100000))).unwrap();
    log::info!("Commencing yak shaving{}", 0);
}

使用Log(Console)

use log::{error, info, warn};
fn  main(){
    fast_log::init(Config::new().console().chan_len(Some(100000))).unwrap();
    log::info!("Commencing yak shaving{}", 0);
}

使用Log(Console Print)

use log::{error, info, warn};
fn  main(){
    fast_log::init(Config::new().console().chan_len(Some(100000))).unwrap();
    fast_log::print("Commencing print\n".into());
}

使用Log(File)

use fast_log::{init_log};
use log::{error, info, warn};
fn  main(){
    fast_log::init(Config::new().file("target/test.log").chan_len(Some(100000))).unwrap();
    log::info!("Commencing yak shaving{}", 0);
    info!("Commencing yak shaving");
}

分割日志(ByLogDate)

use fast_log::config::Config;
use fast_log::plugin::file_split::{RollingType, KeepType, DateType, Rolling};
use std::thread::sleep;
use std::time::Duration;
use fast_log::plugin::packer::LogPacker;
fn main() {
    fast_log::init(Config::new().chan_len(Some(100000)).console().file_split(
        "target/logs/",
        Rolling::new(RollingType::ByDate(DateType::Day)),
        KeepType::KeepNum(2),
        LogPacker {},
    ))
        .unwrap();
    for _ in 0..60 {
        sleep(Duration::from_secs(1));
        log::info!("Commencing yak shaving");
    }
    log::logger().flush();
    println!("you can see log files in path: {}", "target/logs/")
}

分割日志(ByLogSize)

use fast_log::config::Config;
use fast_log::consts::LogSize;
use fast_log::plugin::file_split::{RollingType, KeepType, Rolling};
use fast_log::plugin::packer::LogPacker;
fn main() {
    fast_log::init(Config::new().chan_len(Some(100000)).console().file_split(
        "target/logs/",
        Rolling::new(RollingType::BySize(LogSize::KB(500))),
        KeepType::KeepNum(2),
        LogPacker {},
    ))
        .unwrap();
    for _ in 0..40000 {
        log::info!("Commencing yak shaving");
    }
    log::logger().flush();
    println!("you can see log files in path: {}", "target/logs/")
}

自定义日志(impl do_log方法)
use fast_log::{LogAppender};
use log::{error, info, warn};

pub struct CustomLog{}
impl LogAppender for CustomLog{
    fn do_log(&mut self, record: &FastLogRecord) {
        print!("{}",record);
    }
}
fn  main(){
    let wait = fast_log::init(Config::new().custom(CustomLog {}).chan_len(Some(100000))).unwrap();
    info!("Commencing yak shaving");
    log::logger().flush();
}

依赖项

~3–11MB
~108K SLoC