rust-brotli

项目需求

Direct no-stdlib port of the C brotli decompressor to Rust

no dependency on the Rust stdlib: this library would be ideal for decompressing within a rust kernel among other things.

This will be useful to see how C and Rust compare in an apples-to-apples
comparison where the same algorithms and data structures and
optimizations are employed.

The current expected performance losses come from
a) an extra indirection in the hgroups
b) array bounds checks on every access
c) no ability to load a full aligned 64 bit or 128 bit item from a [u8]

the system also enables all syscalls to be "frontloaded" in the initial generation
of a memory pool for the allocator. Afterwards, SECCOMP can be activated or
other mechanisms can be used to secure the application, if desired

使用方法

使用 brotli 有 3 个步骤：a) 设置内存管理器 b) 设置 BrotliState c) 在循环中调用 BrotliDecompressStream

详细说明

// at global scope declare a MemPool type -- in this case we'll choose the heap to
// avoid unsafe code, and avoid restrictions of the stack size

declare_stack_allocator_struct!(MemPool, heap);

  // at local scope, make a heap allocated buffers to hold uint8's uint32's and huffman codes
  define_allocator_memory_pool!(u8_buffer, 4096, u8, [0; 32 * 1024 * 1024], heap);
  define_allocator_memory_pool!(u32_buffer, 4096, u32, [0; 1024 * 1024], heap);
  define_allocator_memory_pool!(hc_buffer, 4096, HuffmanCode, [0; 4 * 1024 * 1024], heap);
  let heap_u8_allocator = MemPool::<u8>::new_allocator(u8_buffer, bzero);
  let heap_u32_allocator = MemPool::<u32>::new_allocator(u32_buffer, bzero);
  let heap_hc_allocator = MemPool::<HuffmanCode>::new_allocator(hc_buffer, bzero);


  // At this point no more syscalls are going to be needed since everything can come from the allocators
  // feel free to activate SECCOMP jailing or other mechanisms to secure your application if you wish


  // now it's possible to setup the decompressor state
  let mut brotli_state = BrotliState::new(calloc_u8_allocator, calloc_u32_allocator, calloc_hc_allocator);


  // at this point the decompressor simply needs an input and output buffer and the ability to track
  // the available data left in each buffer
  loop {

      result = BrotliDecompressStream(&mut available_in, &mut input_offset, &input.slice(),
                                      &mut available_out, &mut output_offset, &mut output.slice_mut(),
                                      &mut written, &mut brotli_state);

      // just end the decompression if result is BrotliResult::ResultSuccess or BrotliResult::ResultFailure
  }

此接口与 C 语言 brotli 解压缩器使用的接口相同

也可以自由使用直接调用 Box 的自定义分配器。此示例说明了避免在初始分配之后的后续系统调用的机制