swap-pool

你需要存储大量数据，并且可能无法适应你的计算机的RAM吗？那么这个库是为你创建的！

使用说明

use swap_pool::prelude::*;

// Create a new swap pool with 128 bytes of memory available
// and designate a "swap" folder for it
let mut pool = SwapPool::new(128, "swap");

// Spawn 3 entities in the pool
// Each entity must implement From and Into Vec<u8> traits
let a = pool.spawn(vec![0; 128]).unwrap();
let b = pool.spawn(vec![1; 128]).unwrap();
let c = pool.spawn(vec![2; 128]).unwrap();

// Check if spawned entities are hot (stored in the RAM)
// "b" and "c" will be saved to the "swap" folder because
// "a" will take all 128 available bytes (a bit more actually)
dbg!(a.is_hot()); // a.is_hot() = true
dbg!(b.is_hot()); // b.is_hot() = false
dbg!(c.is_hot()); // c.is_hot() = false

// Flush all the entities to the disk ("a" will become cold)
pool.handle().flush().unwrap();

// Read entities values. Note that this operation
// will always clone the value so use x2 amount of RAM
// due to some inner magic
// (I need to share ownership of the value if there's no memory available)
assert!(a.value().unwrap() == vec![0; 128]);
assert!(b.value().unwrap() == vec![1; 128]);
assert!(c.value().unwrap() == vec![2; 128]);

// Check entities status
// Since we can keep only one entity hot at a time
// the pool will free an old one and replace it by a new one
// so firstly we allocated "a", then it was replaced by "b",
// and finally it was replaced by "c"
dbg!(a.is_hot()); // a.is_hot() = false
dbg!(b.is_hot()); // b.is_hot() = false
dbg!(c.is_hot()); // c.is_hot() = true

// Update the value stored in entity "c"
// Second update will return "false" because we can't
// allocate at least 1024 bytes in the current swap pool
// (maximum 128 bytes available)
dbg!(c.update(vec![0; 64]).unwrap());   // c.update(vec![0 ; 64]).unwrap() = true
dbg!(c.update(vec![0; 1024]).unwrap()); // c.update(vec![0 ; 1024]).unwrap() = false

// Show some statistics about the memory use
// Note: "used" will report 0 because second "update"
// has flushed the entity and didn't update its value
// because it didn't have enough free space available
println!("Total: {}", pool.handle().allocated());
println!(" Used: {}", pool.handle().used());
println!(" Free: {}", pool.handle().available());

注意

1. 你可以使用entity.value_allocate()来忽略池内存限制，并始终使实体保持活跃。当你想尽可能长时间地将实体保留在RAM中时，调用此方法。
2. 相反，entity.value_unallocate()将返回存储的值（或从磁盘读取它）并清除实体，使其变为非活跃。当你不需要经常访问实体时，调用此方法。
3. 你可以使用entity.replace(value)替换实体的值。它不会尝试释放内存以存储新值。
4. 你可以通过调用handle.free(memory)来释放所需的任何内存量。它还会说明是否成功释放了给定的内存量。
5. 你也可以调用handle.flush()来刷新所有实体。
6. 调用handle.collect_garbage()以删除对已丢弃实体的弱引用。否则，它们将在池的实体列表中堆积。

实体管理器

实体管理器决定哪些实体应该在其他实体之前刷新。默认情况下，SwapPool将使用SwapLastUsedManager，该管理器保存了最后一次使用实体的时间戳（调用value()或upgrade()方法）。还有一个SwapUpgradesCountManager，它计算升级并将它们用作实体的排名。

您可以实现自己的管理器

use std::collections::HashSet;
use std::cell::Cell;

use swap_pool::prelude::*;

pub struct ExampleManager {
    entities: Cell<HashSet<u64>>
}

impl Default for ExampleManager {
    #[inline]
    fn default() -> Self {
        Self {
            entities: Cell::new(HashSet::new())
        }
    }
}

impl SwapManager for ExampleManager {
    #[inline]
    fn upgrade(&self, uuid: u64) -> u64 {
        let mut entities = self.entities.take();

        // Store the entity's unique id
        entities.insert(uuid);

        self.entities.set(entities);

        self.rank(uuid)
    }

    #[inline]
    fn rank(&self, uuid: u64) -> u64 {
        let entities = self.entities.take();

        // Get stored entity's position
        // and return it as a rank
        // Later an entity was used - later
        // it will be unallocated
        let rank = entities.iter()
            .position(|entity| entity == &uuid)
            .unwrap_or_default();

        self.entities.set(entities);

        u64::try_from(rank).unwrap()
    }
}

实体转换器

转换器用于在读取或写入交换文件时修改实体的值。默认情况下，使用SwapIdentityTransformer，它不对原始数据做任何处理。

让我们创建一个反转输入数据的转换器

use swap_pool::prelude::*;

struct ReverseDataTransformer;

impl SwapTransformer for ReverseDataTransformer {
    // Called to change the value which will be saved to the swap file
    fn forward(&self, data: Vec<u8>) -> Result<Vec<u8>, Box<dyn std::error::Error>> {
        Ok(data.into_iter().rev().collect())
    }

    // Called to change the value read from the swap file
    fn backward(&self, data: Vec<u8>) -> Result<Vec<u8>, Box<dyn std::error::Error>> {
        Ok(data.into_iter().rev().collect())
    }
}

池构建器

use swap_pool::prelude::*;

let mut pool = SwapPoolBuilder::default()
    .with_manager(ExampleManager::default())
    .with_transformer(ReverseDataTransformer)
    .with_thread_safe(false)
    .build();

let entity = pool.spawn(b"Hello, World!".to_vec()).unwrap();

// Hello, World!
println!("{}", String::from_utf8_lossy(&entity.value().unwrap()));

功能

注意

1. 启用random-uuid将禁用timestamp-uuid和基于值生成UUID的使用（结果将仅基于随机生成的数字）。当两者都禁用时，UUID生成将仅基于实体的值。
2. 如果同时启用crc32-uuid和xxhash-uuid，则优先考虑最新者。如果没有启用任何功能，则使用默认的HashMap哈希器。
3. 您不能同时启用size-of-crate和dyn-size-of-crate功能，因为这会导致兼容性问题。考虑只启用其中之一。

作者：Nikita Podvirnyi
许可协议：MIT