datafrost

数据格式和加速结构管理

datafrost 是一个面向数据的资源管理和调度库。它实现了一个受图形API启发的接口，允许用户干净且高效地

• 创建主要数据对象，并定义“派生”数据类型，其内容由主要格式生成。
• 跟踪主要对象的变化，并自动更新派生格式的受影响部分。
• 映射数据对象的内容，并在主线程上读取它们。
• 调度命令以异步和并发地读取或修改数据对象。
  • datafrost 通过构建表示挂起操作的有向无环图来保证最优调度。
  • 引用不同数据或不可变引用相同数据的多个命令将并行执行。
  • 访问相同数据的可变命令将按顺序执行，没有数据竞争的可能性。

用法

以下是如何使用 datafrost 的简略示例。完整代码可以在示例文件夹中找到。首先，我们定义代码将使用的数据格式

use datafrost::*;
use std::ops::*;

/// First, we define a general "kind" of data that our program will use.
/// In this case, let's imagine that we want to efficiently deal with
/// arrays of numbers.
pub struct NumberArray;

/// Defines the layout of an array of numbers.
pub struct NumberArrayDescriptor {
    /// The length of the array.
    pub len: usize
}

impl Kind for NumberArray { .. }

/// Next, we define the primary data format that we would like
/// to use and modify - an array of specifically `u32`s.
pub struct PrimaryArray(Vec<u32>);

impl Format for PrimaryArray { .. }

/// Now, let's imagine that we want to efficiently maintain an
/// acceleration structure containing all of the numbers in
/// the array, but doubled. So, we define the format.
pub struct DoubledArray(Vec<u32>);

impl Format for DoubledArray { .. }

/// Our goal is for `datafrost` to automatically update the doubled
/// array whenever the primary array changes. Thus, we implement
/// a way for it do so.
pub struct DoublePrimaryArray;

impl DerivedDescriptor<PrimaryArray> for DoublePrimaryArray {
    type Format = DoubledArray;

    fn update(&self, data: &mut DoubledArray, parent: &PrimaryArray, usages: &[&Range<usize>]) {
        // Loop over all ranges of the array that have changed, and
        // for each value in the range, recompute the data.
        for range in usages.iter().copied() {
            for i in range.clone() {
                data.0[i] = 2 * parent.0[i];
            }
        }
    }
}

现在我们的数据和其派生格式已定义，我们可以创建其实例，并调度命令来处理数据

// Create a new context.
let ctx = DataFrostContext::new(ContextDescriptor {
    label: Some("my context")
});

// Allocate a new primary array object, which has a doubled
// array as a derived format.
let data = ctx.allocate::<PrimaryArray>(AllocationDescriptor {
    descriptor: NumberArrayDescriptor { len: 7 },
    label: Some("my data"),
    derived_formats: &[&Derived::new(DoublePrimaryArray)]
});

// Create a command buffer to record operations to execute
// on our data.
let mut command_buffer = CommandBuffer::new(CommandBufferDescriptor { label: Some("my command buffer") });

// Schedule a command to fill the primary number array with some data.
let view = data.view::<PrimaryArray>();
let view_clone = view.clone();
command_buffer.schedule(CommandDescriptor {
    label: Some("fill array"),
    views: &[&view.as_mut(4..6)],
    command: move |ctx| ctx.get_mut(&view_clone).0[4..6].fill(33)
});

// Schedule a command to map the contents of the derived acceleration structure
// so that we may view them synchronously.
let derived = command_buffer.map(&data.view::<DoubledArray>().as_const());

// Submit the buffer for processing.
ctx.submit(command_buffer);

// The doubled acceleration structure automatically contains the
// correct, up-to-date data!
assert_eq!(&[0, 0, 0, 0, 66, 66, 0], &ctx.get(&derived).0[..]);