ATE 文件系统

什么是 ATE 文件系统？

ATE 文件系统是建立在 ATE 分布式数据存储之上的分布式文件系统。虽然作者完全清楚文件系统生态已经相当饱和，但就大局而言，这是一个好问题，这个特定版本有一些独特的特性，使其与其他版本区分开来。

什么是 ATE

请参阅此处

摘要

ATE 文件系统使用 FUSE 将文件系统命令重定向到响应 I/O 的用户空间程序。此用户空间程序查询与许多消费者和生产者同步的内存中分布式数据链。数据被复制、加密、签名并确保多用户安全。

ATE 文件系统的另一个关键特性是它是完全软件定义的，这允许结构化非文件数据模型的物化视图也存在于文件系统的模拟部分。

特性

• 高度可扩展（相对于其他文件系统）
• 通过本地重做日志复制实现低延迟读取
• 具有分布式提交的写入通过缓存
• 对文件的分布式锁定
• 完全加密的文件和元数据
• 全面量子抗加密
• 模拟文件的可编程 API

高级设计

.--[   App  ]---. .--[   App  ]---. .--[   App  ]---.
|               | |               | |               |
|>local redo-log| |>local redo-log| |>local redo-log|
|.-------------.| |.-------------.| |.-------------.|
|| Chain     1 || || Replica P1  || || Replica P1  ||
||             || || Chain     2 || || Replica P2  ||
|| Replica P3  || ||      ^      || || Chain     3 ||
|*-------------*| |*------|------*| |*-------------*|
|               |       subscribe                   
|                \________|__________________________
|                         |                          
|  >local redo-log                                   
|  >Crypto-Graph Materiaized View< (in memory)       
|  .----------------------------------.      session 
|  |             root                 |   .-----------.
|  |              |                   |   |  -token   |
|  |      dao----dao                  |---|  -claims  |
|  |              \                   |   |  -keys    |
|  |               dao                |   |  -timeout |
|  |                                  |   *-----------*
   +----------------------------------+----------------+
   |                      atefs                        |
   +---------------------------------------------------+
   |                 fuse (/dev/fuse)                  |
   +---------------------------------------------------+
   |                   Linux Kernel                    |
   +---------------------------------------------------+
   |           Linux System Calls (e.g. read)          |
   +---------------------------------------------------+

安装

sudo apt install cargo make pkg-config libfuse-dev libfuse3-dev openssl libssl-dev
cargo install atefs

手册

atefs 1.3
John S. <johnathan.sharratt@gmail.com>

USAGE:
    atefs [FLAGS] [OPTIONS] <SUBCOMMAND>

FLAGS:
    -d, --debug      Logs debug info to the console
        --dns-sec    Determines if ATE will use DNSSec or just plain DNS
    -h, --help       Prints help information
    -n, --no-auth    No authentication or passcode will be used to protect this file-system
        --no-ntp     No NTP server will be used to synchronize the time thus the server time will be
                     used instead
    -v, --verbose    Sets the level of log verbosity, can be used multiple times
    -V, --version    Prints version information

OPTIONS:
    -a, --auth <auth>
            URL where the user is authenticated [default: ws://tokera.sh/auth]

        --dns-server <dns-server>
            Address that DNS queries will be sent to [default: 8.8.8.8]

        --ntp-pool <ntp-pool>
            NTP server address that the file-system will synchronize with

        --ntp-port <ntp-port>
            NTP server port that the file-system will synchronize with

    -t, --token <token>
            Token used to access your encrypted file-system (if you do not supply a token then you
            will be prompted for a username and password)

        --token-path <token-path>
            Token file to read that holds a previously created token to be used to access your
            encrypted file-system (if you do not supply a token then you will be prompted for a
            username and password)

        --wire-encryption <wire-encryption>
            Indicates if ATE will use quantum resistant wire encryption (possible values are 128,
            192, 256). The default is not to use wire encryption meaning the encryption of the event
            data itself is what protects the data


SUBCOMMANDS:
    group    Groups are collections of users that share same remote file system
    help     Prints this message or the help of the given subcommand(s)
    mount    Mounts a local or remote file system
    token    Tokens are needed to mount file systems without prompting for credentials
    user     Users are needed to access any remote file systems

--------------------------------------------------------------------------

Users are needed to access any remote file systems

USAGE:
    atefs user <SUBCOMMAND>

FLAGS:
    -h, --help       Prints help information
    -V, --version    Prints version information

SUBCOMMANDS:
    create     Creates a new user and generates login credentials
    details    Returns all the details about a specific user
    help       Prints this message or the help of the given subcommand(s)

--------------------------------------------------------------------------

Groups are collections of users that share same remote file system

USAGE:
    atefs group <SUBCOMMAND>

FLAGS:
    -h, --help       Prints help information
    -V, --version    Prints version information

SUBCOMMANDS:
    add-user       Adds another user to an existing group
    create         Creates a new group
    details        Display the details about a particular group (token is required to see role
                   membership)
    help           Prints this message or the help of the given subcommand(s)
    remove-user    Removes a user from an existing group

--------------------------------------------------------------------------

Tokens are needed to mount file systems without prompting for credentials

USAGE:
    atefs token <SUBCOMMAND>

FLAGS:
    -h, --help       Prints help information
    -V, --version    Prints version information

SUBCOMMANDS:
    gather      Gather the permissions needed to access a specific group into the token using
                either another supplied token or the prompted credentials
    generate    Generate a token with normal permissions from the supplied username and password
    help        Prints this message or the help of the given subcommand(s)
    sudo        Generate a token with extra permissions with elevated rights to modify groups
                and other higher risk actions

--------------------------------------------------------------------------

Mounts a local or remote file system

USAGE:
    atefs mount [FLAGS] [OPTIONS] <mount-path> [ARGS]

ARGS:
    <mount-path>    Path to directory that the file system will be mounted at
    <remote>        URL where the data is remotely stored on a distributed commit log (e.g.
                    ws://tokera.sh/db/myfs). If this URL is not specified then data will only be
                    stored locally
    <log-path>      (Optional) Location of the local persistent redo log (e.g. ~/ate/fs)

FLAGS:
        --allow-other        Allow other users on the machine to have access to this file system
        --allow-root         Allow the root user to have access to this file system
        --compact-now        Forces the compaction of the local redo-log before it streams in the
                             latest values
    -h, --help               Prints help information
    -i, --impersonate-uid    For files and directories that the authenticated user owns, translate
                             the UID and GID to the local machine ids instead of the global ones
        --non-empty          Allow fuse filesystem mount on a non-empty directory, default is not
                             allowed
    -r, --read-only          Mount the file system in readonly mode (`ro` mount option), default is
                             disable
        --temp               Local redo log file will be deleted when the file system is unmounted,
                             remotely stored data on any distributed commit log will be persisted.
                             Effectively this setting only uses the local disk as a cache of the
                             redo-log while it's being used
    -V, --version            Prints version information
    -w, --write-back         Enable write back cache for buffered writes, default is disable

OPTIONS:
        --compact-mode <compact-mode>
            Mode that the compaction will run under (valid modes are 'never', 'modified', 'timer',
            'factor', 'size', 'factor-or-timer', 'size-or-timer') [default: factor-or-timer]

        --compact-threshold-factor <compact-threshold-factor>
            Factor growth in the log file which will trigger compaction - this
            argument is ignored if you select a compact_mode that has no growth trigger [default:
            0.4]

        --compact-threshold-size <compact-threshold-size>
            Size of growth in bytes in the log file which will trigger compaction (default: 100MB) -
            this argument is ignored if you select a compact_mode that has no growth trigger
            [default: 104857600]

        --compact-timer <compact-timer>
            Time in seconds between compactions of the log file (default: 1 hour) - this argument is
            ignored if you select a compact_mode that has no timer [default: 3600]

        --configured-for <configured-for>
            Configure the log file for <raw>, <barebone>, <speed>, <compatibility>, <balanced> or
            <security> [default: speed]

        --data-format <data-format>
            Format of the data in the log file as <bincode>, <json> or <mpack> [default: bincode]

    -g, --gid <gid>
            GID of the group that this file system will be mounted as

        --meta-format <meta-format>
            Format of the metadata in the log file as <bincode>, <json> or <mpack> [default:
            bincode]

    -p, --passcode <passcode>
            User supplied passcode that will be used to encrypt the contents of this file-system
            instead of using an authentication. Note that this can 'not' be used as combination with
            a strong authentication system and hence implicitely implies the 'no-auth' option as
            well

        --recovery-mode <recovery-mode>
            Determines how the file-system will react while it is nominal and when it is recovering
            from a communication failure (valid options are 'async', 'readonly-async', 'readonly-
            sync' or 'sync') [default: readonly-async]

    -u, --uid <uid>
            UID of the user that this file system will be mounted as

贡献

如果您想帮助建立一个社区以继续开发此项目，请联系我 [email protected]