pyrus-cramjam

API文档

安装

pip install --upgrade cramjam  # Requires no Python or system dependencies!

为Rust中的解/压缩算法提供的极薄的Python绑定。允许使用如Snappy等算法，无需任何系统依赖。

这在AWS Lambda等环境中非常有用，在这些环境中安装像python-snappy这样的包变得困难，因为这些包有系统级依赖。

基准测试

一些基本基准测试可在基准测试目录中找到

支持的算法

• Snappy
• Brotli
• Lz4
• Gzip
• Deflate
• ZSTD

所有这些都可以用作

>>> import cramjam
>>> import numpy as np
>>> compressed = cramjam.snappy.compress(b"bytes here")
>>> decompressed = cramjam.snappy.decompress(compressed)
>>> decompressed
cramjam.Buffer(len=10)  # an object which implements the buffer protocol
>>> bytes(decompressed)
b"bytes here"
>>> np.frombuffer(decompressed, dtype=np.uint8)
array([ 98, 121, 116, 101, 115,  32, 104, 101, 114, 101], dtype=uint8)

API是 cramjam.<-compression>.compress/decompress 并接受 bytes/bytearray/numpy.array/cramjam.File/cramjam.Buffer 对象。

解/压缩到 此外，所有变体都支持 decompress_into 和 compress_into。例如：

>>> import numpy as np
>>> from cramjam import snappy, Buffer
>>>
>>> data = np.frombuffer(b'some bytes here', dtype=np.uint8)
>>> data
array([115, 111, 109, 101,  32,  98, 121, 116, 101, 115,  32, 104, 101,
       114, 101], dtype=uint8)
>>>
>>> compressed = Buffer()
>>> snappy.compress_into(data, compressed)
33  # 33 bytes written to compressed buffer
>>>
>>> compressed.tell()  # Where is the buffer position?
33  # goodie!
>>>
>>> compressed.seek(0)  # Go back to the start of the buffer so we can prepare to decompress
>>> decompressed = b'0' * len(data)  # let's write to `bytes` as output
>>> decompressed
b'000000000000000'
>>>
>>> snappy.decompress_into(compressed, decompressed)
15  # 15 bytes written to decompressed
>>> decompressed
b'some bytes here'

特别说明！
如果您知道解/压缩输出的长度，可以向任何 de/compress 提供 output_len=<<some int>> 来获得 ~1.5-3 倍的性能提升，因为这允许单次缓冲区分配；如果您使用 cramjam.Buffer 或 cramjam.File 对象，则不适用。