BodyCompress

This library compresses and serializes the output of nonparametric 3D human mesh estimators such as Neural Localizer Fields (NLF) to disk.

Without compression, a sequence of 3D human meshes extracted from a video can take up huge amounts of disk space, as we need to store the coordinates for thousands of vertices in every frame. At 30 fps and 6890 vertices (like SMPL), this amounts to almost 9 GB/person/hour. If you want to save the estimation result for a multi-person video, it will be proportionally more.

This library achieves a compression ratio of over 8x on temporal human mesh data, with minimal loss in information. It consists of the following steps:

1. Quantization: The floating-point coordinates of the vertices are quantized at 0.5 mm resolution.
2. Vertex Reordering: The vertices are transparently reordered with a bundled TSP-optimized order (auto-detected for SMPL and SMPL-X) so that consecutive vertices are spatially adjacent, which makes the differential encoding more effective. The original order is restored on decompression.
3. Differential Encoding: The quantized coordinates are differentially encoded in the (reordered) vertex order, so the differences between adjacent vertices tend to be small.
4. Serialization: msgpack-numpy is used to serialize the NumPy arrays to a byte stream.
5. Compression: The serialized byte stream is compressed losslessly with xz (LZMA), using the multi-threaded lzma-mt library, which is reasonably fast at compression level 5. (The Python standard library lzma module does not have multi-threading support and is too slow for our use case.) Alternatively, zstd compression can be selected for faster (de)compression at a somewhat lower compression ratio.

The format supports storing additional metadata in the header, and several per-frame pieces of information, such as vertices, joints, uncertainties, and camera parameters, compressing it all into one sequentially readable file.

Installation

pip install bodycompress

Usage

Use the BodyCompressor and BodyDecompressor classes to compress and decompress the data. The compressor should be used as a context manager and has an append method which should be called with keyword arguments. The decompressor is an iterable over dictionaries with the same keys; it also knows the number of frames upfront (len(bdecompr)) and can be iterated multiple times (each pass decompresses the file again from the start).

Note that seeking is not supported, the stream is compressed as a whole to achieve the best compression ratio.

Compression

from bodycompress import BodyCompressor

with BodyCompressor('out.xz', metadata={'whatever': 'you want'}) as bcompr:
    for frame in frames:
        vertices, joints = estimate(frame)
        bcompr.append(vertices=vertices, joints=joints)

Any keyword arguments can be passed to append that are nested dicts/lists/tuples of primitive types or NumPy arrays. However the following keywords are handled specially:

• vertices: a (..., num_verts, 3) NumPy array of vertex coordinates (in millimeters)
• joints: a (..., num_joints, 3) NumPy array of joint coordinates (in millimeters)
• vertex_uncertainties: a (..., num_verts) NumPy array of vertex uncertainties (in meters)
• joint_uncertainties: a (..., num_joints) NumPy array of joint uncertainties (in meters)
• camera: a deltacamera.Camera object (or a dict in the format produced by bodycompress.cam_to_dict)

Coordinates are expected in millimeters; a warning is issued if the data looks like it might be in meters (i.e., its value range is tiny compared to the quantization step).

Input is validated and consumed synchronously in append: invalid data (NaN coordinates, malformed cameras, unserializable values) raises immediately without invalidating the file, so you can skip the offending frame and keep going, and you may freely reuse or overwrite the passed arrays after append returns. Only the final compression runs in a background thread; if that fails, the file is finalized with the frames written so far when possible.

If an exception is raised inside the with block, the partially written (unusable) file is deleted. If a compressor is never closed, it is finalized at interpreter exit and a ResourceWarning is emitted; don't rely on this, use the context manager or call close().

Useful options of BodyCompressor (see the API reference for the full list):

• quantization_mm=0.5: coordinate resolution; coarser quantization gives smaller files
• compression='xz': pass 'zstd' for much faster compression at a somewhat lower ratio
• compression_level=None: defaults to 5 for xz and 3 for zstd
• n_threads=0: number of compression threads (0 = auto-detect CPU count)

Decompression

from bodycompress import BodyDecompressor

bdecompr = BodyDecompressor('out.xz')
print(bdecompr.metadata)  # {'whatever': 'you want'}
print(len(bdecompr))  # number of frames
for data in bdecompr:
    render(data['vertices'], data['joints'])

Stored cameras are yielded as plain dicts by default; pass decode_camera=True to BodyDecompressor to get deltacamera.Camera objects back.