metabeam

Point it at any file and read everything inside.

metabeam is a fast metadata extraction engine for any file type. It identifies a file by its magic bytes (never its extension), runs every applicable parser, and returns a single structured JSON report: cryptographic hashes and Shannon entropy, image geometry, EXIF and GPS coordinates, audio tags, video timing, document properties, archive contents, and binary headers.

One Rust core powers a command-line tool and ships as native packages for Python (pip install metabeam) and Node.js (npm install metabeam), so the same extraction logic is available wherever you work.

metabeam photo.jpg

{
  "file": { "size_bytes": 48211, "sha256": "...", "entropy_bits_per_byte": 7.2 },
  "jpeg": { "width": 4032, "height": 3024, "quality_estimate": 85 },
  "exif": { "gps_decimal": { "latitude": 37.77, "longitude": -122.42 } }
}

What metabeam does

• Detects the real file type from content, not the extension. A PNG renamed to .txt is still recognized as a PNG, and the report shows extension_claimed next to mime_detected so the mismatch is obvious. This makes metabeam useful for validating uploads and spotting spoofed files.
• Reads format-specific metadata. EXIF camera tags and decimal GPS, ID3 audio tags, MP4/MOV timing, PDF page and object counts, ZIP and Office document contents, ELF binary headers, and more.
• Computes universal fingerprints. SHA-256, CRC-32, byte-level Shannon entropy, and detected MIME type for every file, in a single streaming pass.
• Never crashes on hostile input. Every parser uses checked indexing, checked offset arithmetic, and bounded loops. A malformed section records an error and the other extractors keep running. This is fuzz-tested.

Why the output looks the way it does

Metadata does not fit one fixed schema, so metabeam does not impose one. Each extractor writes its own namespace of whatever that format actually contains, and several extractors run on a single file. A JPEG yields file, jpeg, and exif namespaces at once. This mirrors how mature tools such as ExifTool expose format-specific detail instead of flattening everything into shared fields.

Supported formats

Namespace	Formats	What it reads
file	all	size, SHA-256, CRC32, Shannon entropy, detected MIME, timestamps
jpeg	JPEG	segment structure, dimensions, quality estimate, ICC/XMP/IPTC detection
png	PNG	every chunk, geometry, text chunks, gamma, DPI, timestamp
gif	GIF	version, canvas, color table, frame count, animation loop
bmp	BMP	DIB header geometry, bit depth, compression
riff	WebP, WAV, AVI	RIFF form type, image geometry, audio format
exif	JPEG, TIFF, PNG, WebP, HEIC	every readable tag per IFD, decimal GPS
id3	MP3 and other ID3 audio	tag version, title, artist, album, year, genre, track
mp4	MP4, MOV, M4A, HEIF	atom tree, brand, duration
elf	ELF binaries	class, endianness, ABI, type, machine
pdf	PDF	version, object and page counts, Info dictionary, encryption
gzip	gzip	original filename, timestamp, OS
zip	ZIP, docx, xlsx, pptx, jar, apk, epub	entry list, compression, container identification

Adding a format is one new module plus two one-line registrations, and it cannot break the existing extractors. See CONTRIBUTING.md.

Install

Command line (Rust)

With a Rust toolchain (1.75 or newer):

cargo install metabeam

Or build from a clone:

make build      # produces target/release/metabeam

Python

pip install metabeam

Node.js

npm install metabeam

Docker

make docker-build
make docker-run FILE=path/to/file

Usage

Command line

metabeam file.jpg                 # pretty JSON
metabeam --compact a.png b.docx   # compact JSON, multiple files

The output is JSON, so it composes with jq:

metabeam photo.jpg | jq '.file.sha256'
metabeam photo.jpg | jq '.jpeg.quality_estimate'

# Detect a spoofed extension: compare claimed vs detected.
metabeam document.txt | jq '{claimed: .file.extension_claimed, real: .file.mime_detected}'

The exit code is non-zero if a file cannot be read, so it behaves well in scripts and pipelines.

Python

import metabeam

# Parse a file from disk. Raises OSError if it cannot be read.
report = metabeam.parse("photo.jpg")
print(report["file"]["mime_detected"])   # "image/jpeg"
print(report["file"]["sha256"])          # full hex digest
print(report["jpeg"]["quality_estimate"])

# Parse bytes that never touch disk (an upload, a download, a DB blob).
with open("photo.jpg", "rb") as f:
    report = metabeam.parse_bytes(f.read())

A practical pattern: confirm an upload really is the type it claims to be.

import metabeam

def is_real_png(data: bytes) -> bool:
    report = metabeam.parse_bytes(data)
    return report["file"]["mime_detected"] == "image/png"

Node.js

const metabeam = require("metabeam");
const fs = require("fs");

// Parse a file from disk. Throws if it cannot be read.
const report = metabeam.parse("photo.jpg");
console.log(report.file.mime_detected);   // "image/jpeg"
console.log(report.jpeg.quality_estimate);

// Parse a Buffer directly (for example, an HTTP upload body).
const fromBytes = metabeam.parseBytes(fs.readFileSync("photo.jpg"));

ESM / TypeScript:

import { parse, parseBytes } from "metabeam";

const report = parse("photo.jpg");

Rejecting a mismatched upload in an Express handler:

const metabeam = require("metabeam");

app.post("/upload", (req, res) => {
  const report = metabeam.parseBytes(req.body);          // req.body is a Buffer
  if (report.file.mime_detected !== "image/png") {
    return res.status(400).send("expected a PNG");
  }
  // ... store it ...
});

Rust library

use std::path::Path;

let report = metabeam_core::parse_path(Path::new("photo.jpg"))?;
println!("{}", serde_json::to_string_pretty(&report)?);

// Or from bytes:
let mut cursor = std::io::Cursor::new(bytes);
let report = metabeam_core::parse(&mut cursor, len);

How it works

metabeam reads the first few kilobytes of a file, runs magic-byte detection to build a probe (detected MIME, canonical extension, head bytes, length), then asks every registered extractor whether it supports that probe. Each matching extractor runs independently and writes its own namespace into the report. If one extractor fails on a malformed section, its error is recorded by name and the rest still run, so you get every piece of metadata the file can yield.

See docs/ARCHITECTURE.md for the full design.

Common questions

Does metabeam trust the file extension? No. Detection is by content (magic bytes). The claimed extension is reported alongside the detected MIME type so you can see when they disagree.

Can a malformed or malicious file crash it? That is the core invariant: no input may cause a panic, an infinite loop, or unbounded memory use. It is verified by per-format malformed-input tests, a randomized never-panic test in the regular test suite, and a dedicated cargo-fuzz target.

Is the output the same across the CLI, Python, and Node? Yes. All three front ends serialize the same core report, so the JSON object / dict / object you get back has an identical shape.

How do I add a new format? Write one extractor module and register it in two lines. Existing extractors are never touched. The walkthrough is in CONTRIBUTING.md.

Project layout

crates/core      the library: value model, extractor trait, all extractors
crates/cli       the command-line front end
crates/python    PyO3 bindings (pip package via maturin)
crates/node      napi-rs bindings (npm package)
fuzz             cargo-fuzz target
docs             architecture and design notes

Contributing

Contributions are welcome, especially new format support. Run the same checks CI runs before opening a pull request:

make check

Start with CONTRIBUTING.md, which includes a complete walkthrough of adding a format.

License

Licensed under either of MIT or Apache 2.0 at your option.