Photos that remember where and when they were taken, even after the metadata is gone.
A library that hides location (~1 m), altitude, UTC time (0.1 s) and a live/manual flag inside the pixels of a photo, invisibly, so the record survives WhatsApp, screenshots, crops, black & white filters and even a phone photo of a screen.
One mark format, three engines: 🤖 Android (Kotlin) · 🐍 Python (new) · 🌐 JavaScript reference
Ghostmark-Image · engine 1.1.0 · no UI · no permissions · no network · no native code
⏳ Heads up: this page is long and animation-heavy on purpose. It is a lab notebook as much as a README: 40 real photos from 11 different cameras (phones and drones), 480 attacked copies, 18 charts, 40 animated test reels and a black-and-white special. Skim the pictures, or read every number. Everything is generated from scripts in
tools/, nothing is typed by hand.
One photo, two views. The scan line reveals the invisible pattern that carries the record, then the reader decodes it in 0.15 s.
|
GPS and time live in file metadata. A chat app, a screenshot, a "save as" or a crop deletes them silently, and then the photo is just a picture. For anyone whose photos are evidence (inspectors, surveyors, insurers) that is a real loss. |
Spread a 118-bit record over ~10,000 colour tiles (and a 60-bit location backup over ~50,000 brightness tiles), protect it with a CRC and an error-correcting code, and push each tile by only 1 to 2 grey levels. Invisible to the eye, readable by the reader. |
329 of 360 chat, share, screenshot and edit copies gave back the full record; 0 false alarms in 320 unmarked reads; 0 wrong values across 480 edits. Median image change: 41.8 dB PSNR, SSIM 0.9974. |
Not studio shots: 40 real photos from 11 camera models: iPhone 16 Pro and 17 Pro, OnePlus, vivo, realme, a budget LAVA, and DJI drones, from 4.6 MP to 18.4 MP. Stone carvings, rebar, steel rigs, solar panels, temple spires, a castle in fog, a storm over a night sea, a moon over a bay, rain-soaked streets, blossoms, a plate of fruit and a café table. Easy, textured scenes and the awkward ones: flat sky, dark water, fog, night.
Every test photo, its hidden layer (×26) and 12 result dots: green = full record read, orange = location only, red = mark present but unreadable, grey = nothing found. Hover-free, no cherry-picking: this is the whole bench.
| Camera | Photos | Sensor / type | Avg. edits survived | Median PSNR |
|---|---|---|---|---|
| 📱 iPhone 17 Pro | 17 | iPhone 17 Pro | 8.7 / 12 | 43.0 dB |
| 📱 iPhone 16 Pro | 4 | iPhone 16 Pro | 10.5 / 12 | 39.2 dB |
| 🚁 DJI drone | 8 | FC3411, M3T | 10.4 / 12 | 38.8 dB |
| 📱 OnePlus | 4 | OnePlus 11 5G, OnePlus Nord 3 5G, OnePlus Nord2 5G | 10.2 / 12 | 40.0 dB |
| 📱 vivo | 2 | vivo 1951 | 11.0 / 12 | 39.1 dB |
| 📱 realme | 2 | realme 6i, realme GT Master Edition | 10.0 / 12 | 41.3 dB |
| 📱 LAVA | 3 | LXX518 | 5.0 / 12 | 43.9 dB |
🧬 The hidden layer, made visible
The mark is the difference between the marked and the original photo, magnified 26 times (red/blue = pixels pushed up/down). Notice where it isn't: flat sky, dark water and smooth plates receive almost nothing. That is the texture-adaptive rule keeping the change invisible exactly where your eye would catch it.
A 100 % crop, before and after. The difference map (×24) shows a smooth, low-frequency pattern; the histogram shows no pixel channel moves by more than 9 levels.
How big is the change, across all 40 photos? Median PSNR 41.8 dB (range 37.2 to 47.1), median SSIM 0.9974, median colour shift ΔE 1.5 (a ΔE of about 2 is the edge of what a trained eye notices side by side). Mean absolute change 1.4 grey levels of 255; the file grows by at most 9.0 %.
Every photo was marked with the same demo record (12.97160 N, 77.59457 E, 920 m, 2026-10-25 12:00:00.4 UTC) and put through the same nine steps: original, hidden layer, saved, chat app, 600 px thumbnail, black & white, a quarter painted over, a 10 % crop, and a (simulated) phone photo of a monitor. Green = full record back, orange = location only, red = mark present but unreadable. Failures are shown on purpose.
The question everyone asks: "what if someone just turns the photo black & white?" Colour is where the full record lives, so a grey photo loses it. That is exactly why Ghostmark writes a second, independent layer into the brightness of the picture: it is the part of the photo a black-and-white filter keeps.
Photo #11, start to finish: marked in colour → turned black & white → squeezed through a chat app (1600 px, JPEG q70) → the brightness layer, magnified ×20 → the location, read back. All of it measured, not mocked up.
The two layers are orthogonal by construction. Panel 2 is the colour layer: a black-and-white filter throws it away. Panel 3 is the brightness layer: a grey picture is brightness, so the filter hands it straight through. Panel 4 is the black-and-white copy that went through a chat app, minus the same copy of the unmarked photo: the pattern is still there.
The brightness layer is built to be the survivor, not the star: ~50,000 small tiles, a 60-bit payload (location only, about 10 m), the same pilot tiles, the same error-correcting code, a higher amplitude (up to 4.5 grey levels, still invisible at 100 %). It cannot carry time or altitude, there is no room, but a location that survives the filters people actually apply is the part that matters most.
Every test photo as the receiver sees it: grey, 1600 px, JPEG q70. Green frame = the location was read back (the number is how many metres off it was), red frame = not read. The red ones are mostly the dark, foggy and flat scenes, the same weakness we saw in colour.
29 of 40 photos gave their location back after a black-and-white filter and a chat app, at the default strength, and every one of those reads landed on the same spot, 6.6 m from the true position: that is the rounding of the 60-bit location grid, not noise.
Not every "black & white" is the same. A phone filter, a photo editor, a red-filter look and a plain desaturate all mix the colour channels differently, and each is followed by a different delivery path. We tried nine, on every photo:
| Black-and-white path | Location read | Median z (brightness layer) |
|---|---|---|
| B&W, saved as JPEG q95 | 31 / 40 | 8.1 |
| B&W + chat app (1600 px, q70) | 29 / 40 | 7.3 |
| B&W + social (1080 px, q50) | 24 / 40 | 6.8 |
| B&W + thumbnail (800 px, q40) | 18 / 40 | 5.3 |
| B&W + strong contrast + chat app | 24 / 40 | 7.1 |
| B&W + 75% centre crop + chat app | 0 / 40 | 0.0 |
| Equal-weight grey + chat app | 29 / 40 | 7.3 |
| Red-filter B&W + chat app | 27 / 40 | 7.3 |
| Green-filter B&W + chat app | 29 / 40 | 7.3 |
| Written at 1.5× strength, B&W + chat app | 37 / 40 | 10.8 |
| Written at 2× strength, B&W + chat app | 39 / 40 | 14.4 |
Left: how many photos keep their location. Middle: signal strength per variant against the 3.5 detection limit. Right: signal strength against scene texture for the black-and-white chat copy: busy scenes keep their location, flat and dark ones do not.
- It works, with a limit you can see. 29 of 40 photos at default strength after grey + chat app; the failures are the low-texture photos, the same ones that are fragile in colour (#2, #6, #9, #13, #21, #28, #31, #32, #33, #35, #37).
- A stronger mark buys most of it back. Written at 1.5× the location was read for 37 of 40 photos, at 2× for 39 of 40 (mean PSNR 38.5 dB and 36.0 dB instead of the default 41.8 dB). The Android self-test today checks the colour path (a chat app); teaching it to check a grey copy too is the obvious next step.
- Heavy sharing hurts more than the grey, and a crop is a hard stop. The 800 px / q40 thumbnail reads 18 of 40. A black-and-white copy that is also cropped reads 0 of 40: the crop search finds the original frame by its colour pilots, and a grey picture has none. Full-frame grey copies are fine; grey crops are not.
- The grey recipe matters a little. Equal-weight grey: 29, red filter: 27, green filter: 29, against 29 for the standard luma conversion. A red or green filter lets some of the colour layer leak into the grey picture as noise, an equal-weight grey slightly reweights the brightness layer.
- Never a wrong location. Every location that passed the CRC was within 6.6 m of the truth (the format's grid is about 10 m), across 211 reads. The same grey + chat path on the unmarked originals returned "no mark" for 40 of 40 photos.
- Two engines, same answer. The Python engine read the 320 grey copies and agreed with the JavaScript reference engine on status and location in 320 of 320 cases.
Each cell is the colour-layer detection strength z (threshold 3.5) or a status: LOC = location only (orange frame), DMG = mark present but unreadable (red frame), -- = no mark found. Photo labels are coloured by camera.
| Edit applied to the stored JPEG | Full record | Location only (layer 2 alone) | Median z |
|---|---|---|---|
| Stored JPEG q95 (full size) | 38 / 40 | 31 / 40 | 22 |
| Chat app: 1600 px, q70 | 38 / 40 | 28 / 40 | 20 |
| Social: 1080 px, q50 | 36 / 40 | 23 / 40 | 15 |
| Thumbnail: 800 px, q40 | 24 / 40 | 18 / 40 | 10 |
| Tiny: 600 px, q50 | 18 / 40 | 12 / 40 | 10 |
| Shared three times (1600 > 1280 > 1080) | 30 / 40 | 21 / 40 | 13 |
| Screenshot-style PNG, 1080 px | 38 / 40 | 27 / 40 | 22 |
| Brightness/contrast edit + 1600 q70 | 38 / 40 | 27 / 40 | 19 |
| Colour drained to 40% + 1600 q70 | 36 / 40 | 28 / 40 | 16 |
| Blur (sigma 1.2) + 1600 q70 | 38 / 40 | 22 / 40 | 20 |
| 25% of area painted over + 1600 q70 | 37 / 40 | 23 / 40 | 16 |
| Black and white + 1600 q70 | 0 / 40 | 29 / 40 (as PART) |
n/a |
| Crop to 81% of the area (centre 90%), then chat app | 35 / 40 | n/a | n/a |
| Crop to 56% of the area (centre 75%), then chat app | 33 / 40 | n/a | n/a |
| Crop to 36% of the area (centre 60%), then chat app | 32 / 40 | n/a | n/a |
| Crop to 20% of the area (centre 45%), then chat app | 0 / 40 | n/a | n/a |
| Crop to 90% of the area (right 10% trimmed), then chat app | 35 / 40 | n/a | n/a |
| Crop to 25% of the area (top-left quarter), then chat app | 0 / 40 | n/a | n/a |
| Simulated phone photo of a monitor (4 tapped corners) | 36 / 40 | n/a | n/a |
| Simulated phone photo of a print (4 tapped corners) | 33 / 40 | n/a | n/a |
Every dot is one photo. Boxes are quartiles; cyan is the median. Healthy copies sit at z = 15 to 35, the threshold is 3.5, and the unmarked control stays flat on the floor.
Top: z versus long edge (grey = each photo, cyan band = middle 50 %). Bottom left: the share of photos still read in full. Bottom right: JPEG quality alone, on a 1600 px copy.
Left: 320 reads of unmarked photos (8 degradations × 40 photos) never crossed the threshold; real reads start at z ≈ 5.5. Right: detection strength follows scene texture (r = 0.62).
Crop, then chat app: a 10 % trim or centred 90 % / 75 % / 60 % crop is found and read in most photos (35, 35, 33, 32 of 40); a 45 % or quarter-frame crop is not (0 and 0 of 40). Simulated phone photos of a monitor: 36 of 40 read in full; of a print: 33 of 40.
Left: edits survived per camera (dots = individual photos). Middle: marking time on the JavaScript reference engine, measured one photo at a time on one PC core (about 13.9 ms per megapixel, median 306.5 ms; the Kotlin engine on a phone takes ~1.2 s for 12 MP including the self-test). Right: a full-size read, median 110.5 ms.
The mark strength is a dial. We re-marked every photo at five strengths and measured both invisibility and survival.
| Strength (× default) | Mean PSNR | Chat app (1600 px q70) read in full | Thumbnail (800 px q40) read in full |
|---|---|---|---|
| 0.5× | 46.5 dB | 85% | 12% |
| 0.75× | 43.9 dB | 90% | 28% |
| 1× (default) | 41.7 dB | 95% | 60% |
| 1.5× | 38.5 dB | 98% | 88% |
| 2× | 36.0 dB | 100% | 95% |
Take-away: the default (1×) already reads 38 of 40 photos after a chat app, but only 60.0 % survive an 800 px / q40 thumbnail. Raising the strength to 1.5× to 2× rescues most of the rest, at a visible cost (mean PSNR drops from 41.7 to 36.0 dB). That is exactly why Ghostmark ships a capture-time self-test: it simulates a chat app on the fresh photo and raises the strength (up to 1.9×) only for the photos that need it, so a textured temple stays subtle and a smooth plate gets help. In this benchmark 16 photos are lost at 800 px / q40 with the 1× mark, and 14 of them are read in full once the strength is doubled. The photos that fail even as a stored copy at 1× are #22, #25 (a dark plate of fruit, and blossoms against a dusk sky): exactly the case the self-test exists for.
|
The five most robust photos
|
The five most fragile photos
|
- Texture is destiny. The marks that survive best live in rubble, rebar, stone and foliage. Flat sky, dark water and smooth plates are the weak spots, by design: the adaptive rule withholds the mark where it would be visible, and there is less pattern to hide in.
- Resolution matters more than JPEG quality. The reader keeps working down to JPEG q10 at 1600 px, but falls apart below roughly 600 to 800 px on the long edge. Chat apps resize; they rarely crush quality that far.
- The luma layer earns its place. Convert a photo to black & white and the colour layer is gone for every photo. The brightness layer still returned the location for 29 of 40 photos after a chat app (median error 6.6 m), and 39 of 40 when the mark is written at twice the strength. That is the whole point of having two independent layers: see the black & white special.
- Never a wrong answer. The CRC-16 plus the convolutional code turned every uncertain read into an honest
PARTorDAMAGED, not into a plausible-looking wrong coordinate: 0 wrong values across 480 edits and 320 unmarked controls. - A new camera generation, same behaviour. The 17 iPhone 17 Pro photos survived 8.7 of 12 edits on average (iPhone 16 Pro: 10.5). Many of them are the hard kind (dusk, fog, night sea), which is why the average dropped when they joined the bench.
- Photos of screens and prints are viable, with help. With four tapped corners the reader aligned and decoded 36 of 40 simulated monitor shots and 33 of 40 print shots; that takes about 20.3 s per photo on a PC core, and 24 to 55 s on a phone. Real hardware is harsher than the simulator, see the limits below.
- Drones are fine. Aerial scenes (fields, scrub, solar panels) are almost pure texture: the 8 DJI photos survived 10.4 of 12 edits on average, on par with the flagship phones (10.5); the weakest group was the budget LAVA phone (5.0), whose photos here were the smoothest and darkest.
|
🎨 Layer 1: colour ~10,000 tiles in Cb/Cr, ±(≤2) grey levels. 118 bits: location, altitude, time, 0.1 s, live/manual, CRC-16. Rate-½ convolutional code → 248 coded bits, ~32 tiles per bit. |
💡 Layer 2: brightness ~50,000 tiles in luma carry only the location (60 bits, ~10 m). Weaker, but it survives black & white. |
📍 Pilot tiles 20% of tiles have a known sign. The reader correlates against them to detect, to align after a crop, to fix a tilted photo of a screen, and as a built-in false-alarm test. |
🛡️ Self-test At capture the engine squeezes a copy like a chat app would, reads it, and strengthens the mark (up to ×1.9) if it did not survive. |
Exact bit layout: FORMAT-SPEC.md. The JavaScript implementation in reference/ wins over the text if they ever disagree.
|
|
The Android engine is the production one. For everything that is not an Android app (a server that checks uploaded photos, a batch tool for a survey team, a Jupyter notebook, a desktop app, a Raspberry Pi on a drone) there is now a Python engine in python/: the same mark format, written with numpy only (Pillow optional, for file helpers and the self-test).
A real run of python/examples/mark_and_read.py (output not edited).
pip install "ghostmark-image[files]" # numpy (+ Pillow for the file helpers); Python 3.9+
ghostmark mark photo.jpg marked.jpg --lat 12.9716 --lon 77.5946 --alt 920 --time 2026-10-25T12:00:00Z
ghostmark read shared_copy.jpg # status ok = full record; part = location only (+-10 m); none = no markimport numpy as np
from PIL import Image
import ghostmark as gm
rgb = np.asarray(Image.open("photo.jpg").convert("RGB"))
marked, strength = gm.mark_verified(rgb, gm.Record(lat=12.9716, lon=77.5946, alt=920, t=1792929600))
Image.fromarray(marked).save("marked.jpg", quality=95, subsampling=0)
r = gm.read(np.asarray(Image.open("whatsapp_copy.jpg").convert("RGB")))
r.status # 'ok' | 'part' | 'damaged' | 'none'
r.location # (lat, lon) from the best layer that was read, or None
r.time # UTC datetime (full record only)| Python engine | |
|---|---|
| Same format | reads every mark the Kotlin and JavaScript engines write, and the other way round (marks written by Python were read in full by the JavaScript engine for 38 of 40 photos, and marks written by JavaScript were read in full by Python for 38 of 40) |
| API | mark · mark_verified (chat-app self-test) · read · read_cropped · read_perspective · auto_corners · mark_file · read_file · CLI ghostmark |
| Dependencies | numpy only; Pillow for files and the self-test |
| Speed (12 MP, one core) | mark ≈ 2.1 s · read ≈ 0.5 s · crop search ≈ 5.5 s · screen/print ≈ 48.9 s |
| Tests | 14 pytest tests, including the Android engine's cross-language fixtures |
| Not in the Python port | native speed (it is numpy, not NEON) |
Full guide: python/README.md.
// settings.gradle.kts → include(":engine") | app → implementation(project(":engine"))
// 1. At capture, after your overlay is drawn and before JPEG encoding (background thread):
val record = MarkRecord(lat, lon, altitudeMeters = alt, timeSeconds = Math.floorDiv(shutterMillis, 1000L),
tenth = (Math.floorMod(shutterMillis, 1000L) / 100).toInt())
InvisibleMark.mark(finalBitmap, record) // null = failed: save the photo unmarked
// 2. Any time later, on any copy:
val read = InvisibleMark.read(context, uri, ReadOptions(searchCrops = true))
when (read?.result?.status) { // OK · PART · DAMAGED · NONE
MarkStatus.OK -> read.result.layer1 // lat, lon, alt, time, manual flag
MarkStatus.PART -> read.result.layer2 // location only, ~10 m
else -> { /* tell the user. never invent a value */ }
}Full API, four integration flows and an AI-assistant checklist: engine/ENGINE.md. Not on Android? See Ghostmark for Python.
| Measured on a real phone (12 MP) | |
|---|---|
| Mark (embed + self-test) | ~0.65 s + ~0.56 s |
| Read | ~0.15 s |
| Read with crop search | 2.5 to 4.6 s |
| Read a photo of a screen / print | 24 s (clean case) to ~55 s |
engine/ the Android library: source, ENGINE.md, FORMAT-SPEC.md, 29 JVM tests with cross-language fixtures
python/ the Python engine (numpy only): package, CLI, 14 tests, examples
PR-KIT/ launch toolkit: social preview, post drafts, release notes, FAQ (not needed to use the library)
dist/ prebuilt ghostmark-1.1.0.aar (97 KB)
trial/ small Compose sample app: Capture, Read, Screen/print with a corner editor
reference/ JavaScript reference implementation + CLI
samples/ demo photos, marked (1600 px, EXIF-free): send them through any messenger and read them back
tools/ run_experiments.py → run_bw.py → make_figures.py → make_bw.py → make_showcase.py → make_readme.py (builds this page from the data)
assets/ README media
gradlew :engine:testDebugUnitTest :engine:assembleRelease :trial:assembleDebug
cd python && pip install -e .[test] && pytest -q
cd tools && python run_experiments.py && python run_bw.py && python make_figures.py && python make_bw.py && python make_showcase.py && python make_readme.py
# needs numpy, opencv-python, matplotlib, pillow, node. Set MARK_PHOTOS=<folder> to use your own photos.
Benchmark note: attacks ran on the JavaScript reference engine, which shares format and maths with the Kotlin engine (cross-checked by shared fixtures in both directions). Photos above 24 MP are downscaled to a 4096 px long edge first, as the Android engine does (that affects only the 50 MP photo). On-phone timings and the two real monitor photos come from the Kotlin engine on a 1220 × 2712 Android phone.
Open source under the Apache License 2.0. Use it, change it, ship it, commercially or not, in any app or tool. The one thing we ask is the thing the licence already requires: keep the credit. Keep the NOTICE file with your distribution, and we would love a line in your About screen or docs:
Uses Ghostmark by Vijay Parmar, github.com/thevijayparmar/Ghostmark-Image
Copyright © 2026 Vijay Parmar. Contributions are welcome: see CONTRIBUTING.md.
Contributors: Vijay Parmar · Prathvi Bhatti (co-author)
👻 Built by Vijay Parmar · the photo remembers, even when the file forgets.
















































