Tools for pulling data off a USB flash drive whose controller has failed — the kind that disappears from the bus instead of returning a read error, and that ordinary recovery software gives up on immediately.
If your drive does any of the following, this is the right place:
ddrescueorddhangs forever and rescues nothingdmesgfills withdevice descriptor read/64, error -71,reset high-speed USB device, or the drive re-enumerating every few seconds- the drive vanishes and reappears as a different
/dev/sdXconstantly photorecandtestdiskbail out or find nothing- Windows offers to format it; Linux mounts it then throws I/O errors
- it reads a little bit, then stops dead until you unplug it
That is not usually dead flash. It is usually the controller.
Ordinary recovery tools assume a disk that answers, even if the answer is "read error". A drive with a failing controller doesn't answer at all — it drops off the USB bus mid-transfer. Two things follow, and both break normal tooling:
ddrescuecannot make progress. It retries the same sector forever because from its point of view the device simply ceased to exist. It never reaches the good data further in.- The kernel makes it worse. When a USB storage device faults, the
usb_storagedriver's error handler does a full port reset and re-enumeration. That costs several seconds per failure, and it's the dominant cost of the whole operation.
On the drive these tools were written for (an HP v195b, 32 GB), measurement showed something specific and useful:
- The controller served exactly one read of about 128 KiB, then stopped answering until roughly two minutes had passed.
- Successful reads landed at almost perfectly regular intervals. That is a recovery timer, not bad flash.
- Whichever address you happened to ask for when the timer came up was the one you got — meaning no address was unreachable. The data wasn't lost, it was rationed.
The consequences drive the whole design here:
- Hammering gains nothing. Yield is set by elapsed time, not by number of attempts. Pacing requests costs you no data and spares the hardware.
- Bulk imaging is hopeless. 128 KiB per 2 minutes is about 1 KiB/s. Imaging 30 GB at that rate takes roughly a year.
- Targeted file recovery is entirely feasible. A 2 MB photo is 16 windows — about half an hour. If you can read the filesystem's allocation table, you know precisely which 128 KiB windows hold the files you actually care about, and you can skip everything else.
That last point is the whole trick: stop trying to image the drive, and go fetch specific files instead.
A bus reset is not a power cycle, and the difference can be everything. This
drive latched into a state where it refused all data reads, and no amount of
waiting cleared it — after 30 seconds of rest it returned 0 KiB. Immediately
after physically unplugging and replugging it, it returned 160 KiB. Every reset
performed by the kernel, by ddrescue, and by the other tools here is a bus
reset, which does not drop VBUS, so the latch never cleared. If your drive
behaves like this, resetting harder will never help and cutting power is the only
thing that will. usb-powercycle.py automates that through sysfs port control so
it can run unattended.
Read size is not a detail — it was worth 15x. Every tool here inherited a
32 KiB request size from ddrescue's default cluster, and nobody questioned it
for a long time. Sweeping it properly showed a sharp optimum:
bytes per READ(10) |
what the drive gave per power cycle |
|---|---|
| 32 KiB | 96–160 KiB, steady and small |
| 128 KiB | bursts of 1.9 MiB and 5.2 MiB |
| 256 KiB | a flat 256 KiB, less overall |
| one 8 MiB command | nothing — rejected outright |
Same drive, same minute, same code path. Too small and the controller rations
you; too large and it refuses the command entirely; in between it will sometimes
hand over megabytes. If throughput matters, sweep this parameter on your drive
before settling in for a long run — usb-powercycle.py --test --cmd N does it in
about a minute per value.
SCSI devices will tell you what's wrong, if you ask. Every tool treats a
failed read as opaque. REQUEST SENSE returns a precise reason — medium error,
hardware error, not ready, unit attention. usb-sense.py asks.
UNIT ATTENTION will silently eat your one read per cycle. After any reset,
a SCSI device answers its first command with CHECK CONDITION / UNIT ATTENTION, meaning "I was reset, acknowledge that". You clear it with REQUEST SENSE. The kernel does this for you; a naive raw driver does not — it sees a
failure and abandons the window. On a drive that gives you exactly one read per
enumeration, that means throwing away all of your data. If you write your own
raw reader, handle this or you will conclude the drive is deader than it is.
This is a last resort, for when the drive is not worth a professional lab and the alternative is losing everything. Set expectations accordingly:
- On the drive this was built for, the final haul was 19 recoverable photos. The rest of the 30 GB stayed out of reach at any sane time cost.
- Recovery of this kind is measured in days of unattended running, not minutes.
- If the data genuinely matters — the only copy of something irreplaceable — stop now and pay a data recovery lab. They can desolder the flash chips and read them directly, bypassing the broken controller entirely. Every hour you spend hammering a dying drive is a small risk to that option.
- Nothing here writes to the failing drive. But powering a failing drive up repeatedly is itself a risk. Decide once whether you're doing this.
- The freezer trick does not work on flash. It never did.
| script | what it does |
|---|---|
usb-sense.py |
Asks the drive, via SCSI REQUEST SENSE, why reads are failing. Start here. |
usb-maxread.py |
Finds the largest single read the drive will complete per enumeration. |
usb-rest.py |
Measures how yield varies with how long the drive rested. Tells you whether to pace. |
usb-rawread.py |
Probes whether cheap recovery (endpoint halt clear, bulk-only reset) works, versus a full port reset. |
| script | what it does |
|---|---|
usb-recovery.sh |
Chunked ddrescue sweep. Abandons a chunk that kills the drive and moves on, so it always makes forward progress. |
usb-targeted.sh |
Reads only a supplied list of byte offsets. Use once you know which windows matter. |
usb-harvest.py |
Talks USB Mass Storage directly, bypassing usb_storage and its expensive error handler. The fastest option on a timer-limited drive. |
usb-powercycle.py |
Cuts power to the USB port between reads via sysfs, rather than resetting the bus. The only thing that works on a drive that latches until VBUS drops. |
usb-scrape.sh |
Late-stage ddrescue scrape of the ragged edges of a band that's been yielding. |
| script | what it does |
|---|---|
usb-extract.py |
Walks FAT32 in the image and writes files out, marking incomplete ones .PARTIAL and unreadable ones .MISSING. No root, no mounting. |
usb-autocarve.py |
Re-runs the carver on a loop while a harvest is in progress, so recovered photos appear as they complete rather than only at the end. Rebuilds a contact sheet when the count rises. |
usb-verify.py |
Decodes every fully-read image and un-marks the ones that fail, so a harvester refetches them. A mapfile records what was read, not what is correct - only decoding closes that gap. |
usb-gaps.py |
Finds files that are partially recovered and emits the exact windows needed to finish them, ranked cheapest-first. A sequential sweep always leaves holes; this fills them precisely instead of re-sweeping. |
usb-pick.py |
Reads the FAT and emits the exact 128 KiB windows needed for the files you want, under a time budget. |
usb-carve.py |
Signature-carves images out of the rescued regions, for when the filesystem itself is gone. |
usb-merge.py |
Merges two rescue images and unions their mapfiles. |
usb-extract.py, usb-carve.py and usb-merge.py work on an image file only —
they never touch the drive, need no root, and are useful on any ddrescue image,
not just a USB one.
sudo apt install gddrescue python3 # python3-pil is optional, for usb-carve.py
git clone https://github.com/callumedmonds/usb-controller-recovery
cd usb-controller-recovery1. Identify the drive. Always use the by-id path — the /dev/sdX letter
changes every time a failing drive re-enumerates, and it will re-enumerate
constantly.
ls -l /dev/disk/by-id/ | grep -i usb
lsusb # note the ID, e.g. 03f0:b7072. Ask it what's wrong.
sudo python3 usb-sense.py --vid 03f0 --pid b7073. Try a chunked image. If the drive is only slow rather than rationed, this may be all you need.
sudo DEV=/dev/disk/by-id/usb-YOUR_DRIVE-0:0 WORK=$HOME bash usb-recovery.sh4. Get the files out of whatever you rescued.
python3 usb-extract.py ~/usb-rescue.img ~/recovered --mapfile ~/usb-rescue.mapIf step 3 is yielding almost nothing, the drive is rationing reads. Switch to
targeted recovery: use usb-pick.py to choose files and emit their windows, then
usb-harvest.py or usb-targeted.sh to fetch only those.
Every tool here is resumable and safe to Ctrl-C. The mapfile records exactly
what has been read, so re-running never repeats work.
The shell scripts take DEV and WORK from the environment:
sudo DEV=/dev/disk/by-id/usb-... WORK=$HOME bash usb-targeted.sh targets.txt 480The Python tools that talk to hardware take --vid/--pid (defaulting to the
HP v195b these were written against — you will need to change these). The
ones that work on images take --image and --map.
- Linux. The raw USB tools use
usbdevfsioctls and have no Windows or macOS equivalent here. gddrescuefor the shell scripts.- Python 3 with the standard library.
Pillowis optional and only used byusb-carve.py, to check whether carved images actually decode. - Root for anything touching the drive; none for anything touching an image.
These scripts were written to recover one specific failing drive, with AI assistance, over several days of measuring what that drive would actually do. They are shared because the failure mode is common and the existing advice for it is mostly wrong — "just use ddrescue" does not work when the device stops answering, and nothing out there explained why.
The hardware-facing tools were tested against exactly one sick drive. The
image-facing tools (usb-extract.py, usb-carve.py, usb-merge.py,
usb-pick.py) are tested against synthetic FAT32 images and are the most
portable part of this. Expect to adjust constants for your own drive; the
reasoning in each script's docstring matters more than its defaults.
Issues and improvements welcome, especially measurements from other failing controllers.
Written by Callum Edmonds (@callumedmonds).
If these tools got your files back and you'd like to leave a tip, there's a Ko-fi. Entirely optional — the licence is MIT either way, and I'd rather hear that it worked than be paid for it.
MIT — see LICENSE.