Last-Stage Capacity Reduction Library

Patterns for progressively narrowing neural network representations in the later stages of a model.

---

Core Principle

Early layers capture low-level features; later layers must compress these into task-specific representations. Strategic narrowing at this stage forces beneficial compression, improves regularization, and reduces inference cost without significant accuracy loss.

Grounded in: Huang et al., "Exploring Architectural Ingredients of Adversarially Robust DNNs" (NeurIPS 2021).

Installation

pip install last-stage-capacity

Or install from source:

git clone https://github.com/johnmwhitman/last-stage-capacity.git
cd last-stage-capacity
pip install -e .

Library Structure

• capacity_reduction/: Core library (last-stage capacity detection and reduction)
• examples/: Usage examples
• tests/: Test suite

Usage

from last_stage_capacity import LastStageCapacity

# Analyze a model's last-stage capacity
analyzer = LastStageCapacity(model)
report = analyzer.benchmark()

Benchmarks

Benchmark scripts and reference numbers are being computed against torch-pruning and the timm model zoo. The latest reproducible results live in benchmarks/.

Initial scope: ResNet-50 on ImageNet-1k, comparing the four reduction strategies (bottleneck, progressive narrowing, SE-reduction, conditional) against the unmodified baseline. Numbers will land in the next minor release.

Related Work

• agent-mesh — fleet-native agent orchestration for OpenCode. MIT.
• Accumulated — John's Substack on autonomous systems and the seven-layer framework.

Citation

If this library is useful in your work:

@software{whitman2026laststagecapacity,
  author = {Whitman, John},
  title = {last-stage-capacity: Progressive Narrowing for Inference-Efficient CNNs and Transformers},
  year = {2026},
  url = {https://github.com/johnmwhitman/last-stage-capacity}
}

Maintainer

John Whitman — Director of Product Management, America's Car-Mart. Writes at Accumulated.

License

Apache-2.0. See LICENSE for the full text.