✦ D2 Storage

Zero-config, project-scoped cloud asset storage — a secure gateway on top of Cloudinary.

Part of the TheDreamBitLabs open-source ecosystem · Built by Dhairya Darji

Why D2 Storage? · Quick Start · Cloudinary Setup · API Reference · AI Integration · Deployment

What Is D2 Storage?

D2 Storage is a self-hosted storage microservice that sits in front of Cloudinary. You deploy it once on your own server, create isolated project workspaces with their own API keys and CORS rules, and upload files with a single fetch() call — from any app, any framework, any language.

No SDK needed. No Cloudinary account exposed to the client. Just a clean, secure REST API.

❓ Why D2 Storage and Not Cloudinary Directly?

"I already have Cloudinary — why would I use this?"

Great question. Here's what D2 Storage adds that raw Cloudinary cannot:

Problem with raw Cloudinary	What D2 Storage solves
🔴 API Key/Secret must stay server-side — you can't use it from a browser safely	✅ D2 issues per-project bearer tokens — safe to embed in any client app
🔴 One global bucket — all projects share the same namespace	✅ Every project gets its own isolated root folder under `dreambit/<your-root>/`
🔴 No origin restriction — any server can upload to your account if they get the key	✅ Project-level CORS whitelist — only approved origins can upload
🔴 Integrating Cloudinary in every new project means re-reading docs every time	✅ Once deployed, every new project is just `POST /api/upload` with a bearer token — done in 2 minutes
🔴 Explaining Cloudinary SDK setup to an AI assistant is slow and error-prone	✅ Built-in AI Integration Copilot — copy a pre-built system prompt and get working SDK code instantly
🔴 File naming conflicts in shared buckets	✅ Timestamp-appended filenames prevent collisions automatically
🔴 `.zip`, `.pdf`, `.csv` are uploaded as the wrong resource type	✅ Automatic `raw` type routing so downloads work correctly in-browser

The philosophy: "A 500-line storage service that removes 30 minutes of setup from every future project."

✨ Features

🗂️ Project Workspaces — spin up isolated storage containers with unique API keys
🔑 Hashed API Keys — cryptographically hashed bearer tokens, never stored in plaintext
🌐 Per-Project CORS — restrict uploads to your approved origins only
📁 Auto Folders — upload to a folder that doesn't exist yet and it's provisioned automatically
🤖 AI-Ready Copilot — generate a master integration prompt per project, paste into any LLM
💅 Admin Dashboard — beautiful UI at /dashboard with 24-hour session management
📖 In-App API Docs — full interactive reference at /docs, no external site needed
📜 Format Integrity — raw/video/image types routed correctly for proper download behavior

🚀 Quick Start

Prerequisites

Node.js ≥ 18
A free Cloudinary account (10 GB free)
A server with a persistent filesystem — Railway, Render, VPS, or Docker (see Deployment)

1. Clone

git clone https://github.com/TheDreamBitLabs/d2-storage.git
cd d2-storage

2. Install

npm install

3. Set up environment variables

cp .env.example .env.local

Open .env.local and fill in your values:

# Cloudinary (required) — from cloudinary.com → Settings → API Keys
CLOUDINARY_CLOUD_NAME=your_cloud_name
CLOUDINARY_API_KEY=your_api_key
CLOUDINARY_API_SECRET=your_api_secret

# Master admin password (required) — protects the /dashboard panel
# This is NOT a Cloudinary key — pick any strong passphrase you want
API_KEYS=your_master_password

# Your deployment URL (recommended)
# The AI Integration Prompt and /docs page use this as the base URL.
# Change this when you go to production!
NEXT_PUBLIC_APP_URL=http://localhost:3001

4. Run

npm run dev

App starts on http://localhost:3001 · Dashboard at /dashboard · Docs at /docs

☁️ Cloudinary Setup (Free 10 GB)

Cloudinary's free tier gives you 10 GB storage + 25 GB bandwidth/month — plenty for hackathons and indie projects.

Step 1 — Create a free account

Go to cloudinary.com/users/register/free — no credit card required.

Step 2 — Get your API credentials

Log in → Settings → API Keys
Copy three values into your .env.local:

Env Variable	Where to find it
`CLOUDINARY_CLOUD_NAME`	Settings → Account → Cloud name
`CLOUDINARY_API_KEY`	Settings → API Keys → API Key
`CLOUDINARY_API_SECRET`	Settings → API Keys → API Secret (click reveal)

Step 3 — Set the API key role

When Cloudinary asks you to select a role for a new API key, choose Master or Full Access. D2 Storage needs permission to upload, delete, and list assets across folders.

Step 4 — Folder structure (zero manual setup)

D2 Storage manages Cloudinary folders automatically. When you create a project workspace, folders are provisioned on first upload:

cloudinary-bucket/
  dreambit/
    your-project-root/
      avatars/         ← auto-created on first upload
      docs/
      uploads/
      ...

You never need to touch the Cloudinary dashboard for folder management.

Free tier limits

Resource	Free tier
Storage	10 GB
Bandwidth / month	25 GB
Transformations / month	25,000 credits

🔐 Environment Variables

Variable	Required	Description
`CLOUDINARY_CLOUD_NAME`	✅	Your Cloudinary cloud name
`CLOUDINARY_API_KEY`	✅	Your Cloudinary API key
`CLOUDINARY_API_SECRET`	✅	Your Cloudinary API secret
`API_KEYS`	✅	Master admin password(s). Comma-separate for multiple: `pass1,pass2`
`NEXT_PUBLIC_APP_URL`	Recommended	Your public deployment URL — used in AI prompt + docs. Defaults to `window.location.origin` on client. Update this when you host on a custom domain.
`CONVEX_DEPLOYMENT`	❌ Optional	Convex deployment name (e.g. `dev:fancy-llama-123`). Only needed if you run or build with Convex backend (reserved for future multi-tenant hosted version). Not needed for basic self-hosting.
`NEXT_PUBLIC_CONVEX_URL`	❌ Optional	Convex deployment HTTP URL. Only needed if Convex backend is active.

📡 API Reference

All requests need a project-scoped bearer token in the header:

Authorization: Bearer sk_your_project_api_key

Upload a file

POST /api/upload?projectId=<PROJECT_ID>
Content-Type: multipart/form-data

Field	Required	Description
`file`	✅	The file to upload. Max 10 MB via server.
`folder`	optional	Subfolder path. Auto-provisioned if it doesn't exist.

curl -X POST "https://your-app.railway.app/api/upload?projectId=proj_abc123" \
  -H "Authorization: Bearer sk_proj_live_8f3d9a..." \
  -F "file=@logo.png" \
  -F "folder=brand-assets"

Response:

{
  "success": true,
  "file": {
    "id": "dreambit/my-app/brand-assets/logo_1717711234567",
    "name": "logo_1717711234567.png",
    "url": "https://res.cloudinary.com/.../logo_1717711234567.png",
    "size": 45120,
    "type": "image/png",
    "created_at": "2026-06-06T18:00:00Z"
  }
}

List files

GET /api/files?projectId=<PROJECT_ID>&folder=<FOLDER>&cursor=<CURSOR>&limit=<LIMIT>

Delete a file

DELETE /api/files/<PROVIDER_ID>?type=<image|video|raw>

List subfolders

GET /api/folders?projectId=<PROJECT_ID>

Create a subfolder

POST /api/folders?projectId=<PROJECT_ID>
Content-Type: application/json

{ "name": "avatars" }

For the full interactive reference, go to /docs in your running app.

🤖 AI Integration Copilot

D2 Storage has an AI-Ready Copilot built into every project dashboard. This is one of its most powerful features.

How it works

Open your project at /dashboard/<project-id>
Click "AI Integration Prompt"
A complete system prompt is generated with:
- Your Project ID and API key pre-filled
- Your deployment base URL (from NEXT_PUBLIC_APP_URL)
- All endpoint specs in the correct format
Paste it into Claude, ChatGPT, Gemini, Cursor, or any AI tool
Ask "Write me a JavaScript helper to upload files" — get working code in seconds

Note on the base URL: The AI prompt reads from NEXT_PUBLIC_APP_URL. If it shows localhost:3001 after deployment, update that env variable to your real domain and redeploy.

🏗️ Project Structure

d2-storage/
├── src/
│   ├── app/
│   │   ├── api/
│   │   │   ├── upload/         # POST /api/upload
│   │   │   ├── files/          # GET + DELETE /api/files
│   │   │   ├── folders/        # GET + POST /api/folders
│   │   │   └── projects/       # GET + POST /api/projects
│   │   ├── dashboard/
│   │   │   ├── page.tsx        # Project list + session auth
│   │   │   └── [id]/           # Per-project workspace
│   │   ├── docs/               # In-app API documentation
│   │   ├── globals.css         # All design tokens + theme
│   │   └── page.tsx            # Public landing page
│   ├── components/ui/          # Custom UI components
│   ├── lib/
│   │   ├── auth.ts             # Master key validation
│   │   ├── cloudinary.ts       # Cloudinary SDK wrapper
│   │   ├── cors.ts             # Project-scoped CORS enforcement
│   │   ├── crypto.ts           # API key hashing utilities
│   │   ├── db.ts               # JSON flat-file database layer
│   │   └── utils.ts            # Shared utilities
│   └── sdk/                    # Embeddable client SDK (WIP)
├── data/                       # Runtime data — gitignored
│   ├── projects.json           # Project registry
│   └── api_keys.json           # Hashed API key store
├── convex/                     # Convex schema (reserved for future hosted version)
│   └── schema.ts
├── docs/                       # Internal design docs
├── .env.example                # Environment variable template
└── package.json

About the database

D2 Storage uses a local JSON flat-file database (data/projects.json and data/api_keys.json) — no PostgreSQL, no Redis, no external database required. This is intentional: it keeps setup to zero.

⚠️ The data/ folder must persist between restarts. This means:

✅ Works great on Railway, Render, VPS, Docker (persistent filesystem)

❌ Will lose data on Vercel, Netlify, AWS Lambda (ephemeral serverless filesystem)

If you want to deploy to Vercel/serverless, swap src/lib/db.ts for any key-value store (Vercel KV, Upstash Redis, PlanetScale, etc.). The interface is straightforward to replace.

Note on Convex: The convex/ directory and convex npm package are present but not yet active in this version. The schema is defined and ready — Convex is planned for a future multi-tenant hosted version of D2 Storage. It is not required for self-hosting.

🎨 Tech Stack

Layer	Technology
Framework	Next.js 16 App Router
Language	TypeScript 5
UI	React 19
Styling	Tailwind CSS 4 + Shadcn UI + Base UI
Storage Driver	Cloudinary Node SDK v2
Database	Local JSON flat-file — zero external dependencies
Auth	Native Node.js `crypto` module — hashed bearer tokens
Icons	Lucide React

🚀 Deployment

Important: D2 Storage needs a persistent filesystem to store data/projects.json and data/api_keys.json. Choose a platform that supports this.

✅ Railway (Recommended — easiest)

Push your fork to GitHub
Go to railway.app → New Project → Deploy from GitHub
Select your repo
Add environment variables in Railway dashboard:
- CLOUDINARY_CLOUD_NAME, CLOUDINARY_API_KEY, CLOUDINARY_API_SECRET
- API_KEYS
- NEXT_PUBLIC_APP_URL → set to your Railway-generated URL (e.g. https://d2-storage.up.railway.app)
Deploy — Railway handles the persistent volume automatically ✅

✅ Render

New Web Service → connect your repo
Build command: npm install && npm run build
Start command: npm start
Add the same environment variables
Set NEXT_PUBLIC_APP_URL to your Render URL

✅ VPS / Docker

npm run build
npm start     # runs on port 3001

Or with Docker:

FROM node:20-alpine
WORKDIR /app
COPY . .
RUN npm ci && npm run build
EXPOSE 3001
CMD ["npm", "start"]

Mount a persistent volume at /app/data to survive container restarts.

⚠️ Vercel (Not recommended without DB swap)

Vercel's serverless functions have an ephemeral filesystem — your data/ folder will be wiped on each cold start. If you still want to use Vercel, replace src/lib/db.ts with Vercel KV or Upstash Redis. The db.ts interface is minimal and straightforward to swap out.

🛡️ Security

Master password (API_KEYS) is validated server-side only, never returned to the client
Project API keys are stored as one-way hashes — the plaintext is shown exactly once, at creation time
CORS enforcement runs on every API request before any Cloudinary call
data/ is gitignored — your project registry and key hashes never get committed
Dashboard sessions expire after 24 hours automatically via localStorage TTL

📦 Roadmap

Signed upload URLs — client-to-Cloudinary direct uploads for files > 10 MB
Image transformations — proxy Cloudinary transform params (?w=300&h=300)
Usage analytics — storage used, monthly bandwidth, uploads today
Multi-key support — multiple admin keys with role separation
SDK package — publish @dreambitlabs/storage to npm
Convex backend — multi-tenant hosted version with user accounts

🤝 Collaboration & Contributing

D2 Storage is an open-source project, and we welcome contributions, suggestions, and ideas of all kinds! Whether you want to fix a bug, suggest a new feature, or improve the documentation, we'd love to collaborate with you.

💡 How to Get Involved

Suggest an Idea or Feature: Have a recommendation to make D2 Storage better? Please open an Issue with the tag enhancement describing your idea, use cases, and how it could work.
Report a Bug: If you notice unexpected behavior, file an Issue with a clear description, steps to reproduce, and screenshots if possible.
Ask Questions or Discuss: Want to talk about architecture, deployment, or share what you've built? Open a thread in GitHub Discussions (or file a general issue if discussions aren't active).
Write Code (Pull Requests):
1. Fork the repository and create your branch from main: git checkout -b feat/my-amazing-feature
2. Implement your changes. Make sure to keep the PR focused and minimal (this project is designed to be lightweight).
3. Verify your changes compile correctly by running: npm run build
4. Commit your changes with descriptive messages: git commit -m "feat: add support for signed URLs"
5. Push to your branch and open a Pull Request!

🎯 Scope & Design Philosophy

We want to keep D2 Storage extremely fast, lightweight, and zero-config. Before writing complex integrations or adding external databases, please open an issue to discuss your proposal so we can align on the implementation approach!

📄 License

MIT — use it, fork it, ship it. Attribution appreciated but not required.

Built with ☕ by Dhairya Darji · TheDreamBitLabs open-source ecosystem

Name		Name	Last commit message	Last commit date
Latest commit History 9 Commits
convex		convex
public		public
src		src
.env.example		.env.example
.gitignore		.gitignore
README.md		README.md
components.json		components.json
eslint.config.mjs		eslint.config.mjs
next.config.ts		next.config.ts
package-lock.json		package-lock.json
package.json		package.json
postcss.config.mjs		postcss.config.mjs
tsconfig.json		tsconfig.json

Folders and files

Latest commit

History

Repository files navigation