OneDrive Photo Importer

An automated command-line tool that recursively discovers and downloads photos from Microsoft OneDrive to local storage, safely removing each photo from cloud storage after verified download completion. Built with Python and Microsoft Graph API.

---

Table of Contents

• Overview
• Features
• Architecture
• Tech Stack
• Prerequisites
• Azure App Registration Setup
• Installation
• Configuration
• Usage
• Project Structure
• Testing & Verification
• Troubleshooting
• License

---

Overview

OneDrive Photo Importer automates the migration and backup of photos stored in Microsoft OneDrive personal accounts. The agent navigates the entire OneDrive folder hierarchy using Microsoft Graph API v1.0, streams original photo files to a local directory via direct blob storage URLs, and deletes cloud copies only after verifying local file integrity and size.

Designed for resilience during large media migrations, the application includes automatic silent token refreshing, rate-limit backoff (HTTP 429 retry handling), persistent progress tracking, and interactive device flow authentication.

---

Features

• Recursive Folder Discovery: Automatically traverses all nested subdirectories starting from the OneDrive root (/me/drive/root/children).
• Original Quality Preservation: Bypasses Graph API image transcoding by downloading directly from pre-signed Azure Blob URLs (@microsoft.graph.downloadUrl).
• Safe Cloud Deletion: Issues Graph API DELETE requests (/me/drive/items/{id}) only after local file download and non-zero byte size validation complete successfully.
• Date & Range Filtering: Filters photos by EXIF metadata (photo.takenDateTime), filename timestamp patterns (YYYYMMDD_HHMMSS), or item creation timestamps.
• Resume & Progress Tracking: Maintains state in .onedrive_progress.json to resume interrupted imports without re-downloading completed items.
• Dry-Run Mode: Evaluates matching files and calculates total download payload size without performing network writes or cloud deletions.
• Rate-Limit & Transient Retry Handling: Automatically intercepts HTTP 429 response status codes and network timeouts, adhering to server-requested Retry-After intervals.
• Broad Format Support: Detects standard raster formats, raw camera files, and Apple/Samsung image extensions (e.g., HEIC, HEIF, DNG, CR2, NEF, ARW).

---

Architecture

The following diagram illustrates the token acquisition, recursive file discovery, streaming download, and deletion lifecycle executed by onedrive_photo_importer.py.

graph TD
    subgraph Local Environment
        Agent["OneDrive Photo Importer Agent<br/>(onedrive_photo_importer.py)"]
        Cache[".onedrive_token_cache.json"]
        Progress[".onedrive_progress.json"]
        Logs["onedrive_importer.log"]
        Disk["Local Storage<br/>(./onedrive_photos)"]
    end

    subgraph Microsoft Identity & Graph Services
        AzureAuth["Microsoft Identity Platform<br/>(OAuth2 Device Flow)"]
        GraphAPI["Microsoft Graph API v1.0<br/>(graph.microsoft.com)"]
        BlobStore["Azure Blob Storage<br/>(Download URLs)"]
    end

    Agent -->|1. Authenticate / Refresh Token| Cache
    Cache -->|Token Request| AzureAuth
    AzureAuth -->|Access Token| Agent
    Agent -->|2. Walk Folders & Filter Photos| GraphAPI
    Agent -->|3. Query Download Metadata| GraphAPI
    GraphAPI -->|Return @microsoft.graph.downloadUrl| Agent
    Agent -->|4. Stream Binary Data| BlobStore
    BlobStore -->|Save File| Disk
    Agent -->|5. Verify Non-zero Bytes| Disk
    Agent -->|6. Delete Cloud Item| GraphAPI
    Agent -->|7. Persist State & Logs| Progress
    Agent -->|Write Log Entries| Logs

Loading

---

Tech Stack

Languages & Runtimes

• Python: 3.8+

Dependencies

• msal (>=1.28.0): Microsoft Authentication Library for Python (OAuth2 Device Code Flow and Token Cache Management).
• requests (>=2.31.0): HTTP library for Microsoft Graph REST API calls and binary payload streaming.
• tqdm (>=4.66.0): Terminal progress indicator for media processing loops.

External APIs

• Microsoft Graph API: v1.0 (https://graph.microsoft.com/v1.0)

---

Prerequisites

• Python: Version 3.8 or higher installed locally.
• Microsoft Account: Personal Microsoft account (Outlook.com, Hotmail, Live) with access to OneDrive.
• Azure Application Registration: Azure Client ID configured with delegated Files.ReadWrite permissions.

---

Azure App Registration Setup

Before running the application, register a public client application in the Azure Portal to enable Microsoft Graph API access.

1. Sign in to the Azure Portal.
2. Navigate to App registrations and select New registration.
3. Configure registration settings:
   • Name: OneDrive Photo Importer (or preferred name).
   • Supported account types: Personal Microsoft accounts only.
   • Redirect URI: Leave blank.
4. Select Register.
5. Copy the Application (client) ID displayed on the Overview page.
6. Configure authentication settings:
   • Go to Authentication > Add a platform > Mobile and desktop applications.
   • Select the redirect URI: https://login.microsoftonline.com/common/oauth2/nativeclient.
   • Select Configure.
   • Under Advanced settings, set Allow public client flows to Yes.
   • Save changes.
7. Configure API permissions:
   • Go to API permissions > Add a permission > Microsoft Graph.
   • Select Delegated permissions.
   • Search for and select Files.ReadWrite.
   • Select Add permissions.

---

Installation

# 1. Clone the repository
git clone https://github.com/Omiiii04/OneDrive-Agent.git
cd OneDrive-Agent

# 2. Create a virtual environment
python -m venv venv

# 3. Activate the virtual environment
# Windows (PowerShell):
.\venv\Scripts\Activate.ps1
# Linux / macOS:
source venv/bin/activate

# 4. Install dependencies
pip install -r requirements.txt

---

Configuration

1. In-Script Configuration

Open onedrive_photo_importer.py and update the configuration variables located near the top of the file:

# ═══════════════════════════════════════════
#   CONFIGURATION  ← Edit these two lines
# ═══════════════════════════════════════════
CLIENT_ID       = "YOUR_CLIENT_ID_HERE"    # Application (client) ID from Azure Portal
DOWNLOAD_FOLDER = "./onedrive_photos"      # Local directory path for saved photos
# ═══════════════════════════════════════════

2. Runtime State & Configuration Artifacts

File	Type	Description
onedrive_photo_importer.py	Source	Main application script and entry point.
requirements.txt	Dependency Manifest	Required Python packages and version specifications.
.onedrive_token_cache.json	Generated	Encrypted MSAL token cache generated after authentication.
.onedrive_progress.json	Generated	Progress tracker containing lists of processed item IDs (done_ids) and failure details.
onedrive_importer.log	Generated	Plain-text execution log storing timestamped runtime events.

---

Usage

Authentication Flow

On initial execution, onedrive_photo_importer.py triggers an MSAL Device Code authentication flow. The console displays a URL and user code:

--------------------------------------------------------------
To sign in, use a web browser to open the page https://www.microsoft.com/link and enter the code XXXXXXXX to authenticate.
--------------------------------------------------------------

Navigate to https://www.microsoft.com/link, input the code, and sign in. Tokens are saved to .onedrive_token_cache.json and silently refreshed on subsequent executions.

---

Command Line Interface

python onedrive_photo_importer.py [FLAGS]

Supported Arguments

Flag	Argument	Default	Description
--folder	PATH	./onedrive_photos	Destination directory for downloaded photos.
--dry-run	Flag	False	Scans and outputs photo listings without downloading or deleting files.
--no-delete	Flag	False	Downloads photos to local storage while preserving cloud copies on OneDrive.
--resume	Flag	False	Loads.onedrive_progress.json and skips previously completed photo IDs.
--limit	INT	None	Restricts execution to process a maximum ofN matching photos.
--date	YYYY-MM-DD	None	Filters photos created or taken on an exact specified date.
--date-from	YYYY-MM-DD	None	Filters photos created or taken on or after the specified start date.
--date-to	YYYY-MM-DD	None	Filters photos created or taken on or before the specified end date.

---

Execution Examples

Dry-Run Execution

Scan drive contents and display total count and size without modifying storage:

python onedrive_photo_importer.py --dry-run

Test Single Item Download

Download a single photo without deleting from OneDrive:

python onedrive_photo_importer.py --limit 1 --no-delete

Filter by Date Range

Download and delete photos within a specific calendar year:

python onedrive_photo_importer.py --date-from 2024-01-01 --date-to 2024-12-31

Resume Interrupted Import

Resume a cancelled or interrupted execution using saved progress:

python onedrive_photo_importer.py --resume

---

Supported File Formats

Image items are identified via Graph API facets (image, photo) or matched against the following extension list:

Category	File Extensions
Standard Images	.jpg, .jpeg, .png, .gif, .bmp, .webp
Mobile Formats	.heic, .heif
Uncompressed / High Quality	.tiff, .tif
RAW Camera Formats	.raw, .dng, .cr2, .nef, .arw, .orf, .rw2

---

Project Structure

OneDrive-Agent/
├── .gitignore                   # Git exclusion configuration
├── Architecture.png             # Existing architecture diagram asset
├── LICENSE                      # Apache License 2.0 legal text
├── README.md                    # Project documentation
├── image/                       # Documentation image resources
│   └── README/
│       └── 1784627234745.png
├── onedrive_photo_importer.py   # Primary executable application script
└── requirements.txt             # Python dependency specifications

---

Testing & Verification

Static syntax analysis and compilation verification can be executed locally:

# Verify Python syntax and compile bytecode
python -m py_compile onedrive_photo_importer.py

Note: Execution testing of Graph API network operations requires an active Azure Application Client ID and user authentication.

---

Troubleshooting

Unconfigured Client ID

Symptom: Script exits with status code 1 displaying CLIENT_ID is not set!. Cause: CLIENT_ID in onedrive_photo_importer.py retains the default placeholder value ("YOUR_CLIENT_ID_HERE"). Resolution: Replace "YOUR_CLIENT_ID_HERE" with your Azure Application (client) ID.

Rate Limiting (HTTP 429)

Symptom: Console outputs Rate-limited — waiting X s before retry.... Cause: Microsoft Graph API threshold exceeded. Resolution: The script automatically waits for the duration specified in the HTTP Retry-After header before retrying requests. No manual intervention is required.

Authentication Failures

Symptom: RuntimeError: Authentication failed: ... or RuntimeError: Could not start device flow. Cause: Invalid Client ID, missing public client flow configuration in Azure Portal, or network connectivity issues. Resolution: Confirm Allow public client flows is set to Yes under Authentication > Advanced settings in the Azure Portal.

---

License

This project is licensed under the Apache License 2.0. See the LICENSE file for details.