Multi-Modal Document Intelligence System

Technical Evaluation — AI GenMat

This project is split into three independent tasks that together form a document intelligence pipeline. The goal is to take a PDF, extract its content in multiple modalities (pages, images, text), and run analysis on each.

---

Overview

Task	Script	What it does
Part 1	Task1.py	PDF processing & page/image extraction
Part 2	Task2.py	Image processing & enhancement
Part 3	Task3.py	NLP & text analysis

---

Requirements

Install all dependencies with:

pip install PyPDF2 pdfplumber Pillow opencv-python numpy nltk matplotlib wordcloud tqdm

tqdm is optional — everything still runs without it, just without progress bars.

---

Task 1 — PDF Processing & Page Extraction

Script: Task1.py

Handles all the PDF-level work: pulling out specific pages as individual PDFs, extracting embedded images, and writing a metadata report.

What it does:

• Reads a PDF and extracts pages [1, 2, 3, 4, 5, 7, 10, 15, 16, 17, 18, 19], saving each as its own page_N.pdf
• Extracts images from pages [10, 15, 16, 17, 18, 19] using pdfplumber (crops at 300 DPI)
• Also includes a fallback PyPDF2 XObject extractor for embedded images that pdfplumber might miss
• Generates a plain-text + JSON metadata report with file info, page count, extraction stats, and output file sizes

Usage:

python Task1.py <input.pdf> [output_directory]

# Example
python Task1.py research_paper.pdf ./output

Output structure:

output/
  extracted_pages/
    page_1.pdf
    page_2.pdf
    ...
  extracted_images/
    page10_image1.png
    ...
  pdf_metadata_report.txt
  pdf_metadata_report.json

---

Task 2 — Image Processing & Enhancement

Script: Task2.py

Takes images (typically from Task 1 output) and runs them through a set of transformations and enhancements.

Transformations applied to each image:

• Grayscale conversion
• Resize to 800×600 (letterboxed to preserve aspect ratio)
• Rotation by 45°
• Histogram equalization (V-channel only for color images, to avoid hue distortion)
• Gaussian blur (5×5 kernel)
• Canny edge detection
• Morphological operations: erosion, dilation, opening, closing
• 150×150 thumbnail

Also generates a comparison grid PNG showing the original + all versions side by side.

Usage:

# Process a single image
python Task2.py image.png [output_directory]

# Process a whole directory of images
python Task2.py ./extracted_images/ ./output

Output structure:

output/
  processed_images/
    <image_name>/
      <name>_grayscale.png
      <name>_resized_800x600.png
      <name>_rotated_45deg.png
      <name>_histogram_eq.png
      <name>_gaussian_blur.png
      <name>_canny_edges.png
      <name>_morph_erosion.png
      <name>_morph_dilation.png
      <name>_morph_opening.png
      <name>_morph_closing.png
      <name>_thumbnail_150x150.png
      <name>_comparison.png

---

Task 3 — NLP & Text Analysis

Script: Task3.py

Extracts text from pages 1–4 of the PDF and runs a full NLP analysis pipeline on it.

Pipeline steps:

1. Text extraction via pdfplumber
2. Cleaning — strips noise, collapses whitespace, removes short standalone numbers
3. Sentence + word tokenization (NLTK punkt)
4. Stopword removal + custom academic stopword list
5. WordNet lemmatization
6. Statistics: word counts, sentence lengths, vocabulary richness, top-20 words
7. POS tagging — first 3 sentences of each page
8. Named Entity Recognition — persons, organizations, locations
9. Visualizations: word frequency bar chart + word cloud
10. Topic summary and full report

Usage:

python Task3.py <input.pdf> [output_directory]

# Example
python Task3.py research_paper.pdf ./output

Output structure:

output/
  nlp_analysis/
    cleaned_text.txt
    word_frequency_distribution.png
    word_cloud.png
    nlp_analysis_report.txt

---

Running the Full Pipeline

You can run all three tasks in sequence on the same PDF:

python Task1.py research_paper.pdf ./output
python Task2.py ./output/extracted_images/ ./output
python Task3.py research_paper.pdf ./output

Each script is also independently runnable if you only need one part of the pipeline.

---

Notes

• All scripts write to ./output by default if no output directory is given.
• Task 1 tries pdfplumber first for image extraction. If that yields nothing (some PDFs embed images as XObjects rather than page-level resources), the extract_images_pypdf2() function can be used as a fallback.
• Task 3 auto-downloads required NLTK data on first run. This requires an internet connection the first time.
• Tested with Python 3.9+.