IPO Prospectus Scraper — Automated Financial Data Extraction via Gemini AI

A pipeline for automatically extracting financial data from Indonesian IPO prospectus PDF files using the Gemini API, and converting the results into a structured CSV dataset.

Overview

This project automates the extraction of financial data from Indonesian IPO prospectus documents, transforming unstructured PDF text into an organized dataset. Manually extracting this information is extremely time-consuming because each PDF can span hundreds of pages with highly inconsistent formatting across different issuers.

To overcome this challenge, the pipeline leverages the Gemini API—specifically utilizing the Gemini 2.5 Flash model—to intelligently locate and parse key financial variables regardless of their structural placement within the document. The extracted data is currently stored as a clean, structured dataset and serves as a ready-to-use foundation for future analysis or applications.

Repository Structure

ipo-prospectus-scraper/
│
├── assets/
│   └── terminal_output.png       # Example of pipeline execution output
├── extract_prospectus.py         # Main extraction script (PDF → JSONL via Gemini API)
├── convert_jsonl_to_csv.ipynb    # Converter: JSONL → structured CSV
├── ipo_extraction_results.csv    # Final extraction output (76 prospectus files)
├── hasil_ekstraksi_batch.jsonl   # Raw extraction output from Gemini API
├── error_log.txt                 # Log of files that failed extraction
├── .env                          # API key template (fill in before running)
└── README.md

Extracted Variables

For each prospectus, the pipeline attempts to extract the following fields:

Field	Description
kode_saham	Official BEI ticker code
nama	Full company name
total_aset	Total assets (Rp)
total_ekuitas	Total equity (Rp)
laba_bersih	Net profit (Rp)
eps	Earnings per share (Rp)
harga_ipo	IPO offering price (Rp)

Requirements

pip install google-generativeai pdfplumber pandas python-dotenv

Setup

1. Clone the repository

git clone https://github.com/YOUR_USERNAME/ipo-prospectus-scraper.git
cd ipo-prospectus-scraper

2. Fill in your Gemini API key

Open .env and replace the placeholder with your actual key:

GOOGLE_API_KEY=YOUR_ACTUAL_API_KEY_HERE

Get your free API key at: https://aistudio.google.com/apikey

3. Prepare your prospectus folder

Create a folder named prospektus/ in the project directory and place all IPO prospectus PDF files inside it:

prospektus/
├── prdl-prospektus-2026.pdf
├── rans-prospektus-2026.pdf
└── ...

PDF files can be downloaded from:

• idx.co.id → Perusahaan Tercatat → Prospektus
• e-ipo.co.id → IPO → Detail emiten → Prospektus

4. Configure the folder path

Open extract_prospectus.py and update the path if needed:

FOLDER_PDF = r"prospektus"   # relative path, or use absolute path

How to Run

Step 1 — Run the extraction script

python extract_prospectus.py

The script will process each PDF file one by one and save the results to hasil_ekstraksi_batch.jsonl. Files that were successfully processed in a previous run will be skipped automatically (resume mode).

Step 2 — Convert JSONL to CSV

Open and run convert_jsonl_to_csv.ipynb in Jupyter Notebook. The final output will be saved as ipo_extraction_results.csv.

Notes on Extraction Accuracy

• Extraction accuracy is approximately 80–90% for text-based PDFs
• Scanned PDFs (image-based) will be skipped automatically and logged in error_log.txt
• Fields not found in a document are recorded as null — Gemini is instructed never to fabricate values
• The BACH ticker required manual correction for the EPS field due to a unit parsing error

Rate Limiting

The free tier of the Gemini API has a daily request limit. The script includes a 10-second delay between requests to stay within rate limits. If you hit the limit mid-run, simply re-run the script — it will resume from where it left off.

Data Source

Prospectus PDFs were sourced from:

• Indonesia Stock Exchange: https://www.idx.co.id
• e-IPO platform: https://www.e-ipo.co.id