Heritage Explorer

A Flask web app that helps travelers discover Tamil Nadu's heritage sites through an AI chatbot and photo-based landmark recognition.

Overview

Heritage Explorer combines a retrieval-augmented chatbot with an interactive map and an image-based landmark identifier to help tourists plan visits to Tamil Nadu's temples, forts, waterfalls, and beaches.

Who it's for: Travelers researching Tamil Nadu heritage sites, and developers interested in a practical example of combining RAG chatbots, computer vision, and mapping in one small Flask app.

Key features:

• AI chatbot answering travel/tourism questions, grounded in curated place descriptions via a FAISS vector index
• Automatic translation of chatbot answers into the query's detected language
• "Snap & Discover": upload a photo and get it classified against known Tamil Nadu landmarks using a trained Keras model
• Interactive Mapbox globe of Tamil Nadu (token-based; live search disabled by default to conserve API quota — see Activating the Map)
• Curated image carousel and FAQ section

Tech Stack

Backend

• Python 3.12, Flask 3
• LangChain + LangChain Community (RAG orchestration)
• FAISS (vector similarity search)
• Sentence-Transformers (all-MiniLM-L6-v2 embeddings)
• Ollama running llama3.2 locally (the chat LLM)
• TensorFlow / Keras (image classification)
• googletrans (response translation), langid (language detection)

Frontend

• Jinja2 templates, vanilla JavaScript
• Bootstrap 5, Boxicons
• Mapbox GL JS

Project Structure

Heritage Explorers/
├── app.py                     # Flask entry point and routes
├── requirements.txt
├── .env.example
├── labels.txt                 # Class labels for the image classifier
├── places_model.h5            # Trained Keras landmark classifier
├── faiss_index/                # Prebuilt vector index (used at runtime)
│   ├── index.faiss
│   └── index.pkl
├── services/
│   ├── LLMService.py           # RAG chain, language detection, translation
│   └── ImageService.py         # Landmark image classification
├── scripts/                    # Offline/maintenance scripts (not run by the app)
│   ├── build_vector_index.py   # Rebuilds faiss_index/ from Places/
│   └── download_training_images.py
├── static/                     # CSS, images, icons, GIFs
├── templates/
│   └── chatbot.html
├── Places/                     # Source PDFs/DOCX used to build the vector index (gitignored)
└── TamilNadu_Places/            # Raw training images for the classifier (gitignored)

Prerequisites

• Python 3.12 (or 3.10+)
• Ollama installed and running locally, with the llama3.2 model pulled:

  ollama pull llama3.2
  

• A free Mapbox access token — see Activating the Map below
• ~3 GB free disk space if you also want the raw training data (TamilNadu_Places/, Places/) — not required just to run the app

Activating the Map

The "Explore the Map" section is built on Mapbox GL JS, which is token-based: every map render and every geocoding search draws from your Mapbox account's quota. Without a token, the map tiles simply won't load.

1. Sign up for a free account at mapbox.com
2. Go to Access Tokens and copy your default public token (starts with pk.)
3. Paste it into your .env file as MAPBOX_TOKEN=pk.your_token_here
4. Mapbox's free tier includes a generous number of free map loads and geocoding requests per month at the time of writing — check mapbox.com/pricing for current limits, since these can change

Note: the live location-search feature (searchLocation() in chatbot.html) is disabled by default in this repo to avoid burning through your quota while developing — clicking "Search" on the map shows an alert instead of calling the Mapbox Geocoding API. To enable real search, open templates/chatbot.html, find searchLocation(), and delete the two alert(...) / return; lines guarding the fetch(url) call. The base map itself (the globe you see on page load) is not gated by this — it always consumes a map load from your quota on every page render.

Getting Started — How to Run Locally

1. Clone the repository and enter the project folder

   git clone <your-repo-url>
   cd heritage-explorer
   

2. Create and activate a virtual environment

   python -m venv venv
   # Windows
   venv\Scripts\activate
   # macOS/Linux
   source venv/bin/activate
   

3. Install dependencies

   pip install -r requirements.txt
   

4. Set up environment variables

   cp .env.example .env
   

   Then edit .env and fill in:

   • MAPBOX_TOKEN — your Mapbox access token
   • OLLAMA_MODEL — defaults to llama3.2, change only if you're using a different local model

5. Make sure Ollama is running with the model pulled (see Prerequisites above).

6. Start the app

   python app.py
   

7. Open your browser at http://127.0.0.1:5000

The first chatbot query will be slower while the embedding model downloads and loads. Subsequent queries are faster.

Features

AI Travel Chatbot

Ask questions about Tamil Nadu heritage sites in the chat panel. Answers are generated by retrieving relevant passages from a prebuilt FAISS index (sourced from curated PDFs/DOCX per district) and passing them to a local llama3.2 model via Ollama. If your question isn't in English, the detected language is identified automatically and the answer is translated back into that language.

[Screenshot placeholder — add image here]

Snap & Discover (Image Classification)

Upload a photo of a landmark and the app classifies it against a small set of known Tamil Nadu sites using a trained Keras model, returning the predicted location and a confidence score.

Current limitation: the shipped model (places_model.h5) only recognizes 5 classes (see labels.txt), even though training images exist for ~39 places. See Future Improvements.

[Screenshot placeholder — add image here]

Interactive Map

A Mapbox globe centered on Tamil Nadu, with curated quick links to jump straight to well-known sites. Live text search is disabled by default to conserve Mapbox API quota — see Activating the Map in Prerequisites to enable it with your own token.

[Screenshot placeholder — add image here]

FAQ

An accordion of frequently asked questions about how the chatbot and site work.

API Endpoints

Method	Route	Description
GET	/	Renders the main Heritage Explorer page
POST	/predict	Body: {"query": "<question>"}. Returns ["hi", "<answer>"]. Returns 400 if query is missing or empty.
POST	/upload	Multipart form with a file field (image). Returns {"location": "<name>", "confidence": <float>}, or 400 with {"error": "..."} if no file is provided.

Future Improvements

• Retrain places_model.h5 on the full ~39-place dataset already collected in TamilNadu_Places/ — the model currently only distinguishes 5 landmarks
• Improve cross-lingual retrieval quality: the FAISS index is embedded with an English-centric model, so non-English queries sometimes retrieve less relevant context than their English equivalents
• Add automated tests for the /predict and /upload endpoints

License

MIT License. See below.

MIT License

Copyright (c) 2026 Heritage Explorer contributors

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.