TravelAgent System

A dynamic conversational travel system built on a multi-agent architecture. It orchestrates open-domain web scraping, structured API search, personalized itinerary generation, and self-critique–driven refinement, enabling users to plan and book complete trips through an intelligent end-to-end workflow.

Table of Content

• Overview
• Required Services
• How to Setup
  • Create Conda Virtual Environment
  • Setup Docker
  • Setup Redis
  • Setup SearXNG
  • Setup Ollama
  • Setup Perplexica
  • Setup SearchAgent
• How to Run
  • Running the End-to-End Pipeline
  • Running Sub-agents Individually
• Analysis of the System
  • Evaluation Datasets Curation
    • User Cases and Queries (requests) Datasets
    • System Analysis Datasets
    • CriticAgent Analysis Datasets
  • Evaluation Scripts

Overview

TravelAgent is built on Microsoft’s AutoGen framework, the system uses a centrally orchestrated SelectorGroupChat architecture in which a PlanningAgent delegates work to a coordinated set of specialized agents:

Agent	Core Function	Model(s) Used
PlanningAgent	Central orchestrator; decomposes tasks and delegates to specialized agents	qwen2.5:7b
SelectorGroupChat	Routing layer that dynamically selects which agent should act next	qwen2.5:7b
UserProxyAgent	Interfaces with the user via input() and requests clarification whenever the input is missing, ambiguous, or internally inconsistent	/
WebScraperAgent	Retrieves and filters open-domain travel content using Perplexica + SearXNG, followed by multi-stage NLP filtering to ensure factuality, safety, relevance, and preference alignment	qwen2.5:7b (text generation) snowflake-arctic-embed:335m (embeddings)
SearchAgent	Performs agentic API calls to query structured travel data (flights, hotels, routes, POIs) from the Amadeus and Google Maps APIs	qwen2.5:7b
ContentGenerationAgent	Generate the final itinerary in Markdown using the filtered_content from the WebScraperAgent and the searched_results from the SearchAgent	gemma2:9b
CriticAgent	Evaluates itinerary for factuality, feasibility, safety, and hard-constraint compliance; returns ACCEPT / RE-WRITE	deepseek-r1:8b
TransactionAgent	Provides a lightweight placeholder booking confirmation to complete the end-to-end flow (triggered only after an ACCEPT decision from the CriticAgent)	qwen2.5:7b

To improve reliability, TravelAgent combines agent-level fallback mechanisms with a system-level self-critique loop.

The agent-level fallback mechanisms maintain data quality by:

• Triggering a re-scrape in the WebScraperAgent when fewer than five chunks are marked KEEP, and

• Escalating SearchAgent API call failures (e.g., user-requested travel dates earlier than today, mismatched amenity codes, invalid parameters, etc) to a human-in-the-loop path coordinated by the PlanningAgent.

  • In this path, the UserProxyAgent requests any missing information from the user or assists the user in troubleshooting using the error message returned by the SearchAgent.

The system-level self-critique loop ensures the final itinerary satisfies the user’s travel needs by:

• Having the CriticAgent evaluate each draft itinerary for information accuracy, logical feasibility, factual grounding, and explicit hard constraints. The critic outputs a structured raw_response containing:

  • a checklist of criteria met,

  • six evaluation scores (confidence, relevance, accuracy, safety, feasibility, personalization, each on a 0–5 scale),

  • a binary decision (ACCEPT or RE-WRITE),

  • a rationale explaining the decision, and

  • a targeted suggestion for how the plan should be revised.

• Routing the RE-WRITE decision through the PlanningAgent, which forwards the critic’s full raw_response to the ContentGenerationAgent for targeted regeneration conditioned on the critic’s guidance and the current evidence base (filtered_content and search_results).

A more detailed project report — which includes the system architecture, design decisions, and evaluation results — along with the accompanying iteration journal documenting the full development process, are available in the docs folder.

Required Services

TravelAgent requires the following services to be running:

Service	Role	Port	How it Runs	Description
Redis	State cache / intermediate artifact store	6379	Docker	Stores agent states, scraped content, filtered chunks, and runtime artifacts to enable cross-agent coordination.
SearXNG	Local meta-search engine	8080	Docker	Provides privacy-preserving meta-search results that Perplexica consumes before ranking and scraping.
Perplexica	Open-domain web search + agentic RAG	3000	Terminal	Performs agentic search using LLMs + embeddings; returns ranked URLs, extracted snippets, and content for scraping.
Ollama	Local open-source LLM runtime	11434	Terminal	Runs lightweight local inference for text generation, embeddings, and filtering across WebScraperAgent and CriticAgent.

How to Setup

The following instructions describe how to set up all core services required to run TravelAgent, including Python environments, Redis, SearXNG, Perplexica, Ollama, and API credentials for the SearchAgent.

1. Create Conda Virtual Environment

Set up the Python backend environment from the repository root.

conda create -n travelagent python=3.10 -y
conda activate travelagent
pip install -r backend/requirements.txt

---

2. Setup Docker

Docker is required to run services such as Redis and SearXNG.

After installing Docker Desktop, verify that Docker is running:

docker --version

You should see output similar to:

Docker version 25.x.x, build xxxx

If you prefer to test that containers can run correctly:

docker run hello-world

This should print a confirmation message indicating that Docker is installed and functioning properly.

---

3. Setup Redis

Redis is used as the system-wide state cache and intermediate artifact store.

# First-time setup (creates a new Redis container)
docker run -d --name redis -p 6379:6379 redis:7 

# Optional: test connectivity, should return PONG 
docker exec -it redis redis-cli ping  

# Starting Redis on subsequent runs
docker start redis

You may also install Redis Insight for a GUI view of stored data.

---

4. Setup SearXNG

SearXNG provides the meta-search backend used by Perplexica.

Navigate to the directory containing docker-compose.yml:

cd backend/agents/source/searxng
docker compose up -d

Check service availability:

http://localhost:8080

For manual installation instructions, refer to the SearXNG official documentation.

---

5. Setup Ollama

Ollama serves as the local LLM runtime, hosting all models used by the TravelAgent system.

After installing Ollama, start the local LLM runtime:

ollama serve

Download all required models:

ollama pull gemma2:9b
ollama pull qwen2.5:7b
ollama pull deepseek-r1:8b
ollama pull snowflake-arctic-embed:latest

Verify model availability:

curl http://localhost:11434/api/tags

---

6. Setup Perplexica

Perplexica performs open-domain web search + agentic RAG for the WebScraperAgent.

Navigate to the source directory (Note this project is using the travel-agent branch of this repo):

cd backend/autogen/agents/source

Clone the Travel-Agent version of the repo (only if not already present):

# Clone the Perplexica repository 
git clone git@github.com:Victoriakaey/Perplexica.git 

cd Perplexica

# Navigate to the travel-agent branch
git checkout travel-agent

Create a config.toml file in the Perplexica directory:

[GENERAL]
SIMILARITY_MEASURE = "cosine"
KEEP_ALIVE = "5m"

[MODELS.OPENAI]
API_KEY = ""

[MODELS.GROQ]
API_KEY = ""

[MODELS.ANTHROPIC]
API_KEY = ""

[MODELS.GEMINI]
API_KEY = ""

[MODELS.CUSTOM_OPENAI]
API_KEY = ""
API_URL = ""
MODEL_NAME = ""

[MODELS.OLLAMA]
API_URL = "http://localhost:11434"

[MODELS.DEEPSEEK]
API_KEY = ""

[MODELS.LM_STUDIO]
API_URL = ""

[API_ENDPOINTS]
SEARXNG = "http://localhost:8080"

Install and run Perplexica:

npm install
npm run build
npm run start

Verify (In another terminal):

http://localhost:3000

---

7. Setup SearchAgent

The SearchAgent integrates Amadeus (flights/hotels) and Google Maps (places, photos, routing).

You must create a .env file with the following keys.

AMADEUS_CLIENT_ID=
AMADEUS_CLIENT_SECRET=
GOOGLE_MAPS_API_KEY=

To Obtain the API keys, follow the instructions below:

Amadeus API Key

1. Visit: https://developers.amadeus.com

2. Create a free developer account.

3. After login, go to: Dashboard → My Apps → Create New App

4. Enable: Self-Service APIs (Flight Offers Search, Flight Inspiration Search, Hotel Search, etc.)

5. Once the app is created, Amadeus will provide two keys:

   • AMADEUS_CLIENT_ID
   • AMADEUS_CLIENT_SECRET

6. Copy them into your .env file.

Note: The SearchAgent uses the Self-Service Test Environment, so no paid account is required.

Google Maps API Key

1. Visit: https://developers.google.com/maps

2. Create or select a Google Cloud Project.

3. Enable the following APIs:

   • Places API
   • Places API (New)
   • Geocoding API
   • Directions API
   • Maps Routes API (if using routing)

4. Go to: APIs & Services → Credentials → Create API Key

5. Copy them into your .env file.

6. (Recommended) Restrict the key to:

   • HTTP referrers or IP addresses
   • Only the required APIs

Google setup docs: https://developers.google.com/maps/documentation/places/web-service/cloud-setup

Troubleshooting

If your SearchAgent fails to authenticate with Amadeus or Google Maps, refer to the following official guides:

• Amadeus – Quick Start Guide (how to create app, obtain keys, environment setup):
  https://developers.amadeus.com/self-service/apis-docs/guides/developer-guides/quick-start/

• Google Maps Platform – Get Started (create project, enable billing, enable APIs, create API key):
  https://developers.google.com/maps/get-started

Common issues to check:

• Make sure .env is placed in backend/ and loaded correctly.
• Ensure “Self-Service APIs” are enabled in your Amadeus app.
• On Google Cloud, verify you have enabled:
  • Places API (New)
  • Geocoding API
  • Directions API / Routes API (if routing is used)
• Ensure your Google Maps API key is not restricted in a way that blocks your backend usage.

How to Run

Running the End-to-End Pipeline

The TravelAgent system requires several background services to be running. You must have all services are running before launching the main process (main.py).

Here's a recommended order of operations running one by one:

1. Start Redis via Docker

   docker start redis     

2. Start SearXNG via Docker

   docker start searxng      

3. Start Ollama in one terminal (make sure all required models are already pulled)

   ollama serve

4. Start Perplexica in second terminal

   cd backend/agents/source/Perplexica # navigate to the Perplexica folder
   npm run start

5. Run TravelAgent in a third terminal

   conda activate travelagent # activate conda environment
   cd backend # navigate to the backend folder
   python -m autogen.main --case_num <case-num>

   Different case-num: 1 (baseline), 2 (fallback only), 3 (critic only), or 4 (full system - with fallback and critic)

---

Running Sub-agents Individually

To run each sub-agent, modify the test_agents.py accordingly and run

python -m autogen.test.test_agents --test <test-mode>

Different test-mode: "webscraper", "webscraper_fallback", "nlp_filter", "llm_filter", "search", "search_fallback", "content", "critic"， "transaction"

Analysis of the System

Evaluation Datasets Curation

A. User Cases and Queries (requests) Datasets

Data are curated using the selected 10 user cases from an original 53 user cases and the queries (request) are curated using the generate_user_query.

B. System Analysis Datasets

The 40 sets of plans, run time, number of rounds, search mode, total kept and dropped content count are curated by running the end-to-end system via main.py script. Logs of the end-to-end system ran could be found in the logs folder.

python -m autogen.main --case_num <case-num>

Different case-num: 1 (baseline), 2 (fallback only), 3 (critic only), or 4 (full system - with fallback and critic)

C. CriticAgent Analysis Datasets

The 40 sets of critic agent's scores (confidence, relevance, accuracy, safety, feasibility, and personalization) and decisions (ACCEPT or RE-WRITE) are curated by running the critic_agent_evaluation.py script. Logs of the critic agent evaluation process could be found in the logs folder.

python -m autogen.evaluation.ground_truth_curation.critic_agent_evaluation

The 40 sets of ground truth, i.e., human decisions (ACCEPT or RE-WRITE) and human scores (relevance, accuracy, safety, feasibility, and personalization) are curated by 1 human annotator by running the human_evaluation.py script. Logs of the human evaluation process could be found in the logs folder.

python -m autogen.evaluation.ground_truth_curation.human_evaluation

---

Evaluation Scripts

Based on the metrics, analysis scripts were written to test the efficiency of the system and each sub-agents.

To display analysis including: (1) human scores, (2) critic agent scores, (3) human decisions, (4) critic agent decisions, (5) number of rounds each agents were ran, (6) runtimes, and (7) correlation and confunsion matrix based on analysis of human and critic agent's scores and decisions; run the following command to run the analysis.py script:

python -m autogen.evaluation.analysis.analysis

To run the correlation and confunsion matrix alone, by running the correlation_confusion_matrix.py script:

python -m autogen.evaluation.analysis.correlation_confusion_matrix

To run analysis on the WebScraperAgent and the SearchAgent based on: the total dropped and kept content from the filter mechanism from the WebScraperAgent and different search mode conducted by running the web_search_scraper_analysis.py script:

python -m autogen.evaluation.analysis.web_search_scraper_analysis

Note that the output should be outputed in the web_search_analysis_output