Gepeto Test

Tools to test and benchmark LLMs hosted at the Barcelona Supercomputing Center via the Gepeto service (https://gepeto.bsc.es/).

About

Gepeto provides access to various open-source language models through an OpenAI-compatible API. This project demonstrates how to:

• List available models
• Test API connectivity
• Benchmark all models with response time measurements

Quick Start

Get API Key

1. Visit https://gepeto.bsc.es/
2. Go to Settings > Account
3. Generate your API key

Configure

# Install dependencies
uv pip install python-dotenv requests tqdm

# Save API key
echo "GEPETO_API_KEY=your_key_here" > .env

# Run benchmark
uv run benchmark_models.py

Usage

# Benchmark all models
uv run --with python-dotenv --with requests --with tqdm benchmark_models.py

# Benchmark specific models
uv run --with python-dotenv --with requests --with tqdm benchmark_models.py --models qwen3:30b mistral:7b

# Custom query
uv run --with python-dotenv --with requests --with tqdm benchmark_models.py --query "Explain AI" --timeout 120

# Save results
uv run --with python-dotenv --with requests --with tqdm benchmark_models.py --save

# Test API
uv run --with python-dotenv --with requests test_api.py

Options

Flag	Description
--query, -q	Custom test query
--timeout, -t	Request timeout in seconds (default: 60)
--save, -s	Save results to JSON file
--models, -m	Specific models to benchmark
--verbose, -v	Show verbose output
--no-progress	Disable progress bar

Requirements

• Python 3.13+
• Gepeto API key