searxng_search

A Python client library crafted by JamePeng (jame_peng@sina.com) for seamless interaction with your self-hosted SearXNG instance. This library empowers you to programmatically perform various searches (text, images, videos, news, etc.) against your local or LAN-deployed SearXNG server, giving you full control over your search data without relying on external APIs.

---

Features

• Connect to Local/LAN SearXNG: Easily specify the base URL (IP address and port) of your self-hosted SearXNG instance, supporting both local and network-accessible deployments.
• Text Search: Perform general web searches and retrieve structured results.
• Structured Output: Receive search results in a clean, parseable JSON format.
• Robust Error Handling: Comprehensive error management for network issues, server responses, and parsing failures.
• Customizable: Extendable to support more SearXNG categories and search types.

---

Installation

Prerequisites

• Python 3.9+: Ensure you have a compatible Python version installed.
• Docker & Docker Compose: (Recommended for setting up SearXNG) Make sure Docker and Docker Compose are installed on your system.

Install the searxng_search Library

You can install searxng_search by building it from source.

From Source

Clone the repository (or create the project structure):

git clone https://github.com/jamepeng/searxng_search.git
cd searxng_search

Alternatively, manually create the following directory structure and files within your project root:

searxng_search_package/
├── searxng_search/
│   ├── __init__.py
│   ├── searxng_search.py
│   ├── exceptions.py
│   └── utils.py
├── test/
│   ├── demo.py
├── .gitignore
├── pyproject.toml
├── CHANGELOG.md
└── README.md

Install build tools:

pip install build wheel setuptools

Build and Install:

pip install .

Setting up SearXNG with Docker (Recommended)

Before using searxng_search, you need a running SearXNG instance. The recommended container setup now uses SearXNG's official Compose template directly from the main searxng repository.

The old searxng-docker repository layout is no longer the preferred setup path. If you already have an older searxng-docker deployment, see the migration notes below.

1. Create the SearXNG deployment directory

mkdir -p ./searxng/core-config/
cd ./searxng/

2. Download the official Compose template and environment file

curl -fsSL \
  -O https://raw.githubusercontent.com/searxng/searxng/master/container/docker-compose.yml \
  -O https://raw.githubusercontent.com/searxng/searxng/master/container/.env.example

cp -i .env.example .env

The official Compose template currently defines two services:

• core: the SearXNG web application container.
• valkey: the persistent Valkey cache backend used by SearXNG.

A simplified view of the generated docker-compose.yml looks like this:

name: searxng

services:
  core:
    container_name: searxng-core
    image: docker.io/searxng/searxng:${SEARXNG_VERSION:-latest}
    restart: always
    ports:
      - ${SEARXNG_HOST:+${SEARXNG_HOST}:}${SEARXNG_PORT:-8080}:${SEARXNG_PORT:-8080}
    env_file: ./.env
    volumes:
      - ./core-config/:/etc/searxng/:Z
      - core-data:/var/cache/searxng/

  valkey:
    container_name: searxng-valkey
    image: docker.io/valkey/valkey:9-alpine
    command: valkey-server --save 30 1 --loglevel warning
    restart: always
    volumes:
      - valkey-data:/data/

volumes:
  core-data:
  valkey-data:

3. Configure .env

Open .env and adjust the values for your environment:

nano .env

Common examples:

# Use a specific SearXNG image version instead of latest.
# SEARXNG_VERSION=latest

# Localhost-only access, useful for local development.
SEARXNG_HOST=127.0.0.1
SEARXNG_PORT=8080

# LAN / public access: listen on all interfaces.
# Make sure your firewall allows this port before exposing it.
# SEARXNG_HOST=0.0.0.0
# SEARXNG_PORT=8080

With the default port, SearXNG will be available at:

http://localhost:8080

If you expose it on your LAN, replace localhost with the server IP address, for example:

http://192.168.1.100:8080

4. Enable JSON output in core-config/settings.yml

searxng_search works best with SearXNG's JSON output. Add or update the following block in core-config/settings.yml:

use_default_settings: true

search:
  safe_search: 0
  autocomplete: ""
  favicon_resolver: ""
  default_lang: ""
  formats:
    - html
    - json
    - csv
    - rss

At minimum, make sure json is included under search.formats; otherwise requests with format=json may be rejected.

For a private/local instance, you may also want to set a stable secret key:

server:
  secret_key: "change-me-to-a-long-random-string"

5. Start SearXNG

docker compose up -d

Check that the containers are running:

docker compose ps

Expected service names:

searxng-core
searxng-valkey

View logs if something goes wrong:

docker compose logs -f core

Open a troubleshooting shell inside the SearXNG container:

docker compose exec -it --user root core /bin/sh -l

Stop the services:

docker compose down

Update the running images:

docker compose down
docker compose pull
docker compose up -d

6. Test the JSON API

Open this URL in your browser or run it with curl:

curl "http://localhost:8080/search?q=python&format=json"

If the response is JSON, the instance is ready for searxng_search.

Optional: Manual single-container setup

For advanced users or quick testing, you can run SearXNG directly without Compose:

mkdir -p ./searxng/config/ ./searxng/data/
cd ./searxng/

docker run --name searxng -d \
  -p 8888:8080 \
  -v "./config/:/etc/searxng/" \
  -v "./data/:/var/cache/searxng/" \
  docker.io/searxng/searxng:latest

This starts SearXNG at:

http://localhost:8888

Optional: Migrate from old searxng-docker

If you previously used the old searxng-docker repository, create a fresh deployment using the steps above, stop the old services, then move your existing config files into the new core-config directory:

mv ./searxng-docker/searxng/* ./searxng/core-config/

If your old configuration references Redis or Valkey by hostname, update it to match the new Compose service name, usually valkey or searxng-valkey.

---

Usage Examples

Here's how you can use the searxng_search library in your Python code:

from searxng_search.searxng_search import SearXNGSearch
from searxng_search.exceptions import RequestException, ParsingException, SearXNGSearchException

import logging

# Configure logging for better visibility
logging.basicConfig(level=logging.INFO, format='%(asctime)s - %(name)s - %(levelname)s - %(message)s')
logger = logging.getLogger(__name__)

# --- IMPORTANT ---
# Replace this with the actual base URL of your running SearXNG instance.
# Default Compose setup: http://localhost:8080
# LAN example: http://192.168.1.100:8080
SEARXNG_BASE_URL = "http://localhost:8080"

def perform_text_search(keywords: str, language: str = "en-US"):
    """Demonstrates performing a text search with error handling."""
    logger.info(f"Performing text search for: '{keywords}' on {SEARXNG_BASE_URL}")

    # Set verify=False only if you put SearXNG behind HTTPS with a self-signed certificate.
    with SearXNGSearch(base_url=SEARXNG_BASE_URL, timeout=20, verify=True, retries=3, backoff_factor=0.1) as client:
        try:
            # Perform a general text search, requesting JSON format, limit to 8 results
            results = client.text(keywords, category="general", language=language, format="json", max_results=8)

            if results:
                logger.info(f"Successfully retrieved {len(results)} results for '{keywords}':")
                for i, result in enumerate(results):
                    logger.info(f"  Result {i+1}:")
                    logger.info(f"    Title: {result.get('title', 'N/A')}")
                    logger.info(f"    URL:   {result.get('href', 'N/A')}")
                    logger.info(f"    Body:  {result.get('body', 'N/A')}...")
            else:
                logger.info(f"No results found for '{keywords}'.")

        except RequestException as e:
            logger.error(f"Network or HTTP error during search: {e}")
        except ParsingException as e:
            logger.error(f"Error parsing SearXNG response: {e}")
        except ValueError as e:
            logger.error(f"Invalid input parameter: {e}")
        except SearXNGSearchException as e:
            logger.error(f"A general SearXNG search error occurred: {e}")
        except Exception as e:
            logger.critical(f"An unexpected critical error occurred: {e}", exc_info=True)
    print("-" * 50) # Separator for clarity

if __name__ == "__main__":
    perform_text_search("Python programming best practices", language="en-US")
    perform_text_search("MCP是什么？", language="zh-CN")
    perform_text_search("nonexistent query xyz123") # Example for no results
    perform_text_search("") # Example for ValueError

---

API Reference (Planned)

SearXNGSearch(base_url: str, headers: dict | None = None, timeout: int | None = 30, verify: bool = True, retries: int = 3, backoff_factor: float = 0.5)

Initializes the client.

• base_url (str): The full URL to your SearXNG instance (e.g., "http://192.168.1.100:8080/" or "https://your.domain.com/").
• headers (dict, optional): Custom HTTP headers for requests.
• timeout (int, optional): Request timeout in seconds. Defaults to 30.
• verify (bool, optional): Whether to verify SSL certificates. Set to False only for self-signed or custom HTTPS certificates if you encounter SSL errors; keep True for production with valid certificates. Defaults to True.
• retries (int, optional): Number of times to retry a failed HTTP request. Defaults to 3.
• backoff_factor (float, optional): A factor by which to multiply the retry delay. The delay will be backoff_factor * (2 ** (retry_count - 1)). Defaults to 0.5.

SearXNGSearch.text(keywords: str, category: str = "general", language: str = "en-US", pageno: int = 1, format: Literal["json", "html"] = "json", max_results: int | None = None, safesearch: int = 0) -> list[dict[str, str]]

Performs a text search.

• keywords (str): The search query.
• category (str, optional): SearXNG category (e.g., "general", "science", "it", "images", "videos", "news"). Defaults to "general".
• language (str, optional): Language parameter for SearXNG (e.g., "en-US", "zh-CN"). Defaults to "en-US".
• pageno (int, optional): Page number of results to fetch. Defaults to 1.
• format (Literal["json", "html"], optional): Desired output format from SearXNG. "json" is highly recommended for structured data. Defaults to "json".
• max_results (int, optional): Maximum number of results to return from the client side. If None, all results from the requested page are returned.
• safesearch (int,optional): Filter search results based on safe search level. 0: Off, 1: Moderate, 2: Strict. Defaults to 0.

---

Exceptions

• SearXNGSearchException: Base exception for all library errors.
• RequestException: Raised for HTTP communication issues (network errors, timeouts, 4xx/5xx status codes).
• ParsingException: Raised when SearXNG's response cannot be decoded or parsed as expected (e.g., invalid JSON, unexpected HTML structure).
• ValueError: Raised for invalid input parameters provided to library methods.

---

Contributing

Contributions are welcome! If you find a bug, have a feature request, or want to improve the code, please feel free to:

• Open an Issue: Describe the bug or feature you'd like to see.
• Submit a Pull Request: Fork the repository, create a new branch, make your changes, and submit a pull request.

---

License

This project is licensed under the MIT License - see the LICENSE file for details.

---

Author

JamePeng (jame_peng@sina.com)