Hiro Translation

An 8B-parameter LLM patent translation service, deeply optimized for China-US patent writing standards.

Features | Model Hub | Benchmark | Installation | Quick Start | MCP & Web UI | Configuration | API Reference | Examples | Related Documents | 简体中文

Hiro Translation workflow

📄 Related Documents

• Model Card — TranslationGPT-1.2 overview (for Hugging Face and weight consumers)
• MCP Server — HIRO-Translation MCP tool (translation_text), Streamable HTTP, Cursor integration
• Web UI Demo — Gradio demo for streaming translation via HTTP API
• Disclaimer — Terms of use, limitations of liability, and data-handling responsibilities
• Notice — Copyright and trademark notice

💡 Features

• Patent-Domain Adaptation: Supports zh2en (Chinese to English) and en2zh (English to Chinese) translation with professional terminology matching China-US patent application standards.
• Sliding Context Window: Automatically splits long patent texts into fine-grained segments while utilizing a sliding context window to maintain strict semantic consistency across paragraphs.
• High-Performance Streaming: The main translation endpoint delivers real-time responses via the NDJSON streaming protocol, eliminating waiting anxiety for long document processing.
• Robust Breakpoint Resumption: Supports both local file-based and Redis-based distributed caching. Provides multi-level rollback strategies for the same task_id to withstand network jitters or service crashes.
• Quality Assurance (QA) Guardrails: When details=true is enabled, the service automatically detects and flags translation defects such as loops/repetitive text, language mixing, and missing critical digits.

📦 Model Hub

The core model is 8B (8 billion parameters)—compact enough for cost-efficient GPU deployment while retaining strong patent-domain performance.

Source	Model Weights URL	Description
Hugging Face	patsnap/TranslationGPT-1.2 (8B)	Official 8B open-weights checkpoint; deploy with vLLM for high-concurrency inference.

🎯 Benchmark Results

The tables below report Chinese → English (ZH→EN) and English → Chinese (EN→ZH) evaluation results. All scores are on a 0–100 scale.

Dataset: Scores are measured on an internal proprietary patent dataset (private, not publicly released).

Metrics: Except for columns explicitly marked Acc (accuracy-based), dimension scores use BLEU. Overall is the arithmetic mean of all dimension columns in that table (excluding Overall itself).

📊 ZH→EN (Chinese to English)

Model	Overall	Normal Words (BLEU)	Special Words (BLEU)	Terminology Acc	Terminology Consistency (Acc)	Normal Sentences (BLEU)	Special Sentences (BLEU)	Paragraph (BLEU)	Chapter (BLEU)	Patent Writing Standards (Acc)
Hiro-Translation	70.5	73.4	97.9	61.1	56.5	69.6	78.9	72.7	38.5	86.2
Deepseek-v3	67.6	66.3	98.3	65.0	56.0	62.5	73.9	67.2	33.6	86.0
doubao-1.5-lite	69.5	68.8	99.0	63.2	58.9	66.8	80.3	74.9	40.3	73.5
doubao-lite-32k	66.7	67.9	98.1	61.0	57.0	67.4	78.4	74.3	39.3	57.0
Google API	69.4	75.2	99.1	61.6	58.1	71.0	85.2	76.2	41.1	57.0
GPT-OSS-20b	60.5	65.3	95.4	54.9	49.9	60.3	70.8	62.7	28.9	56.7
GPT-OSS-120b	63.0	67.0	93.2	56.0	49.1	62.5	71.8	63.4	33.1	70.5

📊 EN→ZH (English to Chinese)

Model	Overall	Normal Characters (BLEU)	Normal Sentences (BLEU)	Special Sentences (BLEU)	Paragraph (BLEU)	Chapter/Document (BLEU)
Hiro-Translation	65.8	68.3	68.3	72.4	70.6	49.3
Deepseek-v3	65.0	67.3	68.0	74.4	71.5	44.0
doubao-1.5-lite	66.2	71.4	70.4	74.4	71.7	42.9
doubao-lite-32k	64.7	67.8	68.1	72.5	71.3	44.0
Google API	64.6	72.4	68.3	70.8	68.9	42.6
GPT-OSS-20b	56.9	59.5	57.8	65.6	58.9	42.5
GPT-OSS-120b	59.4	62.5	60.7	66.6	62.8	44.6

---

📥 Installation

For the most stable and production-ready experience, this project adopts a hybrid deployment architecture:

1. Inference Backend (vLLM): Highly recommended to run via Docker to bypass complex GPU driver setups and C++ compilation environments.
2. Business API Gateway (Flask): Managed locally via uv for blazing-fast environment isolation and dependency management.

Python version: 3.10.12 exactly. Other 3.10 patch releases and 3.11+ are not supported.

1. Set Up API Gateway (via uv)

# Clone the repository
git clone https://github.com/your-org/HiroTranslation.git
cd HiroTranslation

# Install uv if you haven't already
# Linux / macOS: curl -LsSf https://astral.sh/uv/install.sh | sh

# Create and activate a Python 3.10.12 virtual environment using uv
uv venv --python 3.10.12
source .venv/bin/activate

# Install gateway dependencies at lightning speed
uv pip install -r requirements.txt

2. Set Up Inference Backend (via Docker)

Ensure that Docker and NVIDIA Container Toolkit are installed and correctly configured on your host machine to support GPU containers.

---

🛠️ Quick Start

1. Launch vLLM Inference Engine

⚠️ Why Docker? Installing vLLM natively via pip frequently leads to CUDA version mismatches and compilation errors. The official Docker image comes pre-packaged with all pre-compiled optimization kernels.

Replace /path/to/hiro-translation-checkpoint with your actual local model weights directory:

docker run --gpus all \
  -v /path/to/hiro-translation-checkpoint:/data \
  -p 8004:8000 \
  --ipc=host \
  --name vllm-translation-backend \
  -d \
  vllm/vllm-openai:latest \
  --model /data \
  --served-model-name translationgpt_v1_2 \
  --tensor-parallel-size 1 \
  --max-model-len 16384

Parameter Breakdown:

• -p 8004:8000: Maps the container's default vLLM port 8000 to the host's 8004 port.
• --ipc=host: Shares the host's inter-process communication memory, essential for preventing NCCL out-of-memory errors during LLM inference.
• --served-model-name: Must match the model.name specified in the gateway's configs/default.yaml.

Verify if the inference backend is healthy and ready:

curl http://127.0.0.1:8004/v1/models

2. Launch Flask Translation Gateway

1. Review and edit configs/default.yaml to ensure model.url correctly points to the vLLM host mapping port (defaults to http://127.0.0.1:8004/v1).
2. Run the startup script from the project root:

uv run python main.py

The gateway service will listen on http://0.0.0.0:8978 by default.

Gateway Health Check:

curl http://127.0.0.1:8978/compute/hiro_translation/health
# Expected response: {"status":"OK"}

---

🔌 MCP & Web UI

After the gateway is up, you can use the optional MCP tool server (for Cursor / MCP clients) or the Gradio Web UI (browser demo). Both call the same Flask gateway over HTTP.

MCP Server

Exposes the translation_text tool over Streamable HTTP (default http://127.0.0.1:8980/mcp; see configs/default.yaml → mcp).

Prerequisites: gateway running (python main.py).

# From project root
python -m mcp_server.server
# or
bash run_mcp.sh

Docs: README_MCP.md — tool parameters, Cursor config (.cursor/mcp.json.example), tests (python test_mcp_translation.py).

Web UI (Gradio Demo)

Browser UI with streaming source/target pairs, progress, and one-click copy.

Prerequisites: gateway running (python main.py).

cd demo
python app.py

Open http://127.0.0.1:7868 (port in configs/default.yaml → demo.port).

Docs: demo/README.md — UI layout, streaming vs whole-document mode, gateway URL settings.

---

⚙️ Configuration

The service loads configs/default.yaml by default. Gateway, MCP, and Gradio demo share this file (ports, URLs, timeouts). Override the file path with:

export CONFIG_PATH=/path/to/my-config.yaml
uv run python main.py

configs/default.yaml Explained

model:
  url: http://127.0.0.1:8004/v1           # vLLM inference backend endpoint
  name: translationgpt_v1_2               # Model identifier shared between Gateway and vLLM

server:
  host: 0.0.0.0
  port: 8978

gateway:
  base_url: null                          # null → http://127.0.0.1:<server.port>; used by MCP & demo

mcp:
  host: 0.0.0.0
  port: 8980
  path: /mcp
  timeouts:
    low: 300
    medium: 600
    high: 900

demo:
  host: 0.0.0.0
  port: 7868

cache:
  type: local                             # Cache provider: local (file) | redis (distributed)
  prefix: translation_gpt_cache           # Cache key prefix
  incremental: true                       # Enable incremental breakpoint resumption
  clear_on_start: false                   # Flush historical cache when service starts
  rollback_strategy: minimal              # Rollback strategy on failure: minimal | smart | conservative | disabled
  local:
    dir: null                             # Leave empty (null) to use the system temporary directory
    max_files: 1000
  redis:
    host: redis-rd-qa.patsnap.private
    port: 6379
    db: 0

---

🔌 API Reference

Base URL: http://127.0.0.1:8978

Endpoint Overview

Method	Path	Description	Response Format
GET	/compute/hiro_translation/health	Gateway health check	JSON
POST	/compute/hiro_translation	Streaming segmented translation (Main API)	NDJSON (Stream)
POST	/compute/hiro_translation/wo_window	Direct translation (No sliding window, single full request)	JSON
POST	/compute/hiro_translation/clear_cache	Global cache clearance	JSON
POST	/compute/hiro_translation/clear_task_cache	Clear cache for a specific task_id	JSON

---

Core Endpoint Specification

1. Streaming Segmented Translation POST /compute/hiro_translation

• Request Header: Content-Type: application/json
• Response Header: Content-Type: text/plain (Each line is a separate valid JSON object, i.e., NDJSON)

Request Body (Payload):

Parameter	Type	Required	Default	Description
text	string	Yes	-	The raw, long-form patent source text to be translated.
lang	string	Yes	-	Language direction: zh2en or en2zh.
max_tokens_window	int	No	4000	Sliding window token threshold, range: 512–8192.
details	bool	No	false	Whether to enable translation defect and QA quality flags.
enable_cache	bool	No	true	Toggle breakpoint translation caching.
task_id	string	No	-	Unique task identifier. Auto-generated based on text hash if omitted.

Streaming Response Payload (Single line example):

{
  "text_original": "本发明公开了一种档案管理文件储存用分类标识装置。",
  "text_translated": "The invention discloses a classification identification device for archive management file storage.",
  "progress": "1/5",
  "prompt": "<Optional> Complete prompt structure returned only when details=true",
  "is_repeat": false
}

Error Response (Stream-compatible single line JSON):

{"ERROR": "Invalid language pair specified", "TYPE": "ValueError"}

2. Full Document Translation (No Window) POST /compute/hiro_translation/wo_window

Request Body: Identical to the main endpoint, except it ignores max_tokens_window and cache parameters.

Response Example:

{
  "text_original": "Hello, world.",
  "text_translated": "你好，世界。"
}

---

💡 Examples

Python (Asynchronous Streaming Consumer)

import json
import requests

url = "http://127.0.0.1:8978/compute/hiro_translation"
payload = {
    "text": "本发明公开了一种档案管理文件储存用分类标识装置。",
    "lang": "zh2en",
    "details": False,
    "enable_cache": True,
    "task_id": "patent-demo-task-001"
}

# Stream=True is mandatory for chunk processing
with requests.post(url, json=payload, stream=True, timeout=600) as resp:
    resp.raise_for_status()
    for line in resp.iter_lines(decode_unicode=True):
        if line:
            chunk = json.loads(line)
            progress = chunk.get("progress")
            translated_text = chunk.get("text_translated")
            print(f"[{progress}] -> {translated_text}")

cURL (Command Line Streaming)

curl -N -X POST http://127.0.0.1:8978/compute/hiro_translation \
  -H "Content-Type: application/json" \
  -d '{
    "text": "本发明公开了一种档案管理文件储存用分类标识装置。",
    "lang": "zh2en",
    "enable_cache": false
  }'

---

📂 Repository Layout

.
├── configs/
│   └── default.yaml          # Global gateway and cache configuration file
├── translation_gpt/
│   ├── __init__.py
│   └── req_model.py          # Core routing logic handling vLLM requests
├── main.py                   # Flask gateway server entrypoint
├── mcp_server/               # MCP tool server (FastMCP, Streamable HTTP)
├── demo/                     # Gradio Web UI
├── run_mcp.sh                # MCP launcher
├── README_MCP.md             # MCP tool docs
├── requirements.txt          # Python deps (gateway + MCP + demo)
├── MODEL_CARD.md             # TranslationGPT-1.2 model card (HF / weights)
├── DISCLAIMER.md             # Disclaimer and terms of use
├── NOTICE.md                 # Copyright and trademark notice
└── README.md