Skip to content

Commit 5341b30

Browse files
serjikibmserjikibm
authored
Update documentation to reflect latest models (#78)
* add new 4.2 page, no url changes Signed-off-by: serjikibm <serjikgd@ca.ibm.com> * add url updates to default 4.2 Signed-off-by: serjikibm <serjikgd@ca.ibm.com> * add sunset banner Signed-off-by: serjikibm <serjikgd@ca.ibm.com> * make annoucement bar text and links bold to make them stand out Signed-off-by: serjikibm <serjikgd@ca.ibm.com> * add/merge updates from PR 77 Signed-off-by: serjikibm <serjikgd@ca.ibm.com> * add new icon to 4.2 side nav item Signed-off-by: serjikibm <serjikgd@ca.ibm.com> --------- Signed-off-by: serjikibm <serjikgd@ca.ibm.com>
1 parent 0bc6b75 commit 5341b30

11 files changed

Lines changed: 307 additions & 15 deletions

File tree

‎docusaurus.config.ts‎

Lines changed: 14 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -76,7 +76,10 @@ const config: Config = {
7676
'@docusaurus/plugin-client-redirects',
7777
{
7878
// Pre-wired empty: add entries here if any URLs change after the IBM.com cutover.
79-
redirects: [],
79+
redirects: [
80+
// The legacy /models/granite URL always lands on the latest generation.
81+
{from: '/models/granite', to: '/models/granite4-2'},
82+
],
8083
} satisfies RedirectOptions,
8184
],
8285
[
@@ -99,6 +102,16 @@ const config: Config = {
99102
{name: 'keywords', content: 'IBM Granite, AI, foundation models, LLM'},
100103
{name: 'description', content: 'IBM Granite documentation — models, serving guides, cookbooks'},
101104
],
105+
announcementBar: {
106+
// Bump this id if the message changes and you want dismissals reset
107+
// (only relevant while isCloseable is true).
108+
id: 'site-sunset-2026',
109+
content:
110+
'⚠️ This documentation site is no longer being updated. For the latest Granite documentation and models, see <a target="_blank" rel="noopener noreferrer" href="https://huggingface.co/ibm-granite">Hugging Face</a> and <a target="_blank" rel="noopener noreferrer" href="https://github.com/ibm-granite">GitHub</a>.',
111+
backgroundColor: '#fcf4d6',
112+
textColor: '#1c1200',
113+
isCloseable: false,
114+
},
102115
navbar: {
103116
title: '',
104117
logo: {

granite/docs/models/granite.mdx renamed to granite/docs/models/granite4-0.mdx

File renamed without changes.

‎granite/docs/models/granite4-2.mdx‎

Lines changed: 267 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,267 @@
1+
---
2+
title: "Granite 4.2"
3+
description: "Dense reasoning language model family in 3B, 8B, and 30B sizes with built-in chain-of-thought thinking, flexible thinking modes, and reasoning-augmented tool calling."
4+
sidebar_custom_props:
5+
icon: "brain"
6+
---
7+
8+
<CardGroup cols={2}>
9+
<Card
10+
title="Model Collection"
11+
icon="arrow-right"
12+
href="https://huggingface.co/collections/ibm-granite/granite-42-language-models"
13+
>
14+
View the full Granite 4.2 collection on Hugging Face
15+
</Card>
16+
<Card
17+
title="GitHub Organization"
18+
icon="github"
19+
href="https://github.com/ibm-granite/granite-4.2-language-models"
20+
>
21+
Granite models and documentation
22+
</Card>
23+
</CardGroup>
24+
25+
## Overview
26+
27+
**Granite 4.2** is a family of dense reasoning language models available in three sizes: 3B, 8B, and 30B parameters. Granite 4.2 introduces native reasoning (thinking) capabilities, allowing models to perform step-by-step chain-of-thought reasoning before producing final answers. This significantly improves performance on complex math, coding, multi-step logic, and agentic tool-calling tasks.
28+
29+
### Model Variants
30+
31+
- **granite-4.2-3b**: Compact reasoning model optimized for edge deployment and resource-constrained environments
32+
- **granite-4.2-8b**: Balanced reasoning model for general-purpose enterprise applications
33+
- **granite-4.2-30b**: Flagship reasoning model for complex reasoning and specialized tasks
34+
35+
All models natively support a 128K context window (with long-context extension to 512K on the 30B model) and are released under the Apache 2.0 license with cryptographic signatures, ISO certification, and full transparency disclosures, enabling unrestricted commercial and academic use.
36+
37+
### Key Capabilities
38+
39+
**Built-in Reasoning**: Granite 4.2 features native chain-of-thought reasoning inside `<think>...</think>` tags, significantly improving performance on math, coding, and complex multi-step problems.
40+
41+
**Flexible Thinking Modes**: Seamlessly switch between full thinking (default), non-thinking, and low-effort modes within a single model, allowing users to balance depth vs. latency on a per-query basis.
42+
43+
**Reasoning-Augmented Tool Calling**: The model reasons about which tools to invoke and why before making the call, producing more accurate function calls for agentic workflows.
44+
45+
**Multilingual Dialog**: Granite 4.2 is tested across English, German, Spanish, French, Japanese, Portuguese, Arabic, Czech, Italian, Korean, Dutch, and Chinese.
46+
47+
## Getting Started
48+
49+
First, install the required libraries:
50+
51+
<CodeGroup>
52+
53+
```bash Install
54+
pip install torch torchvision torchaudio
55+
pip install accelerate
56+
pip install transformers
57+
```
58+
59+
</CodeGroup>
60+
61+
### Generation
62+
63+
This is a simple example of how to use the Granite-4.2-30B model in thinking mode:
64+
65+
<CodeGroup>
66+
67+
```python Python
68+
import torch
69+
from transformers import AutoModelForCausalLM, AutoTokenizer
70+
71+
model_path = "ibm-granite/granite-4.2-30b"
72+
tokenizer = AutoTokenizer.from_pretrained(model_path)
73+
# drop device_map if running on CPU
74+
model = AutoModelForCausalLM.from_pretrained(model_path, device_map="cuda", torch_dtype=torch.bfloat16)
75+
model.eval()
76+
77+
# change input text as desired
78+
messages = [
79+
{ "role": "user", "content": "How many r's are in the word 'strawberry'?" },
80+
]
81+
82+
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True, enable_thinking=True)
83+
inputs = tokenizer(text, return_tensors="pt").to(model.device)
84+
85+
# generate output tokens
86+
output = model.generate(**inputs, max_new_tokens=8192, temperature=1.0, top_p=0.95, do_sample=True)
87+
88+
# decode output tokens into text
89+
print(tokenizer.decode(output[0][inputs.input_ids.shape[-1]:], skip_special_tokens=False))
90+
```
91+
92+
</CodeGroup>
93+
94+
Expected output:
95+
96+
<CodeGroup>
97+
98+
```text Output
99+
<think>
100+
First, I need to write out the word: s t r a w b e r r y.
101+
Now, I count the number of 'r' letters: one at position 3, one at position 8, one at position 9.
102+
Total r's = 3.
103+
</think>
104+
There are **3** r's in the word "strawberry".<|im_end|>
105+
```
106+
107+
</CodeGroup>
108+
109+
> **Generation parameters:** Use `temperature=1.0` and `top_p=0.95` across **all tasks and serving backends**, including general chat, reasoning, and tool calling.
110+
111+
### Thinking Modes
112+
113+
Granite 4.2 supports three thinking modes, selected via chat-template parameters:
114+
115+
| Mode | Template Parameters | Behavior |
116+
|:-----|:-------------------|:---------|
117+
| **Thinking** (default) | `enable_thinking=True` | Full chain-of-thought reasoning inside `<think>...</think>` |
118+
| **Non-thinking** | `enable_thinking=False` | Direct answer with no reasoning overhead |
119+
| **Low-effort** | `enable_thinking=True, low_effort=True` | Brief reasoning for simpler queries |
120+
121+
#### Non-Thinking Mode
122+
123+
Disable reasoning for a direct answer with no chain-of-thought overhead:
124+
125+
<CodeGroup>
126+
127+
```python Python
128+
messages = [
129+
{"role": "user", "content": "What is the capital of France?"},
130+
]
131+
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True, enable_thinking=False)
132+
inputs = tokenizer(text, return_tensors="pt").to(model.device)
133+
134+
output = model.generate(**inputs, max_new_tokens=2048, temperature=1.0, top_p=0.95, do_sample=True)
135+
print(tokenizer.decode(output[0][inputs.input_ids.shape[-1]:], skip_special_tokens=False))
136+
```
137+
138+
</CodeGroup>
139+
140+
Expected output:
141+
142+
<CodeGroup>
143+
144+
```text Output
145+
<think></think>The capital of France is Paris.<|im_end|>
146+
```
147+
148+
</CodeGroup>
149+
150+
### Tool Calling
151+
152+
Granite 4.2 supports tool calling with integrated reasoning — the model thinks about which tool to call and why before making the call. Define a list of tools using OpenAI's function definition schema:
153+
154+
<CodeGroup>
155+
156+
```python Python
157+
import torch
158+
from transformers import AutoModelForCausalLM, AutoTokenizer
159+
160+
model_path = "ibm-granite/granite-4.2-30b"
161+
tokenizer = AutoTokenizer.from_pretrained(model_path)
162+
# drop device_map if running on CPU
163+
model = AutoModelForCausalLM.from_pretrained(model_path, device_map="cuda", torch_dtype=torch.bfloat16)
164+
model.eval()
165+
166+
tools = [
167+
{
168+
"type": "function",
169+
"function": {
170+
"name": "get_current_weather",
171+
"description": "Get the current weather for a specified city.",
172+
"parameters": {
173+
"type": "object",
174+
"properties": {
175+
"city": {
176+
"type": "string",
177+
"description": "Name of the city"
178+
}
179+
},
180+
"required": ["city"]
181+
}
182+
}
183+
}
184+
]
185+
186+
# change input text as desired
187+
messages = [
188+
{ "role": "user", "content": "What's the weather like in Boston right now?" },
189+
]
190+
191+
text = tokenizer.apply_chat_template(messages, tokenize=False, tools=tools,
192+
add_generation_prompt=True, enable_thinking=True)
193+
inputs = tokenizer(text, return_tensors="pt").to(model.device)
194+
195+
# generate output tokens
196+
output = model.generate(**inputs, max_new_tokens=4096, temperature=1.0, top_p=0.95, do_sample=True)
197+
198+
# decode output tokens into text
199+
print(tokenizer.decode(output[0][inputs.input_ids.shape[-1]:], skip_special_tokens=False))
200+
```
201+
202+
</CodeGroup>
203+
204+
Expected output:
205+
206+
<CodeGroup>
207+
208+
```text Output
209+
<think>
210+
The user is asking for the weather in Boston right now. There's a function called
211+
get_current_weather that takes a city parameter. I need to call that with the city set to Boston.
212+
</think>
213+
<tool_call>
214+
<function=get_current_weather>
215+
<parameter=city>
216+
Boston
217+
</parameter>
218+
</function>
219+
</tool_call>
220+
<|im_end|>
221+
```
222+
223+
</CodeGroup>
224+
225+
### Serving with vLLM
226+
227+
Granite 4.2 is optimized for deployment with [vLLM](https://github.com/vllm-project/vllm) (v0.20+).
228+
229+
> **Reasoning parser:** Use the custom `granite_thinking_parser`, available in each model's Hugging Face repository. The models also work with the built-in `nemotron_v3` parser, but `granite_thinking_parser` provides better formatting of reasoning output.
230+
> **Tool calling parser:** Use `qwen3_coder`.
231+
232+
<CodeGroup>
233+
234+
```bash Serve
235+
vllm serve ibm-granite/granite-4.2-30b \
236+
--served-model-name granite-4.2-30b \
237+
--dtype bfloat16 \
238+
--max-model-len 131072 \
239+
--reasoning-parser granite_thinking_parser \
240+
--reasoning-parser-plugin ./granite_thinking_parser.py \
241+
--tool-call-parser qwen3_coder \
242+
--enable-auto-tool-choice
243+
```
244+
245+
</CodeGroup>
246+
247+
The model then exposes an OpenAI-compatible API on `http://localhost:8000/v1`, which integrates with popular agentic coding harnesses such as OpenCode, Pi, and OpenHands out of the box.
248+
249+
<CodeGroup>
250+
251+
```python Python
252+
from openai import OpenAI
253+
254+
client = OpenAI(base_url="http://localhost:8000/v1", api_key="unused")
255+
256+
response = client.chat.completions.create(
257+
model="granite-4.2-30b",
258+
messages=[{"role": "user", "content": "Explain the Riemann hypothesis in simple terms."}],
259+
temperature=1.0,
260+
top_p=0.95,
261+
max_tokens=8192,
262+
)
263+
264+
print(response.choices[0].message.content)
265+
```
266+
267+
</CodeGroup>

‎granite/docs/models/guardian.mdx‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
---
2-
title: "Granite Guardian 4.1"
2+
title: "Granite Guardian"
33
description: "Risk detection models and LoRA adapters for evaluating AI safety, RAG groundedness, and agentic function calling — with support for custom bring-your-own criteria (BYOC)."
44
sidebar_custom_props:
55
icon: "shield"

‎granite/docs/models/speech.mdx‎

Lines changed: 7 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
---
22
title: "Granite Speech"
3-
description: "Compact multilingual models for automatic speech recognition (ASR) and translation (AST) across English, French, German, Spanish, Portuguese, and Japanese."
3+
description: "Compact speech models for automatic speech recognition (ASR)."
44
sidebar_custom_props:
55
icon: "mic"
66
---
@@ -14,9 +14,9 @@ sidebar_custom_props:
1414
View the full Granite Speech collection on Hugging Face
1515
</Card>
1616
<Card
17-
title="Speech Demo"
17+
title="Streaming Audio Demo"
1818
icon="rocket"
19-
href="https://huggingface.co/spaces/ibm-granite/granite-speech"
19+
href="https://huggingface.co/spaces/ibm-granite/granite-speech-streaming-webgpu"
2020
>
2121
Try Granite Speech in action
2222
</Card>
@@ -38,21 +38,20 @@ sidebar_custom_props:
3838

3939
## Overview
4040

41-
The **Granite Speech 4.1** model family provides compact and efficient speech-language models for multilingual automatic speech recognition (ASR) and automatic speech translation (AST), supporting English, French, German, Spanish, Portuguese, and Japanese. All models are trained on 174,000 hours of audio from public corpora and tailored synthetic datasets.
41+
The Granite Speech model family provides compact and efficient speech-language models for multilingual automatic speech recognition (ASR) and automatic speech translation (AST), supporting English, French, German, Spanish, Portuguese, and Japanese.
4242

4343
### Model Variants
4444

45-
The Granite Speech 4.1 suite includes three specialized variants:
45+
The Granite Speech suite includes four specialized variants:
4646

47+
- **[granite-speech-5.0-470m-turboctc](https://huggingface.co/ibm-granite/granite-speech-5.0-470m-turboctc)**: compact English ASR model with very high inference speed that is well suited for deployment on laptops, smartphones and other edge devices
4748
- **[granite-speech-4.1-2b](https://huggingface.co/ibm-granite/granite-speech-4.1-2b)**: Balanced ASR and AST capabilities with improved punctuation and capitalization across all supported languages
4849
- **[granite-speech-4.1-2b-plus](https://huggingface.co/ibm-granite/granite-speech-4.1-2b-plus)**: Speech-to-text model with speaker-attributed ASR, timestamps, and keyword-prompted ASR for enhanced recognition of names, acronyms, and technical jargon
4950
- **[granite-speech-4.1-2b-nar](https://huggingface.co/ibm-granite/granite-speech-4.1-2b-nar)**: Non-autoregressive variant ([NLE architecture](https://arxiv.org/abs/2603.08397)) optimized for fast and accurate ASR with significantly lower latency
5051

5152
### Performance
5253

53-
Granite Speech 4.1 models deliver industry-leading performance on the [OpenASR Leaderboard](https://huggingface.co/spaces/hf-audio/open_asr_leaderboard): **granite-speech-4.1-2b ranks #1** for accuracy, while **granite-speech-4.1-2b-nar places #3** with exceptional speed through its non-autoregressive architecture.
54-
55-
Granite Speech is released under the Apache 2.0 license, making it freely available for both research and commercial purposes, with full transparency into its training data.
54+
Granite Speech models deliver industry-leading performance on the [OpenASR Leaderboard](https://huggingface.co/spaces/hf-audio/open_asr_leaderboard) and the FFASR Leaderboard. Granite Speech is released under the Apache 2.0 license, making it freely available for both research and commercial purposes, with full transparency into its training data.
5655

5756
[Granite Speech Paper](https://arxiv.org/abs/2505.08699)
5857

‎granite/docs/run/granite-with-vllm-containerized.mdx‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -69,6 +69,6 @@ Now run the container with the added parameters:
6969
docker run --runtime nvidia --gpus all -v ~/.cache/huggingface:/root/.cache/huggingface -p 8000:8000 vllm/vllm-openai:latest --model ibm-granite/granite-4.0-h-small --tool-call-parser granite4 --enable-auto-tool-choice
7070
```
7171

72-
Once the container is up, you can start running requests using the OpenAI API. Refer to the documentation on OpenAI API [tool calling](/models/granite#tool-calling) for examples.
72+
Once the container is up, you can start running requests using the OpenAI API. Refer to the documentation on OpenAI API [tool calling](/models/granite4-0#tool-calling) for examples.
7373

7474
To run vLLM with the Granite 3 models and tool calling, use the additional parameters specified in the vLLM documentation [here](https://docs.vllm.ai/en/stable/features/tool_calling/#ibm-granite) as part of the `docker run` command share in Section 2.

‎granite/docs/use-cases/docling-rag.mdx‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@ sidebar_custom_props:
66
icon: "magnifying-glass"
77
---
88

9-
In this tutorial, you will create an AI-powered document retrieval system with [Docling](https://github.com/DS4SD/docling), [LangChain](https://github.com/langchain-ai/langchain), and [Granite 3.1](/models/granite). The tutorial is designed to enable you to gain proficiency in document processing and chunking, integrate vector databases to enhance retrieval capabilities, and utilize RAG to perform efficient and accurate data retrieval for real-world applications.
9+
In this tutorial, you will create an AI-powered document retrieval system with [Docling](https://github.com/DS4SD/docling), [LangChain](https://github.com/langchain-ai/langchain), and [Granite 3.1](/models/granite4-0). The tutorial is designed to enable you to gain proficiency in document processing and chunking, integrate vector databases to enhance retrieval capabilities, and utilize RAG to perform efficient and accurate data retrieval for real-world applications.
1010

1111
<Card
1212
title="Granite Docling RAG notebook"

‎sidebars.ts‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7,8 +7,9 @@ const sidebars: SidebarsConfig = {
77
type: 'category',
88
label: 'Models',
99
items: [
10+
'models/granite4-2',
1011
'models/granite4-1',
11-
'models/granite',
12+
'models/granite4-0',
1213
'models/docling',
1314
'models/vision',
1415
'models/speech',

‎src/components/Icon.tsx‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ import useBaseUrl from '@docusaurus/useBaseUrl';
44
import {FontAwesomeIcon} from '@fortawesome/react-fontawesome';
55
import type {IconDefinition, SizeProp} from '@fortawesome/fontawesome-svg-core';
66
import {
7-
faArrowRight, faBicycle, faBolt, faBook, faBox, faBriefcase, faBug,
7+
faArrowRight, faBicycle, faBolt, faBook, faBox, faBrain, faBriefcase, faBug,
88
faChartLine, faChurch, faCircleExclamation, faCircleQuestion, faCloud,
99
faCode, faCookieBite, faCopy, faCube, faDesktop, faDownload, faEye, faFileCode,
1010
faImages, faLaptop, faMagnifyingGlass, faMicrophone, faNewspaper,
@@ -26,6 +26,7 @@ const FA_MAP: Record<string, IconDefinition> = {
2626
'bolt': faBolt,
2727
'book': faBook,
2828
'box': faBox,
29+
'brain': faBrain,
2930
'briefcase': faBriefcase,
3031
'chart-line': faChartLine,
3132
'church': faChurch,

0 commit comments

Comments
 (0)