Skip to content

Commit 5187c6a

Browse files
committed
docs(useTokenizer): fix circular TokenizerProps type link + migrate to models
- The `tokenizer` prop description linked to `TokenizerProps` as its own type — a circular link that says nothing useful. Drop the "of type" clause; the prop is just an object with a `tokenizerSource`. - Migrate the two code snippets from the bare `ALL_MINILM_L6_V2` constant to `models.text_embedding.all_minilm_l6_v2()`, matching the rest of the migrated docs.
1 parent 6704118 commit 5187c6a

1 file changed

Lines changed: 9 additions & 5 deletions

File tree

docs/docs/03-hooks/01-natural-language-processing/useTokenizer.md

Lines changed: 9 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -29,9 +29,11 @@ We are using [Hugging Face Tokenizers](https://huggingface.co/docs/tokenizers/in
2929
## High Level Overview
3030

3131
```typescript
32-
import { useTokenizer, ALL_MINILM_L6_V2 } from 'react-native-executorch';
32+
import { models, useTokenizer } from 'react-native-executorch';
3333

34-
const tokenizer = useTokenizer({ tokenizer: ALL_MINILM_L6_V2 });
34+
const tokenizer = useTokenizer({
35+
tokenizer: models.text_embedding.all_minilm_l6_v2(),
36+
});
3537

3638
const text = 'Hello, world!';
3739

@@ -52,7 +54,7 @@ try {
5254

5355
`useTokenizer` takes [`TokenizerProps`](../../06-api-reference/interfaces/TokenizerProps.md) that consists of:
5456

55-
- `tokenizer` of type [`TokenizerProps`](../../06-api-reference/interfaces/TokenizerProps.md) containing [`tokenizerSource`](../../06-api-reference/interfaces/TokenizerProps.md#tokenizersource).
57+
- `tokenizer` — an object containing [`tokenizerSource`](../../06-api-reference/interfaces/TokenizerProps.md#tokenizersource).
5658
- An optional flag [`preventLoad`](../../06-api-reference/interfaces/TokenizerProps.md#preventload) which prevents auto-loading of the model.
5759

5860
You need more details? Check the following resources:
@@ -67,10 +69,12 @@ You need more details? Check the following resources:
6769
## Example
6870

6971
```typescript
70-
import { useTokenizer, ALL_MINILM_L6_V2 } from 'react-native-executorch';
72+
import { models, useTokenizer } from 'react-native-executorch';
7173

7274
function App() {
73-
const tokenizer = useTokenizer({ tokenizer: ALL_MINILM_L6_V2 });
75+
const tokenizer = useTokenizer({
76+
tokenizer: models.text_embedding.all_minilm_l6_v2(),
77+
});
7478

7579
// ...
7680

0 commit comments

Comments
 (0)