Skip to content

Commit a8f2ac5

Browse files
committed
fix
1 parent 548c0bc commit a8f2ac5

2 files changed

Lines changed: 37 additions & 2 deletions

File tree

.gitignore

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -31,3 +31,4 @@ Thumbs.db
3131
# Project-specific
3232
/cache/
3333
*.log
34+
smoke_tests/

README.md

Lines changed: 36 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -50,7 +50,8 @@ rs = ReadSight("en-us")
5050

5151
# Syllable counting
5252
rs.syllable_count("banana") # 3
53-
rs.split_syllables("hyphenation") # ['hy', 'phen', 'a', 'tion']
53+
rs.split_syllables("hyphenation") # ['hyp', 'hen', 'ati', 'on'] (4 syllables, heuristic split)
54+
rs.split_word("hyphenation") # ['hy', 'phen', 'a', 'tion'] (TeX hyphenation points)
5455

5556
# Text analysis
5657
stats = rs.analyze("The quick brown fox jumps over the lazy dog.")
@@ -67,6 +68,36 @@ lix = rs.lix(text)
6768
print(f"LIX: {fre.score} - {fre.interpretation}")
6869
```
6970

71+
## Syllable Counting Modes
72+
73+
ReadSightPy has three syllable counting modes, configured per language via `syllableMode` in `data/languages/*.json`:
74+
75+
| Mode | How it works | `count` accuracy | `split` accuracy |
76+
|------|-------------|:---:|:---:|
77+
| **`heuristic`** | Vowel patterns + word list + prefix/suffix rules || ≈ approximate |
78+
| **`tex`** | Frank M. Liang hyphenation algorithm (TeX `.tex` patterns) || ✓ exact |
79+
| **`composite`** | Heuristic first, TeX as fallback || ≈ approximate (uses heuristic split) |
80+
81+
**80 languages use `tex`**, **4 use `composite`** (en-us, en-gb, it, pl), **2 use `heuristic`**.
82+
83+
### Example: "hyphenation" in each mode
84+
85+
```python
86+
rs = ReadSight("en-us") # composite mode — heuristic wins
87+
rs.syllable_count("hyphenation") # 4 ✓ (in problemWords list)
88+
rs.split_syllables("hyphenation") # ['hyp', 'hen', 'ati', 'on'] — heuristic: equal-width split, ≈ approximate
89+
rs.split_word("hyphenation") # ['hy', 'phen', 'a', 'tion'] — TeX hyphenator: exact points
90+
91+
rs = ReadSight("de-1996") # tex mode
92+
rs.syllable_count("hyphenation") # 4 ✓ (TeX patterns)
93+
rs.split_syllables("hyphenation") # ['hy', 'phen', 'a', 'tion'] — TeX: exact
94+
rs.split_word("hyphenation") # ['hy', 'phen', 'a', 'tion'] — same, both use TeX
95+
```
96+
97+
> **Tip:** `split_word()` always uses the TeX hyphenator (exact). `split_syllables()` may use heuristic (approximate). For syllable *counts* both are correct.
98+
99+
> **Note:** `add_hyphenations()` adds overrides to the TeX hyphenator. These affect `split_word()` but NOT `split_syllables()` in `composite`/`heuristic` modes (the heuristic counter doesn't see them).
100+
70101
## Demo
71102

72103
Run the interactive demo to see ReadSightPy in action:
@@ -178,6 +209,8 @@ rs.histogram_syllables(text: str) -> dict[int, int]
178209
rs.analyze(text: str) -> TextStatistics
179210
```
180211

212+
> **split_syllables vs split_word:** `split_syllables` may use heuristic ≈approximate split (depends on language's `syllableMode`). `split_word` always uses the TeX hyphenator for exact hyphenation points. Syllable *counts* are accurate in all modes. See [Syllable Counting Modes](#syllable-counting-modes).
213+
181214
#### Formula Methods
182215

183216
```python
@@ -233,10 +266,11 @@ rs = ReadSight(
233266
cache_dir="/custom/cache",
234267
)
235268

236-
# Add custom hyphenation rules
269+
# Add custom hyphenation rules (affects split_word, not split_syllables)
237270
rs.add_hyphenations({
238271
"customword": "cus-tom-word",
239272
})
273+
rs.split_word("customword") # ['cus', 'tom', 'word']
240274
```
241275

242276
## Architecture

0 commit comments

Comments
 (0)