Skip to content

Commit cb9549a

Browse files
committed
corpus-indexing: BibleBook, UniversalBibleSearch, CorpusExplorer, 50+ tests
Adds comprehensive corpus indexing system for Metanoia Bible Reader: New Files: - models/BibleBook.kt: Canon, TextTradition, BookSection enums - models/BibleConstants.kt: 81+ book catalog with full metadata - bible/UniversalBibleSearch.kt: Search ALL books, no canonical hiding - bible/BibleCacheManager.kt: Cache-first fetching, rate-limit protection - ui/components/search/UniversalSearchComponents.kt: Reusable UI components - ui/screens/CorpusExplorerScreen.kt: Corpus explorer with tradition tabs - CORPUS_INDEXING_ARCHITECTURE.md: Complete architecture documentation Tests: - CanonAwareBibleBookTest.kt (20 tests): Canon correctness, tradition tracking - UniversalBibleSearchTest.kt (12 tests): Search across ALL canons - BibleCachingBehaviorTest.kt (12 tests): Cache-first behavior - BookOfWisdomScraperRoutingTest.kt (6 tests): Wisdom scraper routing Fixes: - Book of Wisdom scraper crash: Routes to Wikisource, handles errors gracefully - BibleModels.kt: Removed duplicate BibleBook declaration - ReadingAnalyticsScreen.kt: Fixed forEachIndexed destructuring - MainViewModel.kt: Fixed gateway initialization Design Principles: - Nothing is hidden: Universal search returns ALL matching books - Tradition-first organization: Masoretic → Septuagint → Ethiopic → NT - Reveal the past: Explicit 'Missing from Protestant' section - Cache-first: Protect endpoints with TTL caching - Full metadata: Every book shows tradition, canon, section
1 parent 010b31b commit cb9549a

13 files changed

Lines changed: 2932 additions & 487 deletions

File tree

CORPUS_INDEXING_ARCHITECTURE.md

Lines changed: 363 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,363 @@
1+
## Metanoia Bible Reader - Component Architecture
2+
3+
### Component: Biblical Corpus Indexing
4+
5+
**Purpose:** Index ALL books across ALL canons. Prevent hiding. Make everything discoverable.
6+
7+
**Location:** `bible/UniversalBibleSearch.kt`
8+
9+
**Responsibilities:**
10+
- Search ALL books without canonical filtering
11+
- Get books by textual tradition (Masoretic, Septuagint, New Testament, Ethiopic)
12+
- Get Septuagint-only books (deuterocanonical)
13+
- Get Ethiopic-only books
14+
- Get universal books (in all canons)
15+
- Get books missing from Protestant canon
16+
- Get corpus statistics
17+
18+
**Key Functions:**
19+
```kotlin
20+
fun searchBooks(query: String): List<BibleBook>
21+
fun getAllBooksByTradition(): Map<TextTradition, List<BibleBook>>
22+
fun getSeptuagintOnlyBooks(): List<BibleBook>
23+
fun getEthiopicOnlyBooks(): List<BibleBook>
24+
fun getUniversalBooks(): List<BibleBook>
25+
fun getMissingFromProtestant(): List<BibleBook>
26+
fun getCorpusStatistics(): CorpusStatistics
27+
```
28+
29+
**Design Principles:**
30+
1. **No Hiding:** Search returns ALL matching books, no canonical filtering
31+
2. **Reveal the Past:** Explicitly surface books Protestantism removed
32+
3. **Tradition-First:** Organize by textual tradition (Hebrew → Greek → Ge'ez)
33+
4. **Statistical Transparency:** Show exactly what's in each tradition
34+
35+
---
36+
37+
### Component: Universal Search UI
38+
39+
**Purpose:** Compose components for displaying search results with full metadata.
40+
41+
**Location:** `ui/components/search/UniversalSearchComponents.kt`
42+
43+
**Responsibilities:**
44+
- Search bar with universal search
45+
- Book cards showing tradition and canon badges
46+
- Tradition badges (color-coded)
47+
- Canon badges (color-coded)
48+
- Search results list
49+
- Corpus statistics card
50+
- Empty states
51+
52+
**Key Components:**
53+
```kotlin
54+
@Composable fun UniversalSearchBar(query, onQueryChange)
55+
@Composable fun BookCard(book, onClick)
56+
@Composable fun TraditionBadge(tradition)
57+
@Composable fun CanonBadges(canons)
58+
@Composable fun SearchResultsList(books, onBookClick)
59+
@Composable fun CorpusStatisticsCard(stats)
60+
```
61+
62+
**Design Principles:**
63+
1. **Full Metadata:** Show tradition, canon, section for every book
64+
2. **Visual Clarity:** Color-coded badges for traditions
65+
3. **No Filter Indicators:** Don't hide books with UI tricks
66+
4. **Statistical Honesty:** Show counts, not "..." ellipses
67+
68+
---
69+
70+
### Component: Corpus Explorer Screen
71+
72+
**Purpose:** Screen for exploring the COMPLETE biblical corpus by tradition.
73+
74+
**Location:** `ui/screens/CorpusExplorerScreen.kt`
75+
76+
**Responsibilities:**
77+
- Show all books organized by textual tradition
78+
- Tab navigation (Corpus, Septuagint, Ethiopic, Missing, Search)
79+
- Statistics overview
80+
- Tradition sections with descriptions
81+
- "Books Missing from Protestant Canon" section
82+
- Universal search (no canonical filtering)
83+
84+
**Key Functions:**
85+
```kotlin
86+
@Composable fun CorpusExplorerScreen(bibleManager, onBookClick, onBackClick)
87+
@Composable fun CorpusOverviewContent(corpusStats, allBooksByTradition, onBookClick)
88+
@Composable fun TraditionSection(tradition, books, onBookClick, description)
89+
@Composable fun MissingBooksSection(missingBooks, onBookClick)
90+
```
91+
92+
**Design Principles:**
93+
1. **Tradition-First Navigation:** Tabs for Masoretic, Septuagint, Ethiopic
94+
2. **Reveal the Missing:** Explicit "Missing from Protestant" section
95+
3. **Explain the History:** Context for WHY books are missing
96+
4. **No Canonical Filtering:** Search across ALL canons
97+
98+
---
99+
100+
### Component: Textual Tradition Tracking
101+
102+
**Purpose:** Track which textual tradition each book belongs to.
103+
104+
**Location:** `models/BibleBook.kt`
105+
106+
**Enums:**
107+
```kotlin
108+
enum class TextTradition {
109+
Masoretic, // Hebrew/Aramaic, 39 books
110+
Septuagint, // Greek, includes deuterocanonical books
111+
NewTestament, // Greek, 27 books
112+
Ethiopic // Ge'ez, Ethiopian-canon-only books
113+
}
114+
```
115+
116+
**Responsibilities:**
117+
- Distinguish Masoretic from Septuagint
118+
- Track Ethiopic texts
119+
- Provide `isSeptuagint` computed property
120+
- Support tradition-based filtering
121+
122+
---
123+
124+
### Component: Canon Membership
125+
126+
**Purpose:** Track which canons include each book.
127+
128+
**Location:** `models/BibleBook.kt`
129+
130+
**Enums:**
131+
```kotlin
132+
enum class Canon {
133+
Protestant,
134+
Catholic,
135+
Orthodox,
136+
Ethiopian
137+
}
138+
```
139+
140+
**Responsibilities:**
141+
- Track canon membership per book
142+
- Support multi-canon books
143+
- Provide `isDeuterocanonical` and `isEthiopianExclusive` computed properties
144+
- Generate human-readable canonical status
145+
146+
---
147+
148+
### Component: Sectional Grouping
149+
150+
**Purpose:** Logical grouping of books within a testament.
151+
152+
**Location:** `models/BibleBook.kt`
153+
154+
**Enums:**
155+
```kotlin
156+
enum class BookSection {
157+
Pentateuch, Historical, Wisdom, MajorProphets, MinorProphets,
158+
Deuterocanonical, EthiopianCanon,
159+
Gospels, Acts, PaulineEpistles, GeneralEpistles, Apocalyptic
160+
}
161+
```
162+
163+
**Responsibilities:**
164+
- Group books logically
165+
- Support section-based navigation
166+
- Preserve canonical order within sections
167+
168+
---
169+
170+
### Component: Error Handling & Routing
171+
172+
**Purpose:** Route books to correct scrapers and handle errors gracefully.
173+
174+
**Location:** `bible/BibleManager.kt`
175+
176+
**Key Functions:**
177+
```kotlin
178+
suspend fun fetchChapter(book, chapter, version)
179+
suspend fun fetchInterlinear(book, chapter)
180+
suspend fun scrapeChapter(book, chapter, version)
181+
suspend fun scrapeInterlinear(book, chapter)
182+
```
183+
184+
**Responsibilities:**
185+
- Route to Wikisource for deuterocanonical books
186+
- Route to Enoch scraper for Ethiopian books
187+
- Route to BibleGateway for standard books
188+
- Throw clear errors for NO_SOURCE books
189+
- Log errors instead of crashing
190+
191+
---
192+
193+
### Component: Cache Management
194+
195+
**Purpose:** Cache-first fetching to protect endpoints from rate limiting.
196+
197+
**Location:** `bible/BibleCacheManager.kt`
198+
199+
**Key Functions:**
200+
```kotlin
201+
suspend fun ensureChapter(book, chapter): Boolean
202+
suspend fun prefetchBook(book)
203+
suspend fun prefetchWholeBible()
204+
fun isBookCached(book): Boolean
205+
fun cachedChapterCount(book): Int
206+
```
207+
208+
**Responsibilities:**
209+
- Check cache before fetching
210+
- Don't refetch cached content
211+
- Prefetch entire books/books
212+
- Track progress and errors
213+
- Respect user canonical preferences
214+
215+
---
216+
217+
### Component: Settings Integration
218+
219+
**Purpose:** User preferences for canonical display.
220+
221+
**Location:** `settings/SettingsManager.kt` (existing)
222+
223+
**Settings:**
224+
- `showApocrypha`: Include Catholic/Orthodox deuterocanonical books
225+
- `showEthiopian`: Include Ethiopian-canon-only books
226+
- `bibleGatewayVersion`: Translation version
227+
228+
**Responsibilities:**
229+
- Store user canonical preferences
230+
- Map preferences to canon presets
231+
- Persist across sessions
232+
233+
---
234+
235+
## Component Dependencies
236+
237+
```
238+
CorpusExplorerScreen
239+
├─ UniversalBibleSearch
240+
│ └─ BibleBook (models)
241+
│ ├─ Canon enum
242+
│ ├─ TextTradition enum
243+
│ └─ BookSection enum
244+
├─ UniversalSearchComponents (UI)
245+
│ └─ BibleBook (models)
246+
└─ BibleManager (for searchVersesEverywhere)
247+
248+
UniversalSearchComponents
249+
└─ BibleBook (models)
250+
251+
BibleManager
252+
├─ WikisourceApocryphaScraper
253+
├─ WikisourceEnochScraper
254+
└─ BibleScraper
255+
256+
BibleCacheManager
257+
├─ BibleManager
258+
└─ SettingsManager
259+
```
260+
261+
---
262+
263+
## Anti-Patterns Avoided
264+
265+
### 1. Hiding by Default
266+
**Avoid:** Defaulting to Protestant canon and hiding everything else
267+
**Do:** Show ALL books by default in corpus explorer, filter only by explicit user choice
268+
269+
### 2. Canonical Filtering in Search
270+
**Avoid:** Search that only returns books in user's selected canon
271+
**Do:** Universal search returns ALL matching books across ALL canons
272+
273+
### 3. Metadata Concealment
274+
**Avoid:** Showing only book name and chapter count
275+
**Do:** Show tradition, canon, section, and canonical status for EVERY book
276+
277+
### 4. "Protestant First" Ordering
278+
**Avoid:** Ordering books by Protestant canonical order
279+
**Do:** Order by textual tradition (Hebrew → Greek → Ge'ez) or section
280+
281+
### 5. Apocrypha as "Optional"
282+
**Avoid:** Labeling deuterocanonical books as "optional" or "extra"
283+
**Do:** Treat all books equally, show which canons include them
284+
285+
### 6. No History Context
286+
**Avoid:** Presenting the 66-book canon as "the Bible"
287+
**Do:** Explain historical context, show which books were removed and why
288+
289+
---
290+
291+
## Test Coverage
292+
293+
### Universal Bible Search Tests
294+
- `UniversalBibleSearchTest.kt` (12 tests)
295+
- Search finds all matching books
296+
- Case-insensitive search
297+
- Tradition-based search
298+
- Books by tradition
299+
- Septuagint-only books
300+
- Ethiopic-only books
301+
- Universal books
302+
- Missing from Protestant
303+
- Canon-exclusive books
304+
- Corpus statistics
305+
- Nothing is hidden
306+
- Can search by section
307+
308+
### Canon-Aware Book Tests
309+
- `CanonAwareBibleBookTest.kt` (20 tests)
310+
- Protestant canon = 66 books
311+
- Catholic canon > 66 books
312+
- Ethiopian canon is broadest
313+
- Canon membership correctness
314+
- Tradition assignments
315+
- Sectional grouping
316+
- Canonical ordering
317+
- Canonical status descriptions
318+
- Strong's prefix correctness
319+
320+
### Caching Behavior Tests
321+
- `BibleCachingBehaviorTest.kt` (12 tests)
322+
- Cache-first behavior
323+
- No refetch when cached
324+
- Prefetch is idempotent
325+
- Error recovery
326+
- Prefetch continues after failure
327+
- Cache fraction accuracy
328+
329+
### Wisdom Scraper Tests
330+
- `BookOfWisdomScraperRoutingTest.kt` (6 tests)
331+
- Wisdom routing to Wikisource
332+
- Wisdom in SUPPORTED_BOOKS
333+
- Wisdom NOT in NO_SOURCE_BOOKS
334+
- Verse parsing
335+
- Network error propagation
336+
337+
---
338+
339+
## Future Enhancements
340+
341+
### 1. Verse-Level Indexing
342+
- Index verses by tradition
343+
- Show multiple translations (KJV, Septuagint, Ge'ez)
344+
- Cross-tradition verse comparison
345+
346+
### 2. Canonical Timeline
347+
- Show when books were added/removed
348+
- Timeline of canon development
349+
- Interactive canon evolution visualization
350+
351+
### 3. Reading Plans by Tradition
352+
- Masoretic reading plan
353+
- Septuagint reading plan
354+
- Ethiopic reading plan
355+
356+
### 4. Annotations by Tradition
357+
- Track which tradition a note applies to
358+
- Show tradition-specific commentary
359+
360+
### 5. Export by Tradition
361+
- Export Masoretic OT only
362+
- Export complete Ethiopian canon
363+
- Export "missing from Protestant" books

0 commit comments

Comments
 (0)