hanfts

Native Full-Text Search for Android — Korean & English, zero dependencies, pure in-memory.

한국어

Overview

hanfts is an Android full-text search library that supports both Korean and English. It uses a bigram tokenizer implemented in C++17 to enable partial Korean search.

Background

With Room's default setup, searching "산책" does not match a document containing "오늘산책했다".

hanfts provides an alternative that works out of the box, tokenizing Korean text into overlapping bigrams at the Unicode codepoint level without a morphological analyzer.

Features

• Korean bigram tokenization — overlapping 2-grams at the Unicode codepoint level; no morphological analyzer required
• English word tokenization — case-insensitive, whitespace-delimited tokens
• Mixed-language support — Korean and English handled independently in the same document
• Prefix search — typing "산" matches "산책", "산보" and more, in real time
• TF-IDF ranking — relevance scoring; title weighted 3× over body
• Thread-safe — concurrent reads via std::shared_mutex; writes are exclusive
• Pure in-memory — no file I/O, no Android Context, no permissions
• Zero native dependencies — pure C++17 STL only
• Closeable — deterministic native memory release via close() or use { }

Installation

hanfts is available on Maven Central. No extra repository configuration needed.

dependencies {
    implementation("io.github.m1n9yu23:hanfts:<version>")
}

Replace <version> with the latest release.

Quick Start

Create and close

val engine = SearchEngine()

engine.close()

SearchEngine().use { engine ->
    // ...
}

Build the index

engine.rebuildIndex(
    listOf(
        Document(id = "walk-001", title = "오늘의 산책", body = "날씨가 맑아서 공원을 걸었다."),
        Document(id = "walk-002", title = "Morning Walk", body = "The park was quiet and peaceful."),
    )
)

engine.indexDocument(id = "post-789", title = "제목", body = "본문 내용")
engine.indexDocument(Document(id = UUID.randomUUID().toString(), title = "Title", body = "Body text"))

Search

val results = engine.search("산책")
// → [SearchResult(id="walk-001", score=1.82f)]

val results = engine.search("산책", limit = 5)

Update and remove

engine.indexDocument(id = "walk-001", title = "updated title", body = "updated body")

engine.removeDocument(id = "walk-001")

engine.clear()

Example with ViewModel

The engine can be integrated into any architecture. Here is an example using ViewModel.

class SearchViewModel : ViewModel() {

    private val engine = SearchEngine()

    init {
        viewModelScope.launch(Dispatchers.Default) {
            engine.rebuildIndex(/* your documents */)
        }
    }

    fun search(query: String): List<SearchResult> =
        engine.search(query)

    override fun onCleared() {
        super.onCleared()
        engine.close()
    }
}

API Reference

SearchEngine

Obtain an instance via the factory function:

val engine = SearchEngine()

Method / Property	Description
documentCount: Int	Number of documents currently in the index.
indexDocument(id, title, body)	Adds or replaces a document. Replaces if id already exists.
indexDocument(document: Document)	Convenience overload accepting a Document.
removeDocument(id: String)	Removes a document by ID. No-op if not found.
clear()	Removes all documents from the index.
search(query, limit = 20)	Returns results ranked by TF-IDF score, descending. Blank query returns empty list.
rebuildIndex(documents: List<Document>)	Clears and rebuilds the entire index atomically.
close()	Releases native memory. Do not use the engine after closing.

Threading: all mutating operations (indexDocument, removeDocument, rebuildIndex, clear) acquire an internal lock and are blocking. Run them on Dispatchers.Default. Concurrent search() calls are always safe.

Document

data class Document(
    val id: String,
    val title: String,
    val body: String,
)

Document IDs are arbitrary strings. Use whatever uniquely identifies your records — Room primary keys (id.toString()), Firebase document IDs, REST slugs, UUID.randomUUID().toString(), etc. IDs are matched by String.equals/hashCode, so Unicode-distinct strings are different IDs.

SearchResult

data class SearchResult(
    val id: String,
    val score: Float,
)

How It Works

Tokenizer

Input	Language	Tokens
"오늘산책"	Korean	["오늘", "늘산", "산책", "오늘산책"]
"오늘산책했다"	Korean (5+ chars)	["오늘", "늘산", "산책", "책했", "했다"]
"Good morning"	English	["good", "morning"]
"meeting 미팅"	Mixed	["meeting", "미팅"]

Korean text is split into overlapping bigrams at the Unicode codepoint level. Words of 4 codepoints or fewer also emit the full word as an additional token to improve exact-match recall.

Korean Unicode ranges covered:

U+AC00–U+D7A3  Hangul Syllables  (가–힣)
U+1100–U+11FF  Hangul Jamo
U+3130–U+318F  Hangul Compatibility Jamo
U+A960–U+A97F  Hangul Jamo Extended-A
U+D7B0–U+D7FF  Hangul Jamo Extended-B

Scoring

TF    = term frequency in document / total weighted terms in document
IDF   = log((N + 1) / (df + 1)) + 1    (smoothed — always ≥ 1)
Score = TF × IDF

N = total document count, df = number of documents containing the term.
Title tokens count 3× relative to body tokens.

Prefix Search

std::map::lower_bound(token) locates the first matching term; the iterator advances while the term starts with the query token. A cap of 64 matching terms per token prevents single-character queries from expanding excessively.

Architecture

SearchEngine            ← Kotlin public interface (factory: SearchEngine())
    │
    │  JNI — libhanfts.so
    ▼
NativeSearchEngine      ← internal Kotlin class, AtomicLong handle
    │
    ▼
fts::SearchEngine       ← C++ API + std::shared_mutex concurrency
    ├── fts::InvertedIndex  — TF-IDF posting lists, prefix scan via std::map
    └── fts::Tokenizer      — UTF-8 → codepoints → bigrams / words

NativeSearchEngine is internal — users interact only with the SearchEngine interface. The C++ object is managed as an opaque jlong handle via AtomicLong. A ReentrantReadWriteLock ensures that close() waits for any in-progress operation to finish before destroying the native engine, preventing use-after-free in concurrent scenarios.

Notes

• Index is in-memory only. The index is not persisted to disk. Call rebuildIndex() each time the app starts, typically in a ViewModel init block on Dispatchers.Default.
• Always call close(). The engine holds native memory. Tie its lifecycle to a ViewModel (onCleared) or use the use { } block to ensure it is released.
• Bigram tokenization has trade-offs. Single-character queries may match a large number of documents. The prefix scan is capped at 64 terms per token to keep results manageable.
• Mutating operations are blocking. rebuildIndex, indexDocument, removeDocument, and clear acquire an exclusive write lock. Do not call them on the main thread.
• search() is safe to call concurrently. Multiple threads can search at the same time without additional synchronization.

Performance

Measured with the Microbenchmark library on an Android 16 emulator (Pixel 6a AVD, 4-core CPU, 2 GHz). Real device results will differ.

Operation	Median
search("morning") — English, 10k docs	673 µs
search("산책") — Korean, 10k docs	594 µs
rebuildIndex(1k docs)	6.95 ms
rebuildIndex(10k docs)	83 ms

Search latency is sub-millisecond for both Korean and English against a 10,000-document index.

Requirements

• Android minSdk 21 (Android 5.0 Lollipop)
• NDK r25+
• CMake 3.22.1+
• C++17

ABI targets: arm64-v8a, armeabi-v7a, x86_64

References

• Introduction to Information Retrieval — Theoretical basis for the inverted index, TF-IDF scoring, and the smoothed IDF variant (log((N+1)/(df+1)) + 1).
• RFC 3629 — UTF-8, a transformation format of ISO 10646 — Reference specification for the manual 1–4 byte UTF-8 decoder.
• Unicode Standard — Hangul Blocks — Definition of Korean Unicode block ranges (Syllables U+AC00–U+D7A3, Jamo, Compatibility Jamo, Extended blocks).
• Android NDK — JNI Tips — Pointer-handle pattern, DeleteLocalRef management, and JNI_ABORT flag.
• cppreference — std::shared_mutex — Readers-writer lock implementation (C++17).

Contributing

All contributions are welcome — bug fixes, performance improvements, new language support, or documentation.

See CONTRIBUTING.md for details.

Support

If hanfts has been useful to you, consider giving it a ⭐ — it means a lot!

License

Copyright 2026 Gyugle

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.