Feat/download manager add custom server - #104
Merged
XDcobra merged 20 commits intoMay 29, 2026
Merged
Conversation
…Download Manager rework - Introduced `download_manager_overview.md` detailing the scope and architecture of the Download Manager rework, including the new `SourceProvider` abstraction and its features. - Added `execution-prompt.md` to guide implementing agents through the rework process, outlining the necessary steps and files to reference. - Created foundational documents for the cross-source contract, source registry, built-in providers, fetcher, and archive layout, establishing a clear migration path for the Download Manager.
- Updated functions to include sourceId as a parameter, enhancing model operation keys for better source differentiation. - Modified download, extraction, and cache management processes to accommodate multiple sources, ensuring consistent handling of models across different sources. - Refactored related paths and state management to support the new source-based structure, improving overall organization and maintainability of the download module.
…et handling - Introduced new source providers for GitHub and Hugging Face, enabling model retrieval and asset management. - Implemented functions for asset extension handling, filtering, and model ID derivation based on category. - Enhanced support for various model categories, ensuring compatibility with different asset types and repository structures. - Established a framework for building source models from GitHub release assets and Hugging Face repositories, improving the overall download process.
…urces and error handling - Introduced a series of tests for download sources, including GitHub and Hugging Face providers, ensuring proper asset mapping and error handling. - Implemented tests for various error codes and their handling within the download process, enhancing robustness against network and authorization issues. - Established tests for archive format validation and source registry functionality, ensuring compliance with expected behaviors and configurations. - Enhanced overall test coverage for the download module, contributing to improved reliability and maintainability of the codebase.
…ind and extraction flag - Modified the extraction logic to ensure it only triggers when the layout kind is 'archive' and extraction is enabled, improving the accuracy of the download process. - This change enhances the handling of different file types and ensures proper completion of downloads.
…handling and integration - Updated `download-manager.md` to clarify the source-aware model discovery and installation process, including support for built-in and custom providers. - Added detailed explanations for model ID retrieval and the mapping of built-in GitHub release tags. - Expanded the migration overview with new prompts for VoiceLab app integration and Hugging Face cross-match, improving guidance for developers. - Included a comprehensive audit log in the cleanup plan to track implementation status across sub-plans, ensuring thorough documentation of changes.
- Introduced a new detect module in the package.json, including source and type definitions. - Updated tsconfig.json and example/tsconfig.json to include paths for the detect module. - Added new source files for unified detection in the Android CMakeLists.txt and podspec, enhancing model detection capabilities.
…d more - Added new source files for unified model detection, allowing sequential detection across TTS, STT, VAD, punctuation, enhancement, and alignment models. - Introduced JNI and Objective-C interfaces for native model detection, enabling integration with Java and Swift environments. - Implemented batch detection capabilities, enhancing the ability to process multiple model inputs efficiently. - Updated the SherpaOnnxModule to include methods for unified model detection and batch processing, improving the overall functionality of the module.
- Introduced a new `UnifiedDetectNativeResult` type for improved detection results. - Added `detectModel` and `detectModelsBatch` methods for unified model detection across various categories. - Refactored import paths for model input resolution to streamline the detection process. - Created new files for model detection logic, enhancing modularity and maintainability. - Updated existing modules to utilize the new detection methods, improving overall functionality.
- Refactored import paths in `detect-apkasset.test.ts` to improve modularity. - Introduced a new test suite in `detectModel.test.ts` for `detectModel` and `detectModelsBatch`, covering various detection scenarios and ensuring accurate results. - Added tests for unified model detection in C++ to validate functionality and maintain expected behavior across different model categories.
… model-detect guide - Added a comprehensive guide for unified model detection in `model-detect.md`, detailing the detection process for various model categories. - Updated existing documentation across multiple files to reference the new `model-detect.md` for unified detection, improving clarity on model detection methods. - Included specific notes on using `detectModel`, `detectEnhancementModel`, and other feature-specific detection methods for better user guidance.
- Changed the return type of `nativeDetectModel` and `unifiedDetectHashMapToWritableMap` to accommodate nullable values, enhancing type safety and preventing potential null pointer exceptions. - This update improves the robustness of the model detection process by ensuring that the functions can handle cases where data may not be present.
- Introduced a new function `HasNameOnlyDetectionSource` to streamline detection source checks. - Refactored existing model detection conditions to utilize `IsCatalogDetectHit`, improving clarity and maintainability. - Added a unit test for name-only detection scenarios, ensuring accurate model category matching for specific inputs.
- Removed redundant check for cached models length in the cache validation condition, streamlining the logic for better readability and maintainability. - This change enhances the efficiency of the refreshModels function by reducing unnecessary complexity in the cache handling process.
- Introduced an extern "C" block to properly define JNI functions, ensuring compatibility with C++ name mangling. - This change enhances the clarity of the JNI interface in the `sherpa-onnx-module-jni.cpp` file, improving maintainability and integration with Java.
- Added support for tracking failed asset indices during downloads, allowing for retries of only the failed assets in subsequent attempts. - Enhanced the download state management to persist failed indices and next asset index, improving the robustness of the download process. - Updated unit tests to reflect changes in error handling and ensure proper functionality when assets fail checksum validation.
- Simplified the logic for determining included HF model paths by normalizing the input path and removing unnecessary checks. - Enhanced the function to avoid VCS metadata paths and ensure proper handling of edge cases, improving overall robustness and maintainability.
… background downloader dependency - Replaced the previous background downloader implementation with a new foreground download manager that supports HTTP Range resume. - Updated the documentation to reflect the new peer dependency on `@dr.pogodin/react-native-fs` and removed references to `@kesha-antonov/react-native-background-downloader`. - Enhanced the download process to allow for pausing, resuming, and canceling downloads while maintaining state across app sessions. - Added event listeners for download progress and completion, improving user feedback during downloads. - Refactored related code to streamline the download management process and improve maintainability.
- Added the `emitModelsListUpdated` function to notify about the updated list of downloaded models after an incomplete download is deleted. - Enhanced error handling to ensure an empty list is emitted if the model list retrieval fails, improving robustness in the download management process.
…sage guidelines - Clarified the model configuration and detection description to emphasize automatic processes. - Revised the "Read before use" section to focus on on-device memory management, detailing strategies for low peak-RAM usage and efficient audio processing. - Removed the outdated table of contents to streamline the document structure and improve readability.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This pull request introduces a unified model detection system for the Android native layer, streamlines model download dependencies, and significantly improves the documentation around memory management and model handling. The most important changes are grouped below:
Unified Model Detection (Android Native)
DetectModel,DetectModelsBatch) that sequentially runs TTS, STT, VAD, punctuation, enhancement, and alignment detectors, returning the first match. This improves automatic model configuration and simplifies detection logic for clients. (android/src/main/cpp/jni/model_detect/common/sherpa-onnx-model-detect-unified.cpp[1]android/src/main/cpp/jni/model_detect/common/sherpa-onnx-model-detect-unified.h[2]android/src/main/cpp/jni/model_detect/common/sherpa-onnx-unified-detect-wrapper.cppandroid/src/main/cpp/jni/model_detect/common/sherpa-onnx-unified-detect-wrapper.cppR1-R81)android/src/main/cpp/CMakeLists.txt[1]SherpaOnnx.podspec[2]Dependency and Build Changes
@dr.pogodin/react-native-fsand no longer requires MMKV. (android/build.gradleandroid/build.gradleL181-R181)Documentation and SDK Guidance
README.mdREADME.mdL50-R82)@dr.pogodin/react-native-fsas a peer dependency and clarifies foreground/background download behavior. (README.mdREADME.mdL105-R127)These changes collectively make model detection more robust and automatic, simplify integration for app developers, and provide clearer guidance for efficient on-device operation.