Skip to content

docs: Update INFERENCE data event reference for LiteRT & LiteRT-LM#76

Draft
javoweb-fc wants to merge 8 commits into
v2from
task/FLCRM-20632-inference-expression-engine
Draft

docs: Update INFERENCE data event reference for LiteRT & LiteRT-LM#76
javoweb-fc wants to merge 8 commits into
v2from
task/FLCRM-20632-inference-expression-engine

Conversation

@javoweb-fc

@javoweb-fc javoweb-fc commented Jun 22, 2026

Copy link
Copy Markdown

Description

This PR updates the developer reference documentation for the INFERENCE data event function to align with the latest on-device AI runtime capabilities and contracts in the fulcrum-expressions and KMP models repositories.

The updated documentation has been streamlined to keep underlying backend engines (LiteRT/LiteRT-LM) completely transparent to developers, while providing a clear and user-friendly guide focusing strictly on Form Reference Files for model loading.

Key Changes

  1. User-Friendly Abstraction:
    • Removed all direct references to LiteRT and LiteRT-LM in user-facing texts, replacing them with generic concepts like Vision ML and Generative LLM.
  2. Strict File-Extension-Based Auto-Detection:
    • Documented that model type auto-detection is determined strictly by the file extension (.tflite for Vision ML vs .gguf/.litertlm/.task for Generative LLM).
    • Clarified that config-level parameters do not affect or override detection, but must still match the model type.
  3. Streamlined Model Loading (Form Reference Files Only):
    • Removed any references to the central Model Catalog, catalog IDs, and resolution priority. The documentation now focuses exclusively on loading custom models via Form Reference Files.
  4. ONNX Support Deprecation:
    • Marked the legacy/ONNX-based configurations as Deprecated and added guidance to update to the newer config-based model definitions.
  5. Execution Modes:
    • Clearly separated the documentation into three clear paths: Vision ML, Generative LLM, and Legacy Vision ML (Deprecated).
  6. Performance and Device Warnings:
    • Added a prominent, highly visible callout warning about computational overhead, device memory footprint, and battery drain.
  7. Updated Examples:
    • Updated the standard Vision ML example to use the modern config syntax.
    • Added a new example demonstrating how to run an on-device LLM text summarization workflow (gemma-4-e2b.litertlm).

JIRA / Issue Reference

Closes: FLCRM-20632

Testing & Verification

  • Verified alignment of all properties, data types, error cases, and structures with functions.coffee (fulcrum-expressions) and InferenceModelResolver.kt (kmp/models).
  • Ran validation scans to ensure zero remaining references to "LiteRT", "LiteRT-LM", "Model Catalog", or "catalog ID" exist in user-facing text.

Copilot AI review requested due to automatic review settings June 22, 2026 20:59
@javoweb-fc javoweb-fc self-assigned this Jun 22, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates the INFERENCE data event developer reference to reflect the current on-device inference runtimes (LiteRT for vision and LiteRT-LM for LLM-style text generation), adds clearer mode-specific parameter documentation, and refreshes examples accordingly.

Changes:

  • Reworked the INFERENCE overview to distinguish Vision ML vs Generative LLM execution modes and added a prominent device resource/battery warning.
  • Documented modern options.config-based parameter schemas for LiteRT (vision) and LiteRT-LM (LLM), and marked legacy ONNX usage as deprecated.
  • Updated and expanded examples to show modern vision config usage and an on-device text summarization workflow.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread docs/DATA EVENTS/data-events-reference/data-events-inference.md Outdated
Comment thread docs/DATA EVENTS/data-events-reference/data-events-inference.md
Comment thread docs/DATA EVENTS/data-events-reference/data-events-inference.md Outdated
HenryTabima
HenryTabima previously approved these changes Jun 22, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated 5 comments.

Comment thread docs/DATA EVENTS/data-events-reference/data-events-inference.md Outdated
Comment thread docs/DATA EVENTS/data-events-reference/data-events-inference.md Outdated
Comment thread docs/DATA EVENTS/data-events-reference/data-events-inference.md Outdated
Comment thread docs/DATA EVENTS/data-events-reference/data-events-inference.md
Comment thread docs/DATA EVENTS/data-events-reference/data-events-inference.md Outdated
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings June 22, 2026 21:17
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated 6 comments.

Comment thread docs/DATA EVENTS/data-events-reference/data-events-inference.md Outdated
Comment thread docs/DATA EVENTS/data-events-reference/data-events-inference.md Outdated
Comment thread docs/DATA EVENTS/data-events-reference/data-events-inference.md Outdated
Comment thread docs/DATA EVENTS/data-events-reference/data-events-inference.md Outdated
Comment thread docs/DATA EVENTS/data-events-reference/data-events-inference.md
Comment thread docs/DATA EVENTS/data-events-reference/data-events-inference.md
Copilot AI review requested due to automatic review settings June 22, 2026 21:21
@javoweb-fc javoweb-fc removed the request for review from Copilot June 22, 2026 21:21
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings June 22, 2026 21:26

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated 3 comments.

Comment thread docs/DATA EVENTS/data-events-reference/data-events-inference.md Outdated
Comment thread docs/DATA EVENTS/data-events-reference/data-events-inference.md
Comment thread docs/DATA EVENTS/data-events-reference/data-events-inference.md Outdated

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated 3 comments.

Comment thread docs/DATA EVENTS/data-events-reference/data-events-inference.md
Comment thread docs/DATA EVENTS/data-events-reference/data-events-inference.md
Comment thread docs/DATA EVENTS/data-events-reference/data-events-inference.md Outdated
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings June 22, 2026 21:55

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated 4 comments.

Comment thread docs/DATA EVENTS/data-events-reference/data-events-inference.md
Comment thread docs/DATA EVENTS/data-events-reference/data-events-inference.md
Comment thread docs/DATA EVENTS/data-events-reference/data-events-inference.md
Comment thread docs/DATA EVENTS/data-events-reference/data-events-inference.md
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants