Skip to content

Add vector deduplication, indent context and litellm replacement - #29

Merged
ccmdi merged 16 commits into
masterfrom
feat/vector-db
Jan 30, 2026
Merged

Add vector deduplication, indent context and litellm replacement#29
ccmdi merged 16 commits into
masterfrom
feat/vector-db

Conversation

@ccmdi

@ccmdi ccmdi commented Jan 30, 2026

Copy link
Copy Markdown
Owner

Summary

Vector-based deduplication has been added to prevent repeated flashcards by checking similarity against embeddings during generation. litellm was replaced with a lightweight custom client, and the Obsidian API integration was refactored to use JSONLogic instead of Dataview.

Changes

  • Implemented a vector storage engine using JSON and API-based embeddings (Gemini/OpenAI) to index flashcard text.
  • Added a multi-turn feedback loop where the AI proposes cards, receives similarity warnings based on the vector index, and revises content before submission.
  • Replaced the litellm library with a custom call.py module that handles API requests for OpenAI, Anthropic, Google, and DeepSeek directly using httpx.
  • Introduced vector CLI commands to index existing decks, check status, clear data, and test similarity matching manually.
  • Refactored Obsidian note searching to use structured JsonLogic filters instead of concatenating DQL strings.
  • Created an IndentedConsole class to manage output indentation scopes and added status spinners for long-running AI tasks.
  • Updated the configuration to support vector deduplication settings and newer AI models like Claude Opus 4.5 and Gemini 3.
  • Added conversation logging to save AI interaction history to the local configuration directory for debugging.
  • Modified flashcard processing to automatically index new cards into the vector store upon creation.
  • Updated unit tests to cover the new vector store, console wrapper, and search logic.

@ccmdi
ccmdi merged commit a2711bd into master Jan 30, 2026
1 check passed
@ccmdi
ccmdi deleted the feat/vector-db branch January 30, 2026 16:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant