Issue generating response with ollama

[rocknegi]

I'm using local LLM via ollama

• LLM config

  ollama:
    api_endpoint_env: OLLAMA_URL
    llm_type: ollama
    models:
      high: qwen3:8b
      low: qwen3:8b

• Embeddings config:

  ollama:
    api_endpoint_env: OLLAMA_URL
    model: embeddinggemma:latest

• I followed the readme to run the project locally with postgress
  scapping the data works fine , everyting is stored in the DB

python -m data_loading.db_load https://feeds.libsyn.com/121695/rss Behind-the-Tech
Fetching content from URL: https://feeds.libsyn.com/121695/rss
Fetching content from URL: https://feeds.libsyn.com/121695/rss
Saved URL content to temporary file: /var/folders/qp/yt2_csw164d_9c9st9d0skrhyvv899/T/tmpkrygjxbp.xml (type: rss)
Detected file type: rss, contains embeddings: No
Computing embeddings for file...
Loading data from /var/folders/qp/yt2_csw164d_9c9st9d0skrhyvv899/T/tmpkrygjxbp.xml (resolved to /var/folders/qp/yt2_csw164d_9c9st9d0skrhyvv899/T/tmpkrygjxbp.xml) for site Behind-the-Tech using database endpoint 'postgres'
Detected file type: rss
Using embedding provider: ollama, model: embeddinggemma:latest
Processing as RSS feed...
Processing RSS/Atom feed: /var/folders/qp/yt2_csw164d_9c9st9d0skrhyvv899/T/tmpkrygjxbp.xml
Processed 74 episodes from RSS/Atom feed
Computing embeddings for batch of 74 texts
Uploading batch 1 of 1 (74 documents)
Attempting to upload batch 1/1
Executing query with 74 rows
Successfully inserted/updated 74 rows
--------------------------------------------------------------------------------
Batch 1 completed: 74 documents inserted/updated
Successfully uploaded batch 1
Processed 74/74 documents
Loading completed. Added 74 documents to the database.
Saved file with embeddings to ../data/json_with_embeddings/tmpkrygjxbp.xml
Cleaned up temporary file: /var/folders/qp/yt2_csw164d_9c9st9d0skrhyvv899/T/tmpkrygjxbp.xml

• but when i ask a question in the app like

Im looking for a podcast about ai engineering

• I get this log

=== EXTRACT_QUERY_PARAMS DEBUG ===
Full message object: Message(message_id='msg_1759246447630_90zuptgsf', sender_type=<SenderType.USER: 'user'>, message_type='user', conversation_id='1759246447630', timestamp='2025-09-30T15:34:07.630Z', content={'query': 'Im looking for a podcast about ai engineering', 'site': 'all', 'mode': 'list', 'prev_queries': []}, sender_info={'id': 'user_jv9p61i0d', 'name': 'user_jv9p61i0d'}, metadata=None)
Message attributes: {'message_id': 'msg_1759246447630_90zuptgsf', 'sender_type': <SenderType.USER: 'user'>, 'message_type': 'user', 'conversation_id': '1759246447630', 'timestamp': '2025-09-30T15:34:07.630Z', 'content': {'query': 'Im looking for a podcast about ai engineering', 'site': 'all', 'mode': 'list', 'prev_queries': []}, 'sender_info': {'id': 'user_jv9p61i0d', 'name': 'user_jv9p61i0d'}, 'metadata': None}
Extracted content: {'query': 'Im looking for a podcast about ai engineering', 'site': 'all', 'mode': 'list', 'prev_queries': []}
Content type: <class 'dict'>
Final query_params: {'query': ['Im looking for a podcast about ai engineering'], 'site': ['all'], 'mode': ['list'], 'prev': [], 'user_id': ['user_jv9p61i0d'], 'oauth_id': ['user_jv9p61i0d'], 'conversation_id': ['1759246447630'], 'streaming': ['true']}
=== END EXTRACT_QUERY_PARAMS DEBUG ===


=== NLWebHandler INIT ===
Query params: {'query': ['Im looking for a podcast about ai engineering'], 'site': ['all'], 'mode': ['list'], 'prev': [], 'user_id': ['user_jv9p61i0d'], 'oauth_id': ['user_jv9p61i0d'], 'conversation_id': ['1759246447630'], 'streaming': ['true']}
=========================


[TOOL-SELECTOR] Initializing for site: all
[TOOL-SELECTOR] Handler site value: all
No response from QueryRewrite prompt, using original query

[MULTI-SITE-HANDLER] Initializing with query: Im looking for a podcast about ai engineering
[MULTI-SITE-HANDLER] Params: {'score': 100, 'justification': 'Only available tool for this query type'}
[Using who_endpoint: http://localhost:8000/who
Calling sites_from_who_streaming with endpoint=http://localhost:8000/who, query=Im looking for a podcast about ai engineering

=== NLWebHandler INIT ===
Query params: {'query': 'Im looking for a podcast about ai engineering', 'streaming': 'true'}
=========================

[RETRIEVER] Development mode - param_endpoint from query_params: None

=== WHO HANDLER: Retrieved 0 items from nlweb_sites ===

Retrieved sites:
============================================================

=== WHO RANKING: Filtered to 0 results with score > 70 ===

Ranked sites (top 10):
============================================================
[MULTI-SITE] WARNING: No sites returned from who endpoint!

• The log from http://0.0.0.0:8000/ask is

=== NLWebHandler INIT ===
Query params: {}
=========================


[TOOL-SELECTOR] Initializing for site: all
[TOOL-SELECTOR] Handler site value: all
No response from QueryRewrite prompt, using original query

[MULTI-SITE-HANDLER] Initializing with query:
[MULTI-SITE-HANDLER] Params: {'score': 100, 'justification': 'Only available tool for this query type'}
[Using who_endpoint: http://localhost:8000/who
Calling sites_from_who_streaming with endpoint=http://localhost:8000/who, query=

=== NLWebHandler INIT ===
Query params: {'query': '', 'streaming': 'true'}
=========================

2025-09-30 17:34:56,529 - ollama_embedding - ERROR - exception:167 - Error generating Ollama embedding
NoneType: None
2025-09-30 17:34:56,531 - ollama_embedding - ERROR - log_with_context:181 - Ollama embedding generation failed | Context: model=embeddinggemma:latest - text_length=0 - error_type=IndexError - error_message=list index out of range
2025-09-30 17:34:56,531 - embedding_wrapper - ERROR - exception:167 - Error during embedding generation with provider ollama
NoneType: None
2025-09-30 17:34:56,533 - embedding_wrapper - ERROR - log_with_context:181 - Embedding generation failed | Context: provider=ollama - model=embeddinggemma:latest - text_length=0 - error_type=IndexError - error_message=list index out of range
2025-09-30 17:34:56,530 - webserver.routes.api - ERROR - Error in streaming who handler: All endpoint searches failed
Traceback (most recent call last):
  File "/Users/rnegi/Documents/NLWeb/code/python/webserver/routes/api.py", line 161, in who_handler
    await handler.runQuery()
  File "/Users/rnegi/Documents/NLWeb/code/python/core/whoHandler.py", line 81, in runQuery
    items = await search(
            ^^^^^^^^^^^^^
    ...<4 lines>...
    )
    ^
  File "/Users/rnegi/Documents/NLWeb/code/python/core/retriever.py", line 1132, in search
    results = await client.search(query, site, num_results, **kwargs)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/rnegi/Documents/NLWeb/code/python/core/retriever.py", line 893, in search
    raise ValueError("All endpoint searches failed")
ValueError: All endpoint searches failed
2025-09-30 17:34:56,534 - postgres_client - ERROR - exception:167 - Error generating embedding for query: list index out of range
NoneType: None
[MULTI-SITE] ERROR: Stream error: All endpoint searches failed
[MULTI-SITE] Traceback: Traceback (most recent call last):
  File "/Users/rnegi/Documents/NLWeb/code/python/methods/multi_site_query.py", line 65, in do
    async for site in sites_from_who_streaming(who_endpoint, self.query):
    ...<13 lines>...
            self.active_tasks.append(task)
  File "/Users/rnegi/Documents/NLWeb/code/python/core/utils/nlweb_client.py", line 406, in sites_from_who_streaming
    raise Exception(f"Stream error: {data.get('error', 'Unknown error')}")
Exception: Stream error: All endpoint searches failed

2025-09-30 17:34:56,537 - multi_site_query - ERROR - error:151 - Error in multi-site query: Stream error: All endpoint searches failed
NoneType: None

[oidebrett]

You’ll likely hit issues using embeddinggemma, since it outputs 768-dim vectors, while the system expects 1536 (hardcoded in several places). I’d suggest using Ollama for the LLM and testing embeddings via an API like OpenAI instead.