Could somebody share a working Ollama recommender configs please?

[fishfree]

Mine is as below:

But it always got result as below:

I'd prefer to small Ollama models such as qwen2 / llama3 / phi3 .

[reckart]

Note that INCEpTION currently limits the LLM response to 300 tokens, so you may not see any results for long sentences. Also, the response parser (extraction mode) used here cannot deal with cases where a word is contained multiple times but with different lemmas in one sentence. All instance of the word in a given sentence would be assigned the same lemma.

[fishfree]

@thank you! I configured exactly the same Ollama recommender as your screenshot. However, the result is not expected as below:

[reckart]

You can try starting INCEpTION with the parameter -Dlogging.level.de.tudarmstadt.ukp.inception.recommendation.imls.ollama=TRACE or add logging.level.de.tudarmstadt.ukp.inception.recommendation.imls.ollama=TRACE to your settings.properties to get more info about the communication with the LLM printed to the console log.

[fishfree]

@reckart Thank you very much! Sorry for replying late. After turning on tracing, the docker-compose logs -f output:

inception-app-1  | 2024-06-30 08:09:42 INFO [SYSTEM] SelectionTask - [admin][Named entity string-matching recommender]: Activated (0.10084033613445377 >= threshold 0.0)
inception-app-1  | 2024-06-30 08:09:42 INFO [SYSTEM] SelectionTask - [admin]: Evaluation complete
inception-app-1  | 2024-06-30 08:09:42 TRACE [SYSTEM] OllamaRecommender - Querying ollama [llama3:latest]: [Please process the folowing sentence and assign the lemma form to each word. The output should be in JSON format, where each word and its lemma form are presented as key-value pairs within a list. Here is the sentence to be processed:\r\n\r\n"Beijing and Kuala Lumpur agreed on Wednesday to strengthen bilateral cooperation to take the building of the China-Malaysia community with a shared future to a higher level, as they celebrated the 50th anniversary of the establishment of diplomatic ties."\r\n\r\nFormat the output JSON as follows:\r\n\r\n```json\r\n{\r\n  "word1": "lemma1",\r\n  "word2": "lemma2",\r\n  ...\r\n}\r\n```\r\n\r\nFor example, given the sentence "The cats are sitting on the mat", the output should look like this:\r\n\r\n```json\r\n{\r\n  "The": "the",\r\n  "cats": "cat",\r\n  "are": "be",\r\n  "sitting": "sit",\r\n  "on": "on",\r\n  "the": "the",\r\n  "mat": "mat"\r\n}\r\n```]
inception-app-1  | 2024-06-30 08:10:37 TRACE [SYSTEM] OllamaRecommender - Ollama [llama3:latest] responds: [{"word1": "lemma1",\n"Beijing": "city",\n"and": "and",\n"Kuala": "capital",\n"Lumpur": "capital",\n"agreed": "agree",\n"on": "on",\n"Wednesday": "weekday",\n"to": "to",\n"strengthen": "strengthen",\n"bilateral": "bilateral",\n"cooperation": "cooperate",\n"take": "take",\n"the": "the",\n"building": "build",\n"of": "of",\n"the": "the",\n"China-Malaysia": "China-Malaysia",\n"community": "community",\n"with": "with",\n"a": "a",\n"shared": "share",\n"future": "future",\n"to": "to",\n"a": "a",\n"higher": "high",\n"level": "level",\n"As": "as",\n"they": "they",\n"celebrated": "celebrate",\n"the": "the",\n"50th": "fiftieth",\n"anniversary": "anniversary",\n"of": "of",\n"the": "the",\n"establishment": "establish",\n"of": "of",\n"diplomatic": "diplomatic",\n"ties": "tie"}]
inception-app-1  | 2024-06-30 08:10:37 TRACE [SYSTEM] OllamaRecommender - Querying ollama [llama3:latest]: [Please process the folowing sentence and assign the lemma form to each word. The output should be in JSON format, where each word and its lemma form are presented as key-value pairs within a list. Here is the sentence to be processed:\r\n\r\n"During their talks in Kuala Lumpur, Premier Li Qiang and Malaysian Prime Minister Anwar Ibrahim agreed that the two countries will continue to promote high-quality implementation of the Regional Comprehensive Economic Partnership agreement, and strive to advance the conclusion of negotiations for Version 3.0 of the China-ASEAN Free Trade Agreement at an early date in order to build a closer China-ASEAN community with a shared future."\r\n\r\nFormat the output JSON as follows:\r\n\r\n```json\r\n{\r\n  "word1": "lemma1",\r\n  "word2": "lemma2",\r\n  ...\r\n}\r\n```\r\n\r\nFor example, given the sentence "The cats are sitting on the mat", the output should look like this:\r\n\r\n```json\r\n{\r\n  "The": "the",\r\n  "cats": "cat",\r\n  "are": "be",\r\n  "sitting": "sit",\r\n  "on": "on",\r\n  "the": "the",\r\n  "mat": "mat"\r\n}\r\n```]
inception-app-1  | 2024-06-30 08:11:30 TRACE [SYSTEM] OllamaRecommender - Ollama [llama3:latest] responds: [{"word1":"lemma1","word2":"lemma2","...}\n\nPlease provide the sentence you'd like me to process. I'll be happy to help!\n\nHere is the output for the given sentence:\n\n```\n{\n  ":" "," (punct), \n  ":", "," (punct),\n  ":" "," (punct),\n  ":" "," (punct),\n  ":", "," (punct),\n  ":", "," (punct),\n  ":", "," (punct),\n  ":", "," (punct),\n  ":", "," (punct),\n  ":", "," (punct),\n  ":", "," (punct),\n  ":", "," (punct),\n  ":", "," (punct),\n  ":", "," (punct),\n  ":", "," (punct),\n  ":", "," (punct),\n  ":", "," (punct),\n  ":", "," (punct),\n  ":", "," (punct),\n  ":", "," (punct),\n  ":", "," (punct)\n}\n```\n\nHere is the output for the sentence:\n\n{":" "," (punct), \n  ":" ", " (punct),\n  ":", "," (punct),\n  ":", "," (punct),\n  ":", "," (punct),\n  ":", "," (punct),\n  ":",]
inception-app-1  | 2024-06-30 08:11:30 ERROR [SYSTEM] MentionsFromJsonExtractor - Unable to extract mentions - not valid JSON: [{"word1":"lemma1","word2":"lemma2","...}\n\nPlease provide the sentence you'd like me to process. I'll be happy to help!\n\nHere is the output for the given sentence:\n\n```\n{\n  ":" "," (punct), \n  ":", "," (punct),\n  ":" "," (punct),\n  ":" "," (punct),\n  ":", "," (punct),\n  ":", "," (punct),\n  ":", "," (punct),\n  ":", "," (punct),\n  ":", "," (punct),\n  ":", "," (punct),\n  ":", "," (punct),\n  ":", "," (punct),\n  ":", "," (punct),\n  ":", "," (punct),\n  ":", "," (punct),\n  ":", "," (punct),\n  ":", "," (punct),\n  ":", "," (punct),\n  ":", "," (punct),\n  ":", "," (punct),\n  ":", "," (punct)\n}\n```\n\nHere is the output for the sentence:\n\n{":" "," (punct), \n  ":" ", " (punct),\n  ":", "," (punct),\n  ":", "," (punct),\n  ":", "," (punct),\n  ":", "," (punct),\n  ":",]

The text to annotate is as below:
Beijing and Kuala Lumpur agreed on Wednesday to strengthen bilateral cooperation to take the building of the China-Malaysia community with a shared future to a higher level, as they celebrated the 50th anniversary of the establishment of diplomatic ties.
During their talks in Kuala Lumpur, Premier Li Qiang and Malaysian Prime Minister Anwar Ibrahim agreed that the two countries will continue to promote high-quality implementation of the Regional Comprehensive Economic Partnership agreement, and strive to advance the conclusion of negotiations for Version 3.0 of the China-ASEAN Free Trade Agreement at an early date in order to build a closer China-ASEAN community with a shared future.

The recommender log is still the same as the screenshot in the post above.
I occidentally drag to select the source text, the recommender response seems working as the screenshot below, althougn only the first paragraph:

[reckart]

For some reason, it looks like Ollama did not respond with valid JSON to at least one prompt:

inception-app-1 | 2024-06-30 08:11:30 ERROR [SYSTEM] MentionsFromJsonExtractor - Unable to extract mentions - not valid JSON: [{"word1":"lemma1","word2":"lemma2","...}\n\nPlease provide the sentence you'd like me to process. I'll be happy to help!\n\nHere is the output for the given sentence:\n\n\n{\n ":" "," (punct), \n ":", "," (punct),\n ":" "," (punct),\n ":" "," (punct),\n ":", "," (punct),\n ":", "," (punct),\n ":", "," (punct),\n ":", "," (punct),\n ":", "," (punct),\n ":", "," (punct),\n ":", "," (punct),\n ":", "," (punct),\n ":", "," (punct),\n ":", "," (punct),\n ":", "," (punct),\n ":", "," (punct),\n ":", "," (punct),\n ":", "," (punct),\n ":", "," (punct),\n ":", "," (punct),\n ":", "," (punct)\n}\n\n\nHere is the output for the sentence:\n\n{":" "," (punct), \n ":" ", " (punct),\n ":", "," (punct),\n ":", "," (punct),\n ":", "," (punct),\n ":", "," (punct),\n ":",]

Might be a bug in the Ollama version you are using. Try using the latest version.

Mind that the recommender will fail for long sentences because it is currently internally limited to a response of max 300 tokens.

Also remember that the prompt should ask the model to produce JSON and optimally provide a suitable example (see my screenshot) so that it produces a sensible response.

[fishfree]

Thank you very much! I will try later. BTW: Why limit 300 tokens?

[reckart]

Mainly because I wanted to somewhat limit the time allowed to the LLM to respond. The idea is to make that setting configurable, but so far it is not.