Hello, I'm trying to use the certainty intrinsic with granite-speech-3.3-8b (which has granite-3-3-8b-instruct as its underlying model). All I get is 0.1 as a certainty measure. Here is an example of what I'm seeing (note: I convert to a percentage using the following code):
try:
chat_input = Granite3Point3Inputs.model_validate(
{
"messages": copy.deepcopy(
self.conv_data[conv_id].get(
PRXC.KEY_CHAT, []
)
),
"documents": [],
"generate_inputs": {
PRXC.KEY_TEMPERATURE: 0.0,
PRXC.KEY_MAX_TOKENS: 4096,
},
}
)
# append the assistant's response
next_chat_input = chat_input.with_next_message(
AssistantMessage(content=response)
)
next_chat_input = (
next_chat_input.with_addl_generate_params(
{"temperature": 0.0}
)
)
# Pass our example input through the I/O processor and retrieve the result
chat_result = await self.certainty_io_proc.acreate_chat_completion(
next_chat_input
)
percent = 0
try:
if (
chat_result.results[0].next_message.content
is not None
):
percent = int(
float(
chat_result.results[
0
].next_message.content
)
* 100.0
)
except ValueError:
pass
cert_sentence = f"This answer is provided with certainty of {percent} percent"
I also had to update consts.py to add:
_GRANITE_SPEECH_3_3_8B_HF = "ibm-granite/granite-speech-3.3-8b"
and add a reference to _GRANITE_SPEECH_3_3_8B_HF wherever _GRANITE_3_3_8B_HF was referenced in:
granite_3_3.py, granite_3_3_input_processor.py, and granite_3_3_output_processor.py
I also had to update certainty.py to reference 3.3 and 3Point3 where 3.2 and 3Point2 were left in tact.
Here is an example of only getting certainty scores of 0.1:
You (please say something): hey diddle diddle, who jumped over the moon?
Assistant: In the traditional nursery rhyme "Hey Diddle Diddle," the character who jumps over the moon is the "little boy" (sometimes referred to as a "cat"). The rhyme goes: "Hey diddle diddle, the cat and the fiddle, The cow jumped over the moon, The little boy blue, came tumbling after." So, the direct answer to your question is the little boy.
This answer is provided with certainty of 10 percent
You (please say something): you're wrong it was the cow
Assistant: Apologies for the confusion. You are correct. In the nursery rhyme "Hey Diddle Diddle," it is the cow that jumps over the moon, not the little boy. The line "The cow jumped over the moon" is indeed part of the rhyme. My previous response was inaccurate in stating that the little boy jumped over the moon. Thank you for pointing this out.
This answer is provided with certainty of 10 percent
You (please say something): who won the 2020 presidential election
Assistant: Joe Biden won the 2020 United States presidential election.
This answer is provided with certainty of 10 percent
You (please say something): did donald trump win the twenty-twenty presidential election?
Assistant: No, Donald Trump did not win the 2020 presidential election. Joe Biden was declared the winner.
This answer is provided with certainty of 10 percent
10:22
I updated the code to reference 3.3 and 3Point3 in the certainty.py but it didn't seem to help...
Perhaps the granite-speech-3.3-8b's underlying instruct model is out of date?
Hello, I'm trying to use the certainty intrinsic with granite-speech-3.3-8b (which has granite-3-3-8b-instruct as its underlying model). All I get is 0.1 as a certainty measure. Here is an example of what I'm seeing (note: I convert to a percentage using the following code):
I also had to update consts.py to add:
_GRANITE_SPEECH_3_3_8B_HF = "ibm-granite/granite-speech-3.3-8b"
and add a reference to _GRANITE_SPEECH_3_3_8B_HF wherever _GRANITE_3_3_8B_HF was referenced in:
granite_3_3.py, granite_3_3_input_processor.py, and granite_3_3_output_processor.py
I also had to update certainty.py to reference 3.3 and 3Point3 where 3.2 and 3Point2 were left in tact.
Here is an example of only getting certainty scores of 0.1:
You (please say something): hey diddle diddle, who jumped over the moon?
Assistant: In the traditional nursery rhyme "Hey Diddle Diddle," the character who jumps over the moon is the "little boy" (sometimes referred to as a "cat"). The rhyme goes: "Hey diddle diddle, the cat and the fiddle, The cow jumped over the moon, The little boy blue, came tumbling after." So, the direct answer to your question is the little boy.
This answer is provided with certainty of 10 percent
You (please say something): you're wrong it was the cow
Assistant: Apologies for the confusion. You are correct. In the nursery rhyme "Hey Diddle Diddle," it is the cow that jumps over the moon, not the little boy. The line "The cow jumped over the moon" is indeed part of the rhyme. My previous response was inaccurate in stating that the little boy jumped over the moon. Thank you for pointing this out.
This answer is provided with certainty of 10 percent
You (please say something): who won the 2020 presidential election
Assistant: Joe Biden won the 2020 United States presidential election.
This answer is provided with certainty of 10 percent
You (please say something): did donald trump win the twenty-twenty presidential election?
Assistant: No, Donald Trump did not win the 2020 presidential election. Joe Biden was declared the winner.
This answer is provided with certainty of 10 percent
10:22
I updated the code to reference 3.3 and 3Point3 in the certainty.py but it didn't seem to help...
Perhaps the granite-speech-3.3-8b's underlying instruct model is out of date?