added datasets and models for text generation evaluation - #291
Conversation
|
Thanks, this is great! Can you also run these evals and add the numbers to the leaderboard readme? |
|
@ashish3586 can you please provide a short description of your transformation? |
Saad-Mahamood
left a comment
Saad-Mahamood
left a comment
There was a problem hiding this comment.
Minor change required. Just add the DocStrings for the input and return parameters for each of the functions.
| dataset = KeyValueDataset.from_huggingface( | ||
| hf_dataset, TaskType.TEXT_TO_TEXT_GENERATION, ["text", "summary"] | ||
| ) | ||
|
|
There was a problem hiding this comment.
Missing return statement for "billsum".
| dataset = KeyValueDataset.from_huggingface( | ||
| hf_dataset, TaskType.TEXT_TO_TEXT_GENERATION, ["text", "summary"] | ||
| ) | ||
|
|
There was a problem hiding this comment.
Also, I would suggest adding the 'else' block and raising exceptions with the proper message.
| "summarization", model=model_name, tokenizer=model_name | ||
| "summarization", model=model_name, tokenizer=model_name, device=0 if is_cuda else -1) | ||
| #percent = f"[{split.split('[')[-1]}" if "[" in split else "" | ||
| #if dataset_name == "wikihow": split = "all[:1%]" # f"all{percent}" |
There was a problem hiding this comment.
I think we can remove this commented code.
| #if dataset_name == "wikihow": split = "all[:1%]" # f"all{percent}" | ||
|
|
||
| dataset = _process_data(dataset_name, split) | ||
| print( |
There was a problem hiding this comment.
Duplicate print statement.
|
|
||
| for example in dataset: | ||
| for i,example in enumerate(dataset): | ||
| print(i) |
There was a problem hiding this comment.
Do we need this print statement?
5 participants
No description provided.