Skip to content

code to convert AgentCoder response to markdown - #25

Open
LainWiredIn wants to merge 2 commits into
mainfrom
AgentCoder_FormatFixing
Open

code to convert AgentCoder response to markdown#25
LainWiredIn wants to merge 2 commits into
mainfrom
AgentCoder_FormatFixing

Conversation

@LainWiredIn

Copy link
Copy Markdown
Collaborator

This addition helps address issues with the AgentCoder responses. The responses were noted to be a list of code snippets in non-markdown format. The python script format_fix.py addresses these issues.

@LainWiredIn

LainWiredIn commented Sep 24, 2024

Copy link
Copy Markdown
Collaborator Author

Please use this script to convert the following files to the correct format:

manualRemove_dataset_HumanEvalComm_model_AgentCoder_topn_1_temperature_1.0.log_1
manualRemove_dataset_HumanEvalComm_model_AgentCoder_topn_1_temperature_1.0.log_2
manualRemove_dataset_HumanEvalComm_model_AgentCoder_topn_1_temperature_1.0.log_3

After they have been converted to the correct format, please proceed to run the following commands:

./scripts/script_stepwise_phase123_unix.sh "AgentCoder" 3
./scripts/script_stepwise_phase123_unix.sh "AgentCoder" 4
./scripts/script_stepwise_phase123_unix.sh "AgentCoder" 5

to generate the pass@1 scores, along with other metrics. Please note that you'll have to do a minor change in measurement_summary_draw_heatmap.py

@LainWiredIn

Copy link
Copy Markdown
Collaborator Author

The minor change has been added to the branch.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant