Meta Prompt Evolution

[silverlight6]

Below is directly copy and pasted from the paper.

• Meta prompt evolution: instructions and context suggested by the LLM itself in an
additional prompt-generation step, co-evolved in a separate database analogous to the
solution programs.

Is there any plans to add this to the repository. It quite significantly improved the overall results?
I was looking through the prompt.py file and couldn't find anything related to meta prompt evolution.

[codelion]

Yes, this is an item on the planned roadmap. As a start I am working on updating the prompt optimization example in this PR - #178

As a start we could take the prompt templates in https://github.com/codelion/openevolve/blob/main/openevolve/prompt/templates.py as initial prompts and evolve them similar to how we do for programs.

[silverlight6]

I think what I am talking is a little bit different. Currently, you remove all non-code artifacts from the LLM response during parsing of the response using a regex function. If we instead divide the artifacts into two sections (code vs non-code) then we can isolate the thinking part and the code part separately. The code would go through it's normal pipeline and the response associated with it would be saved along side the result. It would involve a partial rewrite of the code_utils, database, prompt, iteration, and parallel processing files as well as a few configs to decide if you want to use it or not.

The problem I have is I have 0 compute so I can't test if any of this stuff helps or hurts the overall algorithm.

[codelion]

OpenEvolve already supports something similar through its artifact system! See the "Artifacts Channel" section in README.md.

The framework already separates code from non-code artifacts:

  # From README.md example:
  return EvaluationResult(
      metrics={"compile_ok": 0.0, "score": 0.0},
      artifacts={
          "stderr": "SyntaxError: invalid syntax (line 15)",
          "reasoning": "[LLM thinking about the code]"
      }
  )

The "thinking" part is stored as artifacts and included in future prompts when include_artifacts: true.

[silverlight6]

Lets break down what is said in the original paper since there really isn't much.

Meta prompt evolution: instructions and context suggested by the LLM itself in an
additional prompt-generation step, co-evolved in a separate database analogous to the
solution programs.

Instructions and context suggested by the LLM itself

So all of the meta based prompts come from the LLM output rather than that of things like error messages. Instructions tend to be things that come before the code and context tends to come after the code. This does not directly translate to the reasoning output (although it does sound like it could certainly be included). Lots of LLMs do not use reasoning, and many of the locally runnable models don't use it so I like to believe they are more referring to everything except the code in the generated output.

in an additional prompt-generation step

So we are supposed to do another LLM call? This sounds like it is trying to improve on the context that is not the code context that is sent to the LLM calls.

co-evolved in a separate database analogous to the solution programs.

So this sounds like we are doing the exact same process as standard AlphaEvolve but only with the explanation or context of the code. Sounds like we are supposed to be building two evolution pools with meta prompting.

I think I am probably misunderstanding something here. I would like to get your input here to see what you think.

[codelion]

Yes, based on the paper it would appear we need to run two evolutionary processes, one which is already implemented for the code and another one which evolves the prompts in a similar way. We need to define an evaluation metric of the prompts as well (could be the delta it improves the combined_scores of programs or something similar) and sample the best prompts from a database. Right now the system_prompt is defined once and then used for every call with this the system_prompt itself can change with each iteration. Since there aren't many details in the paper we will need to experiment and test a bit to make things work.

[silverlight6]

I apologize for the slow response. I stepped away last night and this is now my morning again.

Has the second evolutionary process been implemented. I see where it is in the database but looking through the iteration and parallel process code as well as the controller, I am not seeing any place where that second evolutionary process occurs.

I kind of like the delta improvement as a score. It is a bit noisy since the code returned could have an error in it and that would make the delta very negative when in fact, it's not a problem with the idea itself but rather a problem with the code execution.

Right now, we also are only including the first 100 artifacts, should that be changed so that it should be the 100 most recent or the 100 related to the top 100 programs or..

Thinking about the system prompt a little bit. Sometimes you want a system prompt to really favor exploration and other times you want it to favor exploitation. I feel like this meta prompt optimization is where you can really make use of that dynamic but how is the question and that is something I don't have an answer for right now.

[silverlight6]

# Use system template override if set if self.system_template_override: system_message = self.template_manager.get_template(self.system_template_override) else: system_message = self.config.system_message # If system_message is a template name rather than content, get the template if system_message in self.template_manager.templates: system_message = self.template_manager.get_template(system_message)

I spent the day reading over the rest of the paper and looking at some of the linked papers to see if I could find some inspiration. The best I could do is find this system message section as the meta prompt.

If we ask an LLM to create a new system message targeted at optimizing the evaluator, then we do a number of evaluations with each system prompt (lets say 100), we can average and get a general value for each prompt. After that first batch, we can do another LLM call and create a new system prompt with the scores of the previous prompt(s) and maybe some of the explanation as well.

The other option is sample a whole number at once. Lets say we sample 10 then rotate through each or pick via a RL policy. The evaluation would be the reward and after x number of steps, we would reset the actions and weights and generate a new set of prompts based on what did well in the first round.