We're trying to reproduce the ComBEE results on Terminal-Bench 2.0 using offline ACE and ComBEE training (60 train / 29 test tasks), but we're unable to match the reported numbers. We're currently using custom prompts for the reflector, curator (vanilla ACE with ADD/UPDATE/REMOVE), batch curator (ComBEE proposals), and reducer (ComBEE aggregation), and suspect prompt differences may be a key factor. @Hanchenli @momoway
We're trying to reproduce the ComBEE results on Terminal-Bench 2.0 using offline ACE and ComBEE training (60 train / 29 test tasks), but we're unable to match the reported numbers. We're currently using custom prompts for the reflector, curator (vanilla ACE with ADD/UPDATE/REMOVE), batch curator (ComBEE proposals), and reducer (ComBEE aggregation), and suspect prompt differences may be a key factor. @Hanchenli @momoway