Skip to content

Hotfix/0.1.1#78

Merged
laurejt merged 6 commits into
mainfrom
hotfix/0.1.1
May 8, 2026
Merged

Hotfix/0.1.1#78
laurejt merged 6 commits into
mainfrom
hotfix/0.1.1

Conversation

@laurejt

@laurejt laurejt commented May 8, 2026

Copy link
Copy Markdown
Contributor

Associated Issue(s): resolves #76

Changes in this PR

  • Fixes concept-eval recipe by adding a custom session factory and disabling auto_exclude_current as suggested by Prodigy support
  • Updates changelog
  • Bumps version number

Notes

  • To reproduce the task routing error, it's important to restart the server after one language's session completes the majority of examples. On the restart, the other language session should only be able to access the small number of examples that have not be annotated.
  • I recommend using the tiny test dataset I created debug-task1.jsonl which contains 5 examples per language for reproducing the error and testing the fix
  • To test the fix, prodigy_recipe_url needs to be updated to https://raw.githubusercontent.com/Princeton-CDH/muse/refs/heads/hotfix/task-routing-fix/src/muse/annotation/annotation_recipes.py
  • I also recommend updating the dataset name prodigy_dataset to something not in the test database to avoid encountering any preexisting annotation data. You can see what datasets are in the database by running prodigy stats -l

Reviewer Checklist

  • Confirm that annotators are served all examples irrespective of any other annotator's progress
  • Confirm that annotators are not re-served examples they've already annotated

@laurejt

laurejt commented May 8, 2026

Copy link
Copy Markdown
Contributor Author

This is effectively a copy of #77 but with the renamed branch

@laurejt laurejt mentioned this pull request May 8, 2026
2 tasks
@laurejt laurejt requested a review from rlskoeser May 8, 2026 21:45
@laurejt laurejt merged commit 8f86f75 into main May 8, 2026
4 checks passed
@laurejt laurejt deleted the hotfix/0.1.1 branch May 8, 2026 21:50
@laurejt laurejt restored the hotfix/0.1.1 branch May 11, 2026 13:19
@laurejt laurejt deleted the hotfix/0.1.1 branch May 11, 2026 13:36
@laurejt laurejt restored the hotfix/0.1.1 branch May 11, 2026 13:40
@laurejt laurejt deleted the hotfix/0.1.1 branch May 11, 2026 13:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Resolve annotation routing and task sharing in prodigy

2 participants