Skip to content

fix: modifications to granite formatter tests - #703

Merged
jakelorocco merged 2 commits into
mainfrom
jal/fix-g-fmt-tests
Mar 25, 2026
Merged

fix: modifications to granite formatter tests#703
jakelorocco merged 2 commits into
mainfrom
jal/fix-g-fmt-tests

Conversation

@jakelorocco

@jakelorocco jakelorocco commented Mar 19, 2026

Copy link
Copy Markdown
Contributor

Misc PR

Type of PR

  • Bug Fix
  • New Feature
  • Documentation
  • Other

Description

  • Link to Issue: Fixes N/A

Added pytest markers for the LLM based tests. Changed when yaml collection is done so that tests don't fail at collection time.

Tested the changes and am able to run the tests even with an unavailable adapter; tests now fail during execution and other tests can run:

================================================================================== short test summary info ==================================================================================
ERROR test/formatters/granite/test_intrinsics_formatters.py::test_canned_input[answer_relevance_classifier] - ValueError: Intrinsic 'answer_relevance_classifier' as LoRA adapter on base model 'granite-4.0-micro' not found in ibm-granite/granite-lib-rag-r1.0 repository on Hugging Face Hub. Sear...
ERROR test/formatters/granite/test_intrinsics_formatters.py::test_openai_compat[answer_relevance_classifier] - ValueError: Intrinsic 'answer_relevance_classifier' as LoRA adapter on base model 'granite-4.0-micro' not found in ibm-granite/granite-lib-rag-r1.0 repository on Hugging Face Hub. Sear...
ERROR test/formatters/granite/test_intrinsics_formatters.py::test_canned_output[answer_relevance_classifier] - ValueError: Intrinsic 'answer_relevance_classifier' as LoRA adapter on base model 'granite-4.0-micro' not found in ibm-granite/granite-lib-rag-r1.0 repository on Hugging Face Hub. Sear...
ERROR test/formatters/granite/test_intrinsics_formatters.py::test_run_transformers[answer_relevance_classifier] - ValueError: Intrinsic 'answer_relevance_classifier' as LoRA adapter on base model 'granite-4.0-micro' not found in ibm-granite/granite-lib-rag-r1.0 repository on Hugging Face Hub. Sear...
ERROR test/formatters/granite/test_intrinsics_formatters.py::test_run_ollama[answer_relevance_classifier] - ValueError: Intrinsic 'answer_relevance_classifier' as LoRA adapter on base model 'granite-4.0-micro' not found in ibm-granite/granite-lib-rag-r1.0 repository on Hugging Face Hub. Sear...
==================================================================== 74 passed, 1 xfailed, 5 errors in 234.83s (0:03:54) ====================================================================

Testing

  • Tests added to the respective file if code was changed
  • New code has 100% coverage if code as added
  • Ensure existing tests and github automation passes (a maintainer will kick off the github automation when the rest of the PR is populated)

@github-actions

Copy link
Copy Markdown
Contributor

The PR description has been updated. Please fill out the template for your PR to be reviewed.

@mergify

mergify Bot commented Mar 19, 2026

Copy link
Copy Markdown

Merge Protections

Your pull request matches the following merge protections and will not be merged until they are valid.

🟢 Enforce conventional commit

Wonderful, this rule succeeded.

Make sure that we follow https://www.conventionalcommits.org/en/v1.0.0/

  • title ~= ^(fix|feat|docs|style|refactor|perf|test|build|ci|chore|revert|release)(?:\(.+\))?:

@jakelorocco
jakelorocco force-pushed the jal/fix-g-fmt-tests branch from 1710bdd to 33d40a8 Compare March 19, 2026 13:55
@jakelorocco
jakelorocco marked this pull request as ready for review March 19, 2026 14:17
@jakelorocco
jakelorocco requested a review from a team as a code owner March 19, 2026 14:17
@jakelorocco

Copy link
Copy Markdown
Contributor Author

@frreiss please take a look when you get a chance. Low priority.

@frreiss frreiss left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good generally, one minor issue noted below.

Comment thread test/formatters/granite/test_intrinsics_formatters.py Outdated
@jakelorocco
jakelorocco force-pushed the jal/fix-g-fmt-tests branch from 33d40a8 to 9a20ca3 Compare March 25, 2026 19:44
@github-actions github-actions Bot added the bug Something isn't working label Mar 25, 2026
@jakelorocco
jakelorocco force-pushed the jal/fix-g-fmt-tests branch from 9a20ca3 to 323793c Compare March 25, 2026 19:49
@jakelorocco
jakelorocco enabled auto-merge March 25, 2026 19:50
@jakelorocco
jakelorocco added this pull request to the merge queue Mar 25, 2026
Merged via the queue into main with commit c323c99 Mar 25, 2026
6 checks passed
@jakelorocco
jakelorocco deleted the jal/fix-g-fmt-tests branch March 25, 2026 20:24
akihikokuroda pushed a commit to akihikokuroda/mellea that referenced this pull request May 27, 2026
* fix: properly mark generative tests in granite formatter tests

* fix: move g-formatter test yaml retrieval to test time
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants