Skip to content

chore(adapters): bump eval-hub-sdk to v1.0.2 and fix compat issues - #123

Merged
julpayne merged 1 commit into
mainfrom
chore/bump-sdk-1.0.2
Aug 21, 2026
Merged

chore(adapters): bump eval-hub-sdk to v1.0.2 and fix compat issues#123
julpayne merged 1 commit into
mainfrom
chore/bump-sdk-1.0.2

Conversation

@julpayne

Copy link
Copy Markdown
Collaborator

Summary

  • Bumps eval-hub-sdk[adapter] to v1.0.2 across all 10 adapters
  • Fixes breaking changes exposed by the version jump from older SDK versions (0.1.7–0.4.2) to 1.0.2:
    • ragas, deepeval: remove creds.auth_headers fallback — ModelCredentials dropped that attribute; api_key is the canonical field
    • mteb: rename "state" to "status" in test_local_hf.py to match the BenchmarkStatusEvent payload emitted by the SDK
    • clear: set OPENAI_API_KEY in test_clear_happy_path so _ensure_openai_api_key_for_litellm returns early instead of calling resolve_model_credentials with the placeholder secret_ref

This is the base PR in a stack. The follow-up PR wires up configure_telemetry() on top of these changes.

Test plan

  • Verified ragas, deepeval, clear, mteb unit tests pass locally with SDK from main
  • CI should pass once v1.0.2 is available from PyPI (already published)

Made with Cursor

Bump eval-hub-sdk[adapter] to 1.0.2 across all 10 adapters and fix
breaking changes introduced by the major version jump:

- ragas, deepeval: remove auth_headers fallback — ModelCredentials
  dropped that attribute; api_key is the canonical field
- mteb: rename "state" to "status" in test_local_hf.py to match the
  BenchmarkStatusEvent payload emitted by the SDK
- clear: set OPENAI_API_KEY in test_clear_happy_path so
  _ensure_openai_api_key_for_litellm returns early instead of calling
  resolve_model_credentials with the placeholder secret_ref

Assisted-by: Cursor
Signed-off-by: Julian Payne <julpayne@redhat.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
@julpayne
julpayne requested a review from a team as a code owner August 21, 2026 14:16
@coderabbitai

coderabbitai Bot commented Aug 21, 2026

Copy link
Copy Markdown

Warning

Review limit reached

@julpayne, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 54 minutes

Limit details: You’ve used the included review currently available.

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

How can I continue?

Wait for the limit to reset, then comment @coderabbitai review or push new commits to the PR.

An organization admin can change what happens after included review limits in Billing.

How do review limits work?

CodeRabbit enforces per-developer PR review limits within each organization.

For paid Pro and Pro+ reviews, CodeRabbit uses a developer's included PR review attempts over the past 7 days to set the current hourly allowance. At typical activity levels, the full plan allowance applies. Higher sustained activity can lower the allowance until earlier attempts leave the 7-day window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 3b57b5a8-34ea-4445-a7bb-04b65ca53f24

📥 Commits

Reviewing files that changed from the base of the PR and between 3022675 and 603e388.

📒 Files selected for processing (14)
  • adapters/clear/requirements.txt
  • adapters/clear/tests/test_adapter.py
  • adapters/deepeval/main.py
  • adapters/deepeval/requirements.txt
  • adapters/guidellm/requirements.txt
  • adapters/inspect/requirements.txt
  • adapters/lighteval/requirements.txt
  • adapters/mteb/requirements.txt
  • adapters/mteb/tests/test_local_hf.py
  • adapters/ragas/main.py
  • adapters/ragas/requirements.txt
  • adapters/ruler/requirements.txt
  • adapters/swebench/requirements.txt
  • adapters/wildguard/requirements.txt

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@julpayne
julpayne merged commit 6e63f64 into main Aug 21, 2026
17 checks passed
@julpayne
julpayne deleted the chore/bump-sdk-1.0.2 branch August 21, 2026 14:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants