EvalInt v0.2.25 can now read Promptfoo's current per-result JSONL export through automatic detection or --format promptfoo.
Release: https://github.com/CAOShurong/evalint/releases/tag/v0.2.25
Contract and research: https://github.com/CAOShurong/evalint/blob/v0.2.25/docs/PROMPTFOO_JSONL.md
This is a maintainer-created feedback request, not evidence of external users, adoption, or broad compatibility.
Useful feedback would include:
- the exact Promptfoo version and command that produced the artifact;
- whether automatic detection selected
promptfoo;
- whether item/provider counts match the producer;
- any
PROMPTFOO_STRIP_* options used;
- a minimal synthetic or carefully sanitized row when a current export is rejected;
- whether unrelated JSONL was falsely classified as Promptfoo.
Please do not upload secrets, real prompts, model outputs, provider configuration, or private eval data. Reduce a report to a synthetic minimal fixture first.
Known boundaries: EvalInt still reads the complete file text into memory; JSONL support is not a streaming or denial-of-service guarantee. If variables and prompt text are stripped, testIdx is only an export-local identity and should not be assumed stable across runs. A clean audit does not prove that all expected results were exported, that the artifact is authentic, or that its scores are semantically correct.
EvalInt v0.2.25 can now read Promptfoo's current per-result JSONL export through automatic detection or
--format promptfoo.Release: https://github.com/CAOShurong/evalint/releases/tag/v0.2.25
Contract and research: https://github.com/CAOShurong/evalint/blob/v0.2.25/docs/PROMPTFOO_JSONL.md
This is a maintainer-created feedback request, not evidence of external users, adoption, or broad compatibility.
Useful feedback would include:
promptfoo;PROMPTFOO_STRIP_*options used;Please do not upload secrets, real prompts, model outputs, provider configuration, or private eval data. Reduce a report to a synthetic minimal fixture first.
Known boundaries: EvalInt still reads the complete file text into memory; JSONL support is not a streaming or denial-of-service guarantee. If variables and prompt text are stripped,
testIdxis only an export-local identity and should not be assumed stable across runs. A clean audit does not prove that all expected results were exported, that the artifact is authentic, or that its scores are semantically correct.