PSAIBattlecard Compare AI responses across providers from PowerShell.

PSAIBattlecard is a standalone model comparison dashboard for PowerShell. It uses PSAISuite for multi-provider model execution, then stores prompts, responses, latency, errors, and human ratings in a local JSON file.

Screenshots

Start with a prompt, a model roster, and a history pane that stays close while you compare responses. The dashboard is intentionally simple: one screen for running the comparison, reviewing previous runs, and seeing how each provider answered.

The full view shows the core loop: choose models, run a prompt, inspect the cards, then jump back through prior queries when you want to compare behavior over time.

Prompt And Model Setup

The prompt box is free-form, and the model list is deliberately plain text: one provider:model pair per line. That makes it easy to paste a quick roster, swap providers, or save a repeatable setup in a script.

This keeps the UI light while PSAISuite handles the provider-specific model execution underneath.

Side-By-Side Response Cards

Each model gets its own card with provider, model name, latency, status, and the response body. Differences in format, completeness, speed, and tone become visible without digging through logs.

The JSON store preserves those cards so a quick comparison can become a useful history of model behavior.

Human Ratings

Ratings turn a run into signal. Mark accuracy, relevance, completeness, conciseness, and bias directly on each response, then search or revisit those judgments later.

That closes the loop: PSAIBattlecard is not just a response viewer, it is a lightweight battlecard for deciding which AI is best suited to a real task.

Install

Install-PSResource PSAIBattlecard

During local development from this repository:

Import-Module .\PSAIBattlecard.psd1 -Force

To install the current checkout into your user module path:

.\InstallModule.ps1 -Clean
Import-Module PSAIBattlecard -Force

PSAIBattlecard depends on PSAISuite 0.8.1 or later.

Launch The Dashboard

Show-ModelComparison -Open

By default, the dashboard uses:

$env:LOCALAPPDATA\PSAIBattlecard\model-comparisons.json

You can point it at another JSON store:

Show-ModelComparison -StorePath .\model-comparisons.json -Open

Compare Models

The dashboard prompt box accepts a single prompt. The models box accepts one provider/model pair per line:

openai:gpt-4o-mini
anthropic:claude-sonnet-4-6
deepseek:deepseek-v4-flash
google:gemini-3.1-flash-lite

Click Run Comparison or press Ctrl+Enter. Results appear as side-by-side cards with provider, model, latency, status, response text, and human rating controls.

Run From PowerShell

$models = @(
    'openai:gpt-4o-mini'
    'anthropic:claude-sonnet-4-6'
)

Invoke-ModelComparison -Prompt 'capital of france iceland sweden in json' -Models $models

Search previous runs:

Search-ModelComparison -Query 'Paris'

Rate a response:

Set-ModelComparisonRating -RunId '<run id>' -Model 'openai:gpt-4o-mini' -Accuracy Up -Relevance Up

Commands

• Show-ModelComparison: Launch or render the dashboard.
• Invoke-ModelComparison: Run a prompt against multiple models.
• Get-ModelComparison: Read the JSON store.
• Search-ModelComparison: Search prompts, responses, models, tags, and notes.
• Set-ModelComparisonRating: Store human ratings on a model response.

Demo Materials

The docs/demo-script.md file has a three-minute read-aloud demo script. Paste-ready model and prompt lists live in examples/.

Publish To PowerShell Gallery

Set your Gallery API key, run tests, stage a clean module package, and publish:

$env:PSGALLERY_API_KEY = '<your API key>'
.\PublishToGallery.ps1

To preview the publish operation without sending it:

.\PublishToGallery.ps1 -WhatIf

The publish script packages only the module files needed by Gallery into artifacts/PSAIBattlecard before calling Publish-PSResource.