Skip to content

Latest commit

 

History

History
49 lines (33 loc) · 2.9 KB

File metadata and controls

49 lines (33 loc) · 2.9 KB

Prompt Evaluation Rubric

This rubric can be used to evaluate the quality of AI prompts during exercises or experiments. Each criterion is scored on a scale from 1 (weak) to 5 (excellent).

Criterion Score (1–5) What it Measures
Clarity Is the request clear, unambiguous, and easy to understand?
Context Does the prompt provide enough background information, such as who, what, or why?
Constraints Does the prompt specify clear boundaries, such as length, format, time period, or information to exclude?
Specificity Does the prompt clearly describe the desired response, including details such as structure, tone, or examples?
Expected Output Format Does the prompt specify how the response should be presented (e.g., bullet points, table, essay, or report)?
Ethical Considerations Does the prompt encourage responsible use and avoid requests involving plagiarism, deception, or harmful content?
Adversarial Robustness (optional) How well is the prompt likely to handle ambiguity or reduce the chance of inaccurate or unsupported responses?

How to Use the Rubric

  1. Choose a task, such as explaining a concept or summarising a document.
  2. Write an AI prompt for the task.
  3. Score the prompt against each criterion.
  4. Review the scores and revise the prompt where needed.
  5. Repeat the process to compare improvements.

Example

Initial Prompt

"Tell me about Java."

Possible evaluation:

  • Clarity: 1 (The prompt is ambiguous—it could refer to the programming language, the Indonesian island, or coffee.)
  • Context: 1
  • Specificity: 1

Revised Prompt

"Explain the Java programming language to a beginner. Include its main uses, a short code example, and a brief history. Present the response in three paragraphs."

Possible evaluation:

  • Clarity: 5
  • Context: 4
  • Specificity: 5

The revised prompt provides more context and clearer expectations, making it easier for the AI to produce a focused and relevant response.

Purpose

This rubric complements the CLEAR framework by providing a structured way to review prompts. It can be used to identify areas for improvement and to compare different versions of the same prompt during learning activities or practical exercises.