1.6 Justification Writing

Study Time: 2.5 hours Prerequisites: Module 1.2 Learning Objectives:

• Write evidence-based evaluation rationales
• Structure effective justifications using the dimension reference framework
• Apply the Pyramid Principle to lead justifications with the verdict
• Focus on decisive dimensions and identify meaningful deviations
• Align written justifications with numerical scores
• Provide constructive, actionable feedback
• Document evaluation decisions and edge cases consistently

---

Introduction

"How well you explain your judgment is as important as the judgment itself"

Justification writing is a core skill for AI evaluators. Your written rationales explain why one response is better than another, and this explanation is often as valuable as the rating itself.

---

1.6.1 Structure of an Effective Rationale

Every justification follows this structure:

1. State your conclusion (which response is better, or pass/fail)
2. Ground in specific criteria from the rubric
3. Cite specific evidence from the response
4. Explain the connection between evidence and conclusion

---

1.6.2 Common Requirements (verify against your project guidelines)

Requirement	Definition
Evidence-based	Cite specific content from responses
Criterion-linked	Tie to specific rubric items
Concrete	Point to exact errors/successes
Objective tone	No first-person ("I think")
Quantified	Include counts, percentages where possible
Comparative	Explain why A > B, not just describe each

Note: Different platforms may emphasize different requirements. Verify what your specific project guidelines require.

---

1.6.3 The Dual Standard: Concise but Exhaustive

Justifications must be:

• Concise: No unnecessary words, repetition, or filler
• Exhaustive: Address every significant dimension where responses differ

This is not a contradiction. Being exhaustive means covering all relevant points; being concise means doing so efficiently.

---