2.2 Instance-Specific Mastery

Study Time: 2.5 hours Prerequisites: Module 2.1 (Atomicity Bootcamp) Learning Objectives:

• Distinguish between generic and instance-specific criteria
• Write criteria that reference actual task content
• Avoid the #2 cause of evaluation quality failures
• Transform generic templates into task-specific criteria
• Build specificity into every evaluation

---

Introduction

Your ideal response vision from Module 2.0 should naturally include task-specific elements, the exact terms, facts, and content that define excellence for THIS task. This module ensures your criteria capture those specifics rather than falling back to vague, generic templates.

After atomicity, the second most critical property of evaluation criteria is instance-specificity.

Instance-Specific Criterion: References the ACTUAL content of the specific task being evaluated.

Generic Criterion: Could apply to any task of that type without modification.

Why It Matters

Generic criteria fail because they don't actually evaluate the task at hand, they evaluate a category of tasks. This makes them:

• Vague: No clear connection to what's being evaluated
• Unverifiable: Impossible to check if the criterion was properly applied
• Useless for feedback: Doesn't tell creators what specifically needs improvement

Platforms reject generic criteria because they indicate the evaluator didn't engage with the actual content.

---

2.2.1 Generic vs Instance-Specific

The Core Difference

Generic Criterion: "Is the response accurate?"

• Could apply to: math problems, history questions, science explanations, product descriptions, any factual content
• Problem: Doesn't specify WHAT needs to be accurate

Instance-Specific Criterion: "Does the response correctly state that water boils at 100°C (212°F) at sea level?"

• Applies to: THIS SPECIFIC task about water's boiling point
• Success: Clearly specifies the factual claim to verify

---

Visual Comparison

Task: "Evaluate an AI's explanation of photosynthesis."

Generic	Instance-Specific
❌ "Is the explanation accurate?"	✅ "Does the explanation correctly state the chemical equation (6CO₂ + 6H₂O + light → C₆H₁₂O₆ + 6O₂)?"
❌ "Does it include all required information?"	✅ "Does the explanation mention all three key components (chloroplasts, chlorophyll, glucose production)?"
❌ "Is it well-organized?"	✅ "Does the explanation follow the process sequence (light absorption → water splitting → glucose formation → oxygen release)?"
❌ "Are examples provided?"	✅ "Does the explanation provide an example of a plant that performs photosynthesis (trees, grass, algae)?"

---