AI Evaluation Glossary
Key terms and concepts used in AI evaluation, data annotation, and RLHF. Essential vocabulary for AI evaluator certification.
AI Evals
Structured tests that measure AI model output quality against defined criteria such as accuracy, helpfulness, safety, and instruction following.
AI Evaluator
A specialist who assesses AI model outputs for quality, accuracy, safety, and alignment with human values and expectations.
AI Safety
The field focused on ensuring AI systems operate reliably, beneficially, and without causing unintended harm to users or society.
AI Trainer
A professional who provides feedback, labels data, and evaluates AI outputs to help train and improve machine learning models.
Ambiguity Resolution
The process of handling unclear or ambiguous annotation tasks by applying consistent interpretive rules from guidelines.
Annotation Guidelines
Detailed instructions that define how annotators should label data, ensuring consistency and quality across a project.
Annotation Taxonomy
A hierarchical classification system of labels and categories used to organize annotation schemes for AI training data.
Calibration (Annotation)
A training process where annotators align their scoring with project standards through practice examples and feedback.
Cohen's Kappa
A statistical metric that measures agreement between two raters while accounting for chance agreement, widely used in annotation quality assessment.
Confidence Score
A numerical value indicating how certain an annotator or AI model is about a particular label or evaluation decision.
Constitutional AI
An AI alignment approach where models are trained to follow a set of principles or rules, reducing the need for extensive human feedback.
Data Annotation
The process of labeling data with meaningful tags, categories, or descriptions to create training datasets for machine learning models.
Data Labeling
The task of attaching informative labels to raw data including text, images, and audio to make it usable for AI training.
Domain Expertise
Specialized knowledge in a subject area that enables evaluators to assess AI outputs requiring technical or professional understanding.
DPO (Direct Preference Optimization)
An alternative to RLHF that directly optimizes language models using human preference data without requiring a separate reward model.
Red Teaming
An adversarial testing approach where evaluators deliberately try to find vulnerabilities, biases, and failure modes in AI systems.
Reward Model
A model trained on human preference data that scores AI outputs, producing the reward signal used to fine-tune language models during RLHF.
RLHF (Reinforcement Learning from Human Feedback)
A machine learning technique where human evaluators provide feedback to train and align AI models with human preferences and values.
RLHF Meaning
Rubric-Based Scoring
An evaluation methodology using predefined criteria and scales to assess AI model outputs consistently and objectively.