Evaluator Templates

RAG Quality Eval

Retrieval-augmented generation harness with retrieval + synthesis scoring.

Hallucination Detector

Fact-grounded contradiction detection over LLM outputs.

Safety Rubric

PII leakage, jailbreak, and misuse probes.

Cost / Latency Pareto

Model selection front with cost vs. latency vs. score.

Contains key details

An LLM-as-a-judge evaluator for checking whether generated text contains key details.

ROUGE Score

ROUGE (Recall-Oriented Understudy for Gisting Evaluation) scoring — a set of metrics used to evaluate the quality of machine-generated text, particularly in tasks like summarization and translation. Higher ROUGE scores indicate a closer match to the reference text.

Rubric Grader

A general purpose LLM-backed evaluator for grading generated text based on a dynamic marking rubric.