Every claim about a language model -- "it is smarter," "it is safer," "it follows instructions better" -- is ultimately an empirical claim that demands measurement. Without rigorous evaluation, model selection degenerates into anecdote, vibes, and marketing copy. Evaluation is what separates engineering from alchemy.
The stakes are concrete. If you deploy a model into a customer-facing chatbot, a medical triage assistant, or a code-generation tool, you need to know in advance how it will behave on the distribution of inputs it will actually see. A model that scores 82% on MMLU but hallucinates citation metadata in your legal-document summarizer is worse than useless -- it is a liability. Evaluation is the only mechanism that catches this before production.
Evaluation also drives progress in the field. Benchmarks create shared, comparable signals that let researchers and engineers measure the effect of architectural changes, training-data curation, post-training alignment, and quantisation. When a new technique claims a 5-point gain on GSM8K, that claim is checkable. Without benchmarks, the field would have no ratchet -- every paper would be unfalsifiable.
Language is open-ended. Unlike ImageNet, where a "correct" label is a single class out of 1,000, a language model's output for a given prompt can be any of an unbounded set of strings, most of which are acceptable and many of which are subtly wrong. This makes ground-truth scoring far harder than in classification or regression.
For a prompt like "Explain photosynthesis to a 10-year-old," there is no single correct answer. A response can be accurate but pedagogically weak, or engaging but technically imprecise, or both. Any automated metric that compares against a single reference will penalise legitimately good alternative answers. This is the fundamental weakness of reference-based generation metrics.
The same model can swing 10 or more points on MMLU depending on prompt format (zero-shot vs 5-shot), answer extraction regex, and whether sampling or greedy decoding is used. A leaderboard score is really a score for a specific evaluation protocol, not for the model in the abstract. Two teams evaluating the same model with different harnesses can produce non-comparable numbers.
Once a benchmark becomes a target, it ceases to be a good measure. Models are increasingly trained on data scraped from the same web corpus that benchmarks are drawn from, and post-training is tuned to optimise benchmark-style multiple-choice formats. A model can score high on benchmarks while degrading on open-ended conversational quality -- exactly the signal that motivated the shift toward human-preference evaluation.
The gold standard for "is this response good?" is a human judgement, but human evaluation is slow, expensive, noisy across annotators, and hard to reproduce. The LMSYS Chatbot Arena partially solves this with crowdsourced pairwise comparisons, but it too has biases (verbosity preference, style over substance, position effects) that must be accounted for.
Perplexity (PPL) is the most fundamental measure of a language model's quality. It quantifies how well a model predicts a sample of text -- equivalently, how "surprised" the model is by each token. Lower perplexity means the model assigns higher probability to the actual next token, which means it has learned the statistical structure of the language better.
Given a sequence of tokens w_1, w_2, ..., w_N, the cross-entropy loss per token is:
Perplexity is the exponentiation of this loss:
Intuitively, perplexity is the effective branching factor: if PPL is 10, the model is as uncertain about each next token as if it were choosing uniformly among 10 equally likely candidates.
Perplexity is necessary but profoundly insufficient. A model can have excellent PPL and still be a poor chatbot, a poor instruction-follower, or a poor reasoner. PPL measures next-token prediction, which is what the model is trained for, but it says nothing about:
When evaluating text generation against one or more human reference outputs, three classical metrics dominate: BLEU, ROUGE, and METEOR. All three are n-gram overlap measures -- they compare the model's output to a reference at the level of word or character n-grams, with various refinements.
BLEU (Bilingual Evaluation Understudy) was introduced by Papineni et al. in 2002 for machine translation. It computes a modified precision over n-grams (typically 1- through 4-grams) with a brevity penalty to punish outputs that game precision by being short.
BLEU ranges from 0 to 1 (often reported as 0-100). A BLEU of 0.30 on a translation benchmark is considered decent; 0.50+ is strong. BLEU is precision-oriented -- it rewards n-grams in the candidate that also appear in the reference, but does not directly penalise recall of reference n-grams the candidate missed.
Limitations: BLEU is brittle to synonymy and paraphrase. "The cat sat on the mat" and "A feline rested on the rug" have zero 4-gram overlap but mean the same thing. BLEU also rewards surface form over meaning and is essentially useless for open-ended generation where many valid answers exist.
ROUGE (Recall-Oriented Understudy for Gisting Evaluation) was introduced by Lin (2004) for summarisation. Unlike BLEU, it is recall-oriented -- it measures how much of the reference's n-grams are covered by the candidate.
ROUGE-L F-scores in the 0.30-0.45 range are typical for abstractive summarisation. Higher is better, but ROUGE shares BLEU's weakness: it is a surface-form metric that cannot detect semantic equivalence or factual correctness.
METEOR (Metric for Evaluation of Translation with Explicit ORdering) addresses BLEU's synonymy blindness by incorporating word-stemming and synonym matching via WordNet. It computes a harmonic-mean F-score over unigram matches (exact, stemmed, and synonymic), then applies a fragmentation penalty to reward matches that appear in the same order as the reference.
METEOR generally correlates better with human judgement than BLEU for translation tasks, at the cost of requiring a lexical resource (WordNet or equivalent) for the target language.
| Metric | Orientation | Typical Task | Strength | Key Weakness |
|---|---|---|---|---|
| BLEU | Precision | Machine translation | Fast, simple, reproducible | No synonymy, surface-form only |
| ROUGE | Recall | Summarisation | Good for recall of reference content | Same surface-form blindness |
| METEOR | F-measure | Translation | Stem + synonym matching, better human correlation | Requires lexical resource |
When a model is used for classification -- sentiment, spam, intent, topic, toxicity -- the classical metrics apply. These predate LLMs by decades but remain essential because many LLM applications reduce to classification at the top.
For a binary classifier with a positive and negative class, every prediction falls into one of four cells:
| Predicted Positive | Predicted Negative | |
|---|---|---|
| Actual Positive | True Positive (TP) | False Negative (FN) |
| Actual Negative | False Positive (FP) | True Negative (TN) |
| Scenario | Primary Metric | Rationale |
|---|---|---|
| Balanced classes, equal error cost | Accuracy | Simple and meaningful when classes are balanced |
| Spam detection (false positives costly) | Precision | Sending a real email to spam is worse than letting spam through |
| Cancer screening (false negatives costly) | Recall | Missing a real case is far worse than a false alarm |
| Imbalanced classes, both errors matter | F1 (or F-beta) | Single number that balances both errors |
| Multi-class with class imbalance | Macro-F1 or weighted-F1 | Macro-F1 treats all classes equally; weighted-F1 weights by support |
The workhorse benchmarks for modern LLMs are multiple-choice or completion tasks scored by comparing the model's likelihood of each candidate answer. These are fast, reproducible, and require no human grading, which is why leaderboards gravitate toward them.
MMLU, introduced by Hendrycks et al. (2021), is a 15,708-question multiple-choice test spanning 57 subjects: mathematics, history, law, medicine, computer science, philosophy, and more. Questions range from elementary to professional level. The model is given a question and four answer options; the correct option is whichever the model assigns the highest likelihood to.
MMLU is the single most-cited LLM benchmark, which has made it the primary target for contamination and overfitting. The MMLU-Pro extension increases the answer space to 10 options and adds harder questions to reduce ceiling effects and contamination impact.
HellaSwag (Zellers et al., 2019) tests commonsense reasoning via sentence completion. Each item gives a context paragraph and four candidate completions; one is sensible, the other three are adversarially generated to be plausible-sounding but wrong. The model picks the completion with the highest likelihood.
HellaSwag is approaching saturation -- frontier models score above 95%, which limits its discriminative power. It remains useful as a quick sanity check that a model has not catastrophically broken commonsense.
ARC, from Clark et al. (2018), is a set of grade-school science questions split into an Easy set (~5,197 questions) and a Challenge set (~2,590 questions filtered to defeat retrieval and word-correlation baselines). Questions are 4-way (Easy) or variable-choice (Challenge) multiple choice.
WinoGrande (Sakaguchi et al., 2021) is a Winograd-schema-style coreference resolution benchmark. Each item is a sentence with a blank and two candidate fillers; the model must pick the filler consistent with commonsense. Example: "The trophy didn't fit into the brown suitcase because it was too large. What was too large? (a) the trophy (b) the suitcase." The correct answer requires understanding the physical situation.
GSM8K, introduced by Cobbe et al. (2021), is a set of 8,500 grade-school-level math word problems requiring 2-8 steps of arithmetic reasoning. The model must produce a natural-language chain of reasoning ending in a numeric answer, which is checked against the gold answer by string match.
GSM8K is heavily used as a cheap proxy for "can this model reason?" It has spawned harder derivatives: MATH (competition-level), AIME, and OlympiadBench, which are far more discriminative for frontier models.
HumanEval (Chen et al., 2021, OpenAI) is a code-generation benchmark of 164 hand-written Python programming problems. Each problem has a function signature, docstring, and a set of unit tests. The model generates a function body; the pass@k metric measures the fraction of problems where at least one of k sampled completions passes all unit tests.
Larger code benchmarks -- BigCodeBench, SWE-bench, LiveCodeBench -- now exist for real-world software engineering evaluation, but HumanEval remains the standard quick-check due to its simplicity and low compute cost.
The LMSYS Chatbot Arena (chat.lmsys.org) is a crowdsourced platform where users enter a prompt, two anonymous models generate responses side-by-side, and the user votes on which response is better. Votes are aggregated into an Elo rating -- the same system used in chess -- producing a leaderboard of model quality as perceived by humans.
Contamination -- the inclusion of benchmark test data in a model's training corpus -- is the single biggest threat to benchmark validity in 2026. When a model has memorised the test set, its score measures recall, not capability. The problem is severe because the major benchmarks (MMLU, HellaSwag, ARC, GSM8K, HumanEval) are all publicly available on the web, and modern pretraining corpora are web-scale scrapes.
Several detection strategies exist, none perfect:
A contaminated model can score 10-20 points higher on MMLU than an uncontaminated model of identical true capability. This is not a marginal effect -- it is large enough to flip model rankings. The practical consequence is that leaderboard comparisons across models trained by different organisations, with different (often undisclosed) data filtering, are not strictly comparable.
A model that says "I am 90% sure" should be right 90% of the time. When this holds, the model is well-calibrated. When a model's stated confidence does not match its empirical accuracy, it is mis-calibrated -- and mis-calibration is dangerous in any high-stakes deployment where confidence drives downstream behaviour (routing to a human, abstaining, triggering an alert).
The standard visualisation is a reliability diagram: bin predictions by stated confidence, plot mean confidence vs empirical accuracy per bin. A perfectly calibrated model lies on the diagonal. The two quantitative metrics are:
LLMs do not natively output calibrated probabilities; they output tokens. Asking a model "how confident are you?" produces a verbalised confidence, which research shows is systematically overconfident, especially for factual questions the model gets wrong. Models tend to be confident even when wrong -- the "confident hallucination" problem.
A benchmark score is only meaningful if the evaluation protocol is reproducible. Running MMLU by hand -- writing your own prompt template, your own log-likelihood comparison, your own answer extraction -- is a recipe for non-comparable numbers. Harness tools standardise all of this.
The EleutherAI lm-evaluation-harness (often abbreviated lm-eval) is the de facto standard open-source evaluation framework. It supports hundreds of tasks (MMLU, HellaSwag, ARC, WinoGrande, GSM8K, HumanEval via execution, and many more), multiple model backends (HuggingFace Transformers, vLLM, OpenAI API, Anthropic API, local Ollama), and produces results in a standardised JSON format.
OpenCompass is a parallel framework, widely used in the Chinese AI ecosystem, with broad task coverage and strong support for multimodal models. It is comparable in scope to lm-eval-harness but uses a different config DSL and includes a richer set of multimodal and agent benchmarks out of the box.
| Criterion | lm-eval-harness | OpenCompass |
|---|---|---|
| Standard on HF Open LLM Leaderboard | Yes | No |
| Backend flexibility | Excellent (HF, vLLM, API, Ollama) | Good (HF, vLLM, custom) |
| Multimodal tasks | Limited | Strong |
| Agent / tool-use benchmarks | Limited | Better |
| Documentation quality | Good, active community | Good, bilingual |
| Best for | Open-weight LLM leaderboards, quick local eval | Comprehensive multi-modal / agent evaluation |
Below are concrete command-line examples for running the major benchmarks with lm-evaluation-harness. These assume a Linux host with a CUDA GPU and a HuggingFace model ID. Adjust paths and model IDs to your setup.
This runs 5-shot MMLU, logs per-sample predictions, and writes a JSON results file. Expect a 7B-class model to take roughly 30-60 minutes on a single A100, depending on batching.
Using the vLLM backend gives a substantial speedup over native HF for log-likelihood tasks. The above command runs five benchmarks in zero-shot mode in a single invocation.
HumanEval requires a Python execution sandbox to compute pass@1. The harness handles this automatically but you must run it in an environment with Python available. Never run code execution inside a container without isolation -- the generated code is untrusted.
This pattern works for any OpenAI-compatible endpoint (vLLM server, Ollama, TGI, llama.cpp server, LM Studio). Useful when the model is too large to load directly or you want to evaluate a quantised GGUF served by llama.cpp.
lm-eval --version), the exact model commit hash, the task config revision, the few-shot count, the random seed, the batch size, and the backend (HF vs vLLM vs API). Without these, your benchmark number cannot be reproduced by anyone else, including future you.
Benchmark scores are published in a competitive context where every lab has an incentive to present its model favourably. Reading these numbers critically is a skill. Below are the questions to ask of any benchmark claim.
| Pattern | What It Looks Like | What It Means |
|---|---|---|
| Cherry-picked benchmark | "Beats Model X on GSM8K!" with no other benchmarks shown | The model is probably worse on others; only the best number is shown |
| Mixed shot counts | 5-shot for their model, 0-shot for the competitor | Apples-to-oranges; the comparison is rigged |
| Average of averages | "Average of 6 benchmarks: 72.3%" | The average hides per-benchmark regressions; always ask for the breakdown |
| No confidence intervals | Single point estimates with no error bars | Small differences may be noise; the report hides this |
| Self-reported human eval | "85% human preference win rate" with no methodology | Without an external protocol (e.g., LMSYS), self-reported human eval is not credible |
| Selected subset | "On hard prompts we win 55%" with no definition of "hard" | The subset was likely selected post-hoc to favour the model |
The table below summarises the major benchmarks covered in this reference, their format, what they measure, current saturation status, and primary contamination risk.
| Benchmark | Format | Items | Measures | Saturation | Contamination Risk |
|---|---|---|---|---|---|
| MMLU | 4-way MC, log-likelihood | 15,708 | Broad knowledge across 57 subjects | Frontier 85-90%; not fully saturated | High (public web source) |
| MMLU-Pro | 10-way MC, log-likelihood | 12,032 | Harder, broader MMLU; less ceiling | Frontier 70-80%; discriminative | Moderate (newer items) |
| HellaSwag | 4-way MC, log-likelihood | ~10,000 | Commonsense sentence completion | Saturated (>95% frontier) | High |
| ARC-Easy | 4-way MC | 5,197 | Grade-school science | Saturated (>90%) | High |
| ARC-Challenge | MC, variable choice | 2,590 | Harder science reasoning | Frontier 85-95%; still useful | High |
| WinoGrande | 2-way MC | ~44,000 | Commonsense coreference | Frontier 80-90% | Moderate |
| GSM8K | Generative, exact match | 8,500 | Grade-school math reasoning | Frontier 90%+; mid-size 50-80% | High (solutions on web) |
| MATH | Generative, exact match | 12,500 | Competition-level math | Frontier 50-75%; very discriminative | High |
| HumanEval | Generative, execution | 164 | Python function synthesis | Frontier 90%+ pass@1; saturated | Very high (small, public) |
| HumanEval+ | Generative, execution | 164 | HumanEval with more unit tests | Lower than HumanEval; more robust | High |
| MBPP | Generative, execution | 974 | Entry-level Python tasks | Frontier 85%+; near-saturated | High |
| BigCodeBench | Generative, execution | ~1,000 | Real-world code with libraries | Not saturated; discriminative | Lower (newer) |
| SWE-bench | Agent, execution | ~2,200 | Real GitHub issue resolution | Very hard; frontier 30-50% | Moderate (dynamic variants exist) |
| TruthfulQA | Generative / MC | 817 | Resistance to common misconceptions | Partially discriminative | Moderate |
| BBH (Big-Bench Hard) | Mixed, 23 subtasks | 6,511 | Diverse hard reasoning | Frontier 80-90%; still useful | Moderate |
| LMSYS Chatbot Arena | Pairwise human vote | Millions of votes | Perceived chat quality | Open-ended; not saturable | Low (user-generated prompts) |
| LiveCodeBench | Generative, execution | Grows over time | Code problems dated post-cutoff | Frontier 40-60%; very discriminative | Low by design (date-filtered) |
| Perplexity (WikiText / Pile) | Intrinsic, log-likelihood | Variable | Next-token prediction quality | Not a saturated concept; model/comparator-specific | High if test set is in pretraining |
The central disappointment of LLM evaluation is that benchmark performance and real-world usefulness are correlated but loosely. A model that tops every leaderboard can still fail in your application, and a model that is middling on benchmarks can be the best fit for your specific task. The gap has several root causes.
Benchmarks are curated, clean, and narrowly scoped. Real prompts are messy, multi-turn, domain-specific, and often ambiguous. A model optimised for benchmark-style single-turn multiple-choice questions will not necessarily handle a 12-turn customer-support conversation with embedded context, tool calls, and recovery from errors.
Real applications use specific prompt templates, system prompts, and context windows that differ from benchmark defaults. A model's benchmark score is measured under one prompting protocol; your application uses another. Performance can shift by 10+ points purely from prompt format differences.
Most public benchmarks measure single-shot response quality. Modern applications increasingly use models as agents -- calling tools, maintaining state, recovering from errors, planning over multiple steps. SWE-bench and a few agent benchmarks begin to measure this, but coverage is thin and the benchmarks are expensive to run.
A model that refuses 30% of legitimate requests to be "safe" may score fine on MMLU but be unusable in production. Benchmarks rarely measure refusal calibration -- the false-positive rate on benign requests that get refused -- and yet this is one of the most common reasons a deployed model gets rejected by users.
Two models with identical benchmark scores can have wildly different inference cost. A 70B model and a distilled 8B model may score within 5 points on MMLU, but the 8B is 10x cheaper to serve. Benchmarks measure quality in isolation; production decisions weigh quality against cost, latency, and throughput simultaneously.
The only reliable way to know if a model works for your application is to evaluate it on a curated set of your own tasks, with your own prompts, your own context, and your own definition of correctness -- ideally with both automated scoring (where feasible) and human review (always, for a sample). Public benchmarks are a triage tool to narrow the candidate set, not a substitute for application-specific evaluation.