Protocol for Agentic Evaluation: The Performance–Safety Duality

TL;DR — Most LLM-centered evaluation approaches ask whether the answer is accurate and faithful. Agentic evaluation must ask whether the agent reached its goal and whether it stayed safe along the way. I argue performance and safety are a duality: two evaluation axes that must be quantified together. I also lay out a matrix for doing so across the agentic development lifecycle.

---

Defining Agentic Systems

I use agentic in the operational sense: a system that (a) uses tools beyond text generation: APIs, file I/O, code execution, web actions; (b) executes a multi-step trajectory rather than a single response; and (c) holds autonomy over the next action inside that trajectory, choosing the next tool call from its own state rather than waiting for a user turn. Tool-using assistants, browsing agents, and ops bots all qualify; a pure chat model does not.

A Short History of Evaluation

Agentic evaluation did not appear from nowhere; it is the latest turn of a wheel that NLP, vision, and speech have all turned before. The recurring pattern is worth stating plainly:

Every era of evaluation has progressed metric → standardized toolkit → learned / holistic measure — and each step was driven by the previous metric saturating or being gamed. The agentic era is rapidly evolving at the first step: many metrics, no standard protocol.

Pre-LLM Era

Before instruction-following models, evaluation was a formula problem: given a hypothesis and a reference, could a computable proxy predict human judgment at scale? Each field solved it, then hit the same ceiling when models learned to optimize the metric directly.

Metric	Formalization & Note
BLEU (Papineni et al., 2002, ACL)	$\mathrm{BP}\cdot\exp!\big(\sum_n w_n \log p_n\big)$ — $\mathrm{BP}=\min(1,\exp(1{-}r/c))$ penalizes outputs shorter than the reference ($r$ = ref length, $c$ = candidate); $p_n$ = clipped n-gram precision; $w_n=1/N$ (uniform); dominant MT metric for two decades
ROUGE (Lin, 2004, ACL-WS)	$\mathrm{ROUGE\text{-}N}=\tfrac{\text{matched }n\text{-grams in reference}}{\text{all }n\text{-grams in reference}}$ — recall-oriented counterpart to BLEU; ROUGE-L replaces n-gram overlap with longest common subsequence length; standard for summarization
FID (Heusel et al., 2017, NeurIPS)	$|\mu_r{-}\mu_g|^2{+}\mathrm{Tr}(\Sigma_r{+}\Sigma_g{-}2(\Sigma_r\Sigma_g)^{1/2})$ — Fréchet distance between Inception-feature Gaussians; $\mu,\Sigma$ = mean/covariance of real ($r$) vs. generated ($g$) image sets; lower = closer distributions; displaced Inception Score as the generative-image standard
sacreBLEU (Post, 2018, WMT)	BLEU computed with a canonical tokenizer and an explicit version string embedded in the score — eliminates cross-lab tokenization inconsistency; the toolkit step of the metric → toolkit → learned arc
GLUE / SuperGLUE (Wang et al., 2019, ICLR / NeurIPS)	Macro-average across 8 NLU tasks (GLUE) / 8 harder tasks including co-reference and multi-hop QA (SuperGLUE) — first standardized multi-task suites; established holistic multi-metric reporting and public model ranking
BERTScore (Zhang et al., 2020, ICLR)	$F_\text{BERT}=\tfrac{2\,P_\text{BERT}\,R_\text{BERT}}{P_\text{BERT}+R_\text{BERT}}$ — candidate/reference tokens greedily matched by cosine similarity over BERT embeddings; $P_\text{BERT}$ over candidate tokens, $R_\text{BERT}$ over reference tokens; first embedding-based metric to achieve wide adoption
BLEURT (Sellam et al., 2020, ACL)	BERT encoder fine-tuned end-to-end on WMT human segment-level DA ratings via multi-task pre-training on synthetic perturbations — first metric trained directly on human preference scores; showed reference-based scoring can be replaced by a learned regressor
COMET (Rei et al., 2020, EMNLP)	XLM-R encoder regressed on DA/MQM human judgments; takes source $s$, hypothesis $h$, and reference $r$ as input — unlike BLEU/BERTScore (reference only), captures adequacy and fluency jointly; now the standard MT metric in WMT evaluations

Additional metrics from this era (NIST, METEOR, TER, chrF, MoverScore, CIDEr, SPICE, Inception Score) are catalogued in the appendix.

The pattern is consistent across subfields: surface metrics (BLEU, ROUGE) gave way to reproducible toolkits (sacreBLEU), which in turn gave way to learned metrics that regress directly on human preference (BLEURT, COMET). Each transition was forced: BLEU was gamed, ROUGE saturated, toolkit variants proliferated.

LLM Era

Generative, instruction-following models broke the reference-based paradigm: outputs no longer converge on a fixed correct form, making surface similarity against a single reference increasingly meaningless. The field shifted from measuring closeness to a reference to measuring quality as judged by a model or crowd.

Method	Formalization & Note
pass@k (Chen et al., 2021, arXiv)	$1-\binom{n-c}{k}/\binom{n}{k}$ — P(at least 1 of $k$ randomly drawn samples pass unit tests), estimated from $n$ total generations with $c$ passing; avoids upward bias of reporting best-of-$n$; first widely used behavioral metric
HELM (Liang et al., 2023, TMLR)	Multi-model × multi-scenario evaluation matrix: standardized scenarios (QA, summarization, classification, code, …) × metrics (accuracy, robustness, calibration, fairness, efficiency); identical prompting across all models — holistic standardization; the toolkit step of the LLM-era arc
G-Eval (Liu et al., 2023, EMNLP)	GPT-4 fills a task-specific rubric via chain-of-thought, then the score is $\sum_s s\cdot P(\text{tok}=s)$ over score-value tokens — probability-weighting extracts more signal than argmax; higher human alignment than direct scalar ratings
MT-Bench (Zheng et al., 2023, NeurIPS)	GPT-4 scores responses on 80 two-turn questions across 8 categories (coding, math, reasoning, writing, …) on a 1–10 scale (pointwise); also supports pairwise win-rate — established LLM-as-judge from ad-hoc tool to standard practice
Prometheus (Kim et al., 2024, ICLR)	Open-weight 7/13B judge fine-tuned on 100K GPT-4-generated (feedback, score) pairs over custom rubrics — first open model to match GPT-4 Pearson correlation on fine-grained rubric scoring; enables reproducible evaluation without API dependency
Chatbot Arena (Chiang et al., 2024, ICML)	Anonymous pairwise battles rated by humans; votes update Elo via Bradley-Terry — $P(\text{A}{\succ}\text{B})=\tfrac{e^{R_A}}{e^{R_A}+e^{R_B}}$; ratings from 1M+ human votes; crowd preference at scale; de-facto ranking of instruction-following models

The arc runs the same wheel one more time: surface metric (pass@k), toolkit (HELM), learned/judge-based measure (MT-Bench, Prometheus). Yet it stops before a protocol. Agents raise the stakes further: the unit of evaluation is no longer a string but a trajectory of actions, and a single out-of-bounds action can cause real harm. That demands a measure of behavior under autonomy, not just output quality.

Note: Both tables are curated selections, not exhaustive surveys. Notable omissions in the Pre-LLM era include WMT human evaluation campaigns, the STS benchmark family, and model-based MT metrics (BLEURT-20, COMETkiwi). In the LLM era: AlpacaEval, Arena-Hard, EloHF, WinRate variants, and judge-calibration methods (PandaLM, JudgeBench) are active lines of work not represented here. The appendix and Further Reading section catalogue additional benchmarks.

The Missing Protocol

The agentic toolkit landscape is crowded: Inspect¹, HELM², lm-evaluation-harness³, OpenAI Evals⁴, DeepEval⁵, Ragas⁶, LangSmith⁷, Arize Phoenix⁸, Braintrust⁹; the benchmark count runs into the dozens. Yet almost all of it is single-axis: performance harnesses on one side, safety and security suites on the other.

Emerging standards such as NIST AI RMF¹⁰ and ISO 42001 are real governance frameworks, but compliance-shaped, not runnable eval protocols: they specify what to manage, not how to measure.

The gap is not a missing tool. It is a missing protocol:

• Most evaluation treats performance as the whole job: can the system produce the right output?
• Autonomous agents that call tools, write files, run shell commands, and touch real infrastructure raise the stakes significantly.
• Safety (whether the agent stays within operational and risk boundaries) is bolted on after design, development, and release, if it’s measured at all.

This post makes a simple case: safety should be a first-class evaluation dimension, composed with performance into a single ship criterion, from the first design review onward.

The Performance–Safety Duality

Modern agentic evaluation assesses the end-to-end trajectory of an autonomous system along two fundamental dimensions:

• Performance: the agent’s capacity to execute multi-step workflows successfully.
• Safety: the agent’s adherence to operational and risk boundaries.

These are not independent. They trade off: a more capable agent has a larger action surface and thus a larger attack and risk surface; an over-restrictive safety policy degrades task completion. Evaluating one without the other gives a misleading picture, which is why they form a duality rather than a checklist.

The Duality Formalization

Let an agent $\pi$ produce a trajectory $\tau = (s_0, a_0, s_1, a_1, \ldots, s_T)$ on a task $t$ drawn from a distribution $\mathcal{T}$. Define two trajectory-level indicators:

\[P(\pi, t) = \mathbb{1}[\,\text{goal}(t) \text{ satisfied at } s_T\,], \qquad S(\pi, t) = \prod_{i=0}^{T-1} \mathbb{1}[\,a_i \in \mathrm{allowed}(s_i)\,].\]

$P$ asks did it succeed?; $S$ asks did every action stay in bounds? One out-of-bounds action voids the trajectory. The set $\mathrm{allowed}(s_i)$ is not discovered by the eval — it’s specified up front, in the design-stage threat model: a policy file, a sandbox’s capability grants, a tool allow-list, or a learned classifier standing in for one. If you can’t write down $\mathrm{allowed}(\cdot)$, you don’t yet have a safety spec to test against. Aggregating over $\mathcal{T}$ (and over $k$ seeds to handle non-determinism) gives the ship criterion:

\[U(\pi) \;=\; \mathbb{E}_{t \sim \mathcal{T}}\big[\,P(\pi, t)\cdot S(\pi, t)\,\big] \;\geq\; \tau.\]

The Duality Protocol

Four properties make this a protocol rather than a checklist:

• Multiplicative, not additive. A weighted sum $\alpha P + (1-\alpha)S$ lets a high-performance agent buy back safety failures — the exact mode the duality prevents. The product forbids it: only the upper-right quadrant of Figure 1 survives.
• Asymmetric by design. A failed task ($P=0$) wastes a run; a single out-of-bounds action ($S=0$) can leak data, delete a database, or move money. One $S=0$ in production outweighs a hundred $P=0$s, and only a product encodes that unbounded blast radius.
• A gate, not a taxonomy. Recent agent-eval surveys (Mohammadi et al., KDD 2025¹¹; Yehudai et al., 2025¹²) map the space thoroughly but stop at categorization. The closest formal ancestors are the constrained MDP (Altman, 1999¹³), which separates safety as an additive cost budget rather than composing it multiplicatively, and runtime shielding (Alshiekh et al., AAAI 2018¹⁴), which applies the binary action filter at execution time rather than as a retrospective eval metric. Neither the 2025 surveys nor 2026 practitioner literature uses the explicit $U = \mathbb{E}[P \cdot S]$ ship criterion. τ-bench’s pass^k (Yao et al., ICLR 2025¹⁵) is the closest structural analog: a product of per-turn binary indicators; here it falls out as the estimator for $\mathbb{E}[P\cdot S]$ rather than a standalone consistency score.
• Referenced throughout. CI gates enforce $P$ and $S$ per commit, the matrix specifies how each indicator is computed, and pass@k estimates $\mathbb{E}$ under stochastic trajectories.

The Agentic Evaluation Matrix

Dual Gating

Two dimensions are gating; one is secondary. Performance asks whether the agent solved the task, pushing past text similarity to verify state (did the correct SQL row actually change?) and trajectory (the agent that succeeds in 3 steps beats the one that takes 45). Safety asks whether every action stayed in bounds, measured as refusal accuracy on malicious requests and restraint when granted high-risk capabilities (file writes, shell, fund transfers). Operational metrics (latency, cost, UX) matter for shipping but never override the duality.

Dimension	What to measure	How
Performance	Task success, tool-selection precision, trajectory efficiency	State diffs and hardcoded assertions; trace parsing + API-schema validation; step / token counts1617
Safety	Guardrail adherence, injection resistance, data governance	LLM-as-judge + adversarial suites; red-teaming, canary tokens; PII/credential scanners1810
Operational (secondary)	Latency, cost, UX	OpenTelemetry traces, unit billing, HITL audits19

A Worked Example

Consider a coding agent asked to fix a failing test. The gate reads one run along both axes at once, and the multiplicative rule separates three outcomes that a single accuracy number would blur:

Outcome	$P$	$S$	$P\cdot S$	Verdict
Correct patch, no out-of-bounds actions	1	1	1	Ship
Correct patch, but runs a destructive command	1	0	0	Held (Reckless)
Wrong patch	0	(n/a)	0	Held (Broken/Restrictive)

Efficiency (steps to goal, token cost) rides alongside as a secondary signal — a regression to watch, never a release blocker. Only the first row lands in the upper-right Ship quadrant of Figure 1; the gate holds the other two even when performance is perfect.

---

The Toolkit Landscape

The proposed protocol is method-agnostic, however, in practice you reach for existing tools. The honest state of play: the harnesses are capable and overlapping, but each is anchored to one axis (performance), and one has to assemble the duality yourself.

Toolkit	Origin	Primary axis	Where it fits the protocol
Inspect	UK AISI	Performance + safety scaffolding	Closest to a general harness; composable scorers, sandboxed agents — good base for $P$ and $S$
lm-evaluation-harness	EleutherAI	Performance (static)	De-facto academic runner; weak on trajectories
HELM	Stanford CRFM	Holistic (multi-metric)	Standardized reporting; not agent/trajectory-native
OpenAI Evals	OpenAI	Performance	Assertion + model-graded; light on safety
DeepEval / Ragas	OSS	Quality / RAG	Metric libraries; LLM-as-judge built in
LangSmith / Arize Phoenix / Braintrust	Industry	Observability / ops	Trace capture and online eval; the secondary tier

The pattern: no single toolkit composes performance and safety into one gate. You wire a performance harness to a separate red-team suite and reconcile the two by hand. That’s the protocol gap a thin orchestration layer over these tools can close. (Benchmarks the harnesses run are catalogued in the appendix.)

Evaluation Across the Lifecycle

The duality only pays off if evaluation rides alongside the build, not after it. Each stage of development produces a different class of evidence and enforces a different gate — the protocol is the same, but the instrument changes.

Design. Before any agent code is written, define allowed(s): the set of permitted actions at each state. This is the threat model — a policy file, a capability allow-list, or a formal specification. Without it, there is no $S$ to measure. Alongside it, write explicit success criteria for $P$: what state does the task reach if the agent succeeds? Both must be answerable before implementation begins.

Development / CI. Run a fast pass@k suite (small $k$, cheap judges) on every pull request. Performance gate: do state diffs and hardcoded assertions pass? Safety gate: do adversarial probes (jailbreaks, prompt-injection, out-of-scope capability requests) fail? Both must clear before merge. Failures here are cheap to fix; failures in production are not.

Release and Run. Before expanding capabilities (new tools, broader scope, higher autonomy), run the full adversarial suite and require red-team sign-off. In production, capture telemetry (step counts, cost drift, TTFT) and sample a fixed fraction of trajectories (1–5%) for human review. The audit budget catches reward hacking and distribution shift that aggregate metrics miss.

Open Problems

The protocol above is a working framework, not a finished one. Several methodological gaps remain, even after accounting for what the 2026 update and the blog series address:

• Specifying allowed(s) in practice. The formalization requires writing down the allowed-action set at each state before evaluation begins. In novel domains with open-ended tool use, this is genuinely hard: allow-lists miss emergent attack surfaces and over-restrict legitimate actions. Defining allowed(·) rigorously, and verifying that a learned classifier correctly approximates it, is an open research problem.
• Eval cost at scale. pass@k with an LLM judge multiplies eval cost by $k$ times the judge’s token bill. Tiered suites help, but a fast smoke-set at $k=3$ with a cheap judge and a full suite at $k=10$ with GPT-4 can still run into hundreds of dollars per commit on long-horizon tasks. Cost-aware scheduling, sample-efficient estimators, and cheaper-but-reliable judges are active problems.
• LLM-as-judge bias.¹⁹ Judge models inherit training-data preferences, favoring verbose, confident, or stylistically familiar outputs. Calibrate against human ratings on a held-out slice; prefer rubric-anchored judges over open-ended prompts.
• Benchmark contamination.²⁰ SWE-bench, GAIA, and OSWorld are public; newer base models have likely seen them. Treat top-line scores as a ceiling, not ground truth, and pair with held-out internal tasks that mirror your production distribution.
• Reward hacking on the eval itself.¹⁶ Agents optimized against a metric will exploit it: passing assertions without solving the task, padding traces, or short-circuiting tool calls. Audit a sample of trajectories by hand; aggregate metrics alone are insufficient.
• Alignment-side safety, beyond infosec.²¹ Most safety tooling today targets injection and destructive actions. Harmful content generation, deception, sycophancy, and goal mis-generalization sit outside that surface and require their own probes. The next posts in this series address each in turn.

---

The Expanded Framework (2026 Update)

Since this was first drafted, academic literature and industry practice have matured. Anthropic’s Demystifying Evals for AI Agents (2026)²² is the clearest practitioner playbook to date: concrete grader taxonomies (code / model / human), pass@k versus pass^k as the standard non-determinism estimator, and transcript-plus-outcome as the measurement unit. Three changes from the 2025 baseline: efficiency is promoted to a core pillar (joining performance and safety); UX and system metrics are demoted to supplementary; and each core pillar is now anchored to public benchmarks or standards rather than left as an internal definition. The duality formalization above still holds; efficiency enters as a third factor, gated rather than averaged.

Core pillar	What it measures	How (with anchors)
Performance	Task success, tool-selection accuracy, multi-step reasoning	Unit-test assertions, state diffs, LLM-as-judge — benchmarks: SWE-bench Verified, GAIA, OSWorld20
Safety	Guardrail adherence, refusal precision, privilege-escalation prevention	Adversarial red-teaming, jailbreak suites, canary tokens — anchors: OWASP Agentic AI Top 10, NIST AI RMF1810
Efficiency	Trajectory optimization, loop prevention, economic viability	Trace-graph analytics, step-count tracking, cost calculators — CLASSic metrics (Cost, Latency, Accuracy, Security, Stability)19

Supplementary runtime dimensions (useful but not gating):

• UX & handoff: clarification accuracy, HITL fluidity. Measured via mock-user simulation and trace-replay audits; tracked as escalation frequency and clarifying-question rate.²³
• System metrics: latency, infra reliability, model drift. Captured via OpenTelemetry spans and continuous monitoring (TTFT, session duration, API exception rates) on platforms like Bedrock AgentCore, Azure AI Foundry Evals, and Arize Phoenix⁸.¹⁶²¹

A Bar for 2026 and Beyond

If I had to compress the above into a checklist that distinguishes a serious agentic-eval program from a vibes one:

1. Both axes gated in CI: performance and safety regressions both block merge; safety is not a release-time afterthought.
2. pass@k with k ≥ 5 (or seed-averaged equivalent): no single-run shipping decisions.
3. Held-out internal eval distinct from public benchmarks: public scores reported as ceiling; internal scores as ground truth.
4. Red-team sign-off required for privilege expansion: every new tool, broader scope, or higher autonomy triggers an adversarial pass before rollout.
5. Trace-level audit budget: a fixed fraction (say 1–5%) of production trajectories reviewed by humans on a rolling basis, to catch reward hacking and out-of-distribution failures the aggregate misses.

None of this is free: pass@k with an LLM-judge on every commit multiplies eval cost by $k$ times the judge’s token bill. The practical answer is tiered suites: a fast smoke set (small $k$, cheap judges) gating every push, and the full adversarial suite gating releases and running nightly. Budget the eval like you budget CI minutes; an unaffordable protocol gets switched off.

The Blog Series

This post is the base protocol. Each follow-up extends one axis of the duality, closing a specific 2026 gap:

Next	Focus	Duality link
Autonomy & RSI	Long-horizon, self-improvement guardrails, time-to-task	Safety at high autonomy
Compliance Evaluation	ISO 42001 / NIST AI RMF / EU AI Act as executable tests	Maps $\mathrm{allowed}(s)$ to regulation
The Judge Problem	LLM-as-judge reliability as the instrument itself	Validates the measurement of both axes

Appendix

Further Reading

A short, opinionated list of works that have actually moved the practice of agentic evaluation, not just described it:

• τ-bench (Sierra, 2024) — tool-use agents in realistic customer-service domains; first widely-adopted consistency-over-runs metric (pass^k), making non-determinism a first-class measurement.
• SWE-bench Verified (OpenAI, 2024) — human-validated subset of SWE-bench; raised the bar for what counts as a reliable coding-agent benchmark and made contamination-aware reporting normal.
• OSWorld (Xie et al., 2024) — multimodal computer-use agents in real desktop environments; the canonical hard test for long-horizon GUI agents.
• AgentHarm (Andriushchenko et al., 2024) — first benchmark targeting agentic refusal: does the agent decline harmful multi-step tasks, not just harmful prompts.
• Agent Security Bench (ASB) (Zhang et al., 2024) — systematic attack/defense matrix across prompt injection, tool abuse, and memory tampering; widely cited template for safety suites.
• METR Autonomy Evaluations (METR, 2024–2025) — long-horizon task suites that quantify model autonomy as a time-equivalent metric (“can it do what a human takes N hours to do?”).
• Anthropic Sabotage Evaluations (Anthropic, 2024) — frontier-lab template for evaluating deceptive failure modes that don’t surface in capability or refusal benchmarks.
• Demystifying Evals for AI Agents (Anthropic, 2026) — practitioner playbook: grader taxonomy (code / model / human), pass@k vs. pass^k for non-determinism, and judging transcript and outcome — close industry validation of the reliability side of the duality.
• Holistic Agent Leaderboard (HAL) (Princeton, 2024–2025) — aggregates agent benchmarks with cost and reliability axes alongside accuracy, operationalizing the multi-axis framing this post argues for.

Agentic Benchmarks

Catalogued separately from the protocol: these are tasks a harness runs, not the measure itself. Labeled by venue (peer-reviewed, preprint, or lab release).

Benchmark	Year · Venue (label)	Axis stressed
AgentBench	2023 · ICLR 2024 (peer-reviewed)	Performance (multi-env)
WebArena	2023 · ICLR 2024 (peer-reviewed)	Performance (web)
GAIA	2023 · ICLR 2024 (peer-reviewed)	Performance (general assistant)
SWE-bench	2023 · ICLR 2024 (peer-reviewed)	Performance (code)
SWE-bench Verified	2024 · OpenAI (lab release)	Performance (contamination-aware)
OSWorld	2024 · NeurIPS 2024 (peer-reviewed)	Performance (GUI)
τ-bench	2024 · ICLR 2025 (peer-reviewed)	Performance + reliability (pass^k)
τ²-bench	2025 · arXiv (preprint)	Dual-control conversation
AgentHarm	2024 · arXiv (preprint)	Safety (agentic refusal)
Agent Security Bench	2024 · arXiv (preprint)	Safety (attack/defense)
Vending-Bench	2025 · arXiv (preprint)	Long-horizon coherence
Holistic Agent Leaderboard	2025 · Princeton (framework)	Multi-axis incl. cost

Pre-LLM Evaluation Metrics

Metrics trimmed from the main history table: influential in their subfields but less central to the NLP/MT through-line the agentic era inherits from.

Metric	Formalization & Note
NIST (Doddington, 2002, HLT)	BLEU with information-weighted n-grams (rarer n-grams weighted higher) — variance-weighted BLEU variant
METEOR (Banerjee & Lavie, 2005, ACL-WS)	Recall-weighted harmonic mean of unigram P/R with alignment + fragmentation penalty — synonym- and stem-aware
TER (Snover et al., 2006, AMTA)	$e / \bar{r}$; $e$ = word edits (ins, del, sub, shift), $\bar{r}$ = avg reference length — normalized edit distance
chrF (Popović, 2015, WMT)	Character n-gram F-score, $F_\beta$ weighting recall — tokenization-free; useful for morphologically rich languages
CIDEr (Vedantam et al., 2015, CVPR)	Cosine similarity of TF-IDF-weighted n-gram vectors, averaged over $n$ — image-captioning consensus metric
SPICE (Anderson et al., 2016, ECCV)	F1 over scene-graph tuples (objects, attributes, relations) — semantic-graph captioning metric
Inception Score (Salimans et al., 2016, NeurIPS)	$\exp!\big(\mathbb{E}_x\,\mathrm{KL}(p(y\mid x)\,|\,p(y))\big)$ — generative-image quality/diversity; largely superseded by FID
MoverScore (Zhao et al., 2019, EMNLP)	Word Mover’s Distance (min transport cost) over contextual embeddings — embedding earth-mover; less widely adopted than BERTScore

References