Performative Benchmarks vs Self-Reported Proficiency: What Actually Predicts Outcomes
Most enterprises measure AI skill by asking people how good they are at AI. Self-report is biased in a predictable direction: the operators who use AI poorly tend to overestimate their skill, and the operators who use it well tend to underestimate it. Performative benchmarks replace assertion with observation — measuring what operators actually do from content-free token telemetry, not what they say about themselves. This article walks through the self-report problem, the performative alternative, the five canonical metrics MO§ES™ derives from behavior, and why bespoke evaluations close the gap between generic benchmarks and enterprise-specific outcomes.
The self-report problem.
Surveys and self-assessments are the dominant way companies measure AI skill. They are cheap, fast, and require no instrumentation. They are also systematically wrong.
When an enterprise wants to understand how its operators are doing with AI, the default instrument is a questionnaire. "Rate your proficiency with large language models on a scale of 1 to 5." "How often do you use AI in your daily work?" "Do you feel confident prompting for complex tasks?" The results get aggregated into a dashboard, a training plan gets drafted, and leadership believes it has a picture of organizational AI capability.
It does not. It has a picture of organizational AI confidence. The two are not the same thing, and the gap between them is not random — it is structured.
The structure is the one most people know from the Dunning-Kruger literature: operators who use AI poorly tend to overestimate their skill, because the very gaps that make them poor operators also prevent them from recognizing those gaps. An operator who has never structured a cascade, never managed context across a multi-turn session, and never iterated against an evaluation rubric has no internal model of what good operation looks like — so "good" defaults to "I got an answer I was happy with." Meanwhile, operators who use AI well tend to underestimate their skill, because competence in this domain comes with an awareness of how much more is possible. The operator who has built a working cascade architecture knows exactly where it breaks, which makes them more cautious about claiming proficiency than the operator who has never tried.
The net effect is that self-report flattens and inverts the signal. The top of the survey distribution is not the top of the performance distribution. It is the top of the confidence distribution. And confidence, in this field, is inversely correlated with the kind of meta-cognitive awareness that marks actual skill.
This is not a minor measurement problem. It is a foundational one. If your instrument cannot distinguish between an operator who is genuinely effective and an operator who merely feels effective, every downstream decision built on that instrument — training allocation, role assignment, tool selection, intervention design — inherits the error. You train the people who already know they need training and miss the people who don't know they do. You promote the confident and overlook the competent. You build interventions for a population that looks, on paper, more skilled than it is.
Self-report measures confidence, not capability. Capability is what you need to measure.
Performative benchmarks.
Instead of asking "how good are you at AI?", measure what operators actually do. Token telemetry reveals the cascade architecture: how they structure context, how they iterate, how they manage the token economy. This is performative — measured from behavior, not assertion.
The alternative to self-report is not a better survey. It is a different kind of instrument entirely. A performative benchmark measures operator behavior under observed or defined operating conditions, then compares that behavior against a reference population. The operator never has to tell you how good they are. Their behavior tells you.
The key insight is that AI operation leaves a telemetry trail. Every API call carries structural signals — how many tokens went in, how many came out, how much context was reused from a prior turn, how much new context was written. These four signals — INPUT, OUTPUT, CACHE READ, CACHE WRITE — are the canonical telemetry surface. They are content-free: no prompt text is required to compute any core metric. The operator's words are never read. Their behavior is measured.
What that behavior reveals is the cascade architecture — the structure of how an operator runs a multi-turn AI session. A high-performing operator does not simply type a prompt and accept the first response. They build context over multiple turns, reuse prior context efficiently, write new context when the task demands it, and produce output that represents a meaningful share of the total token flow. A low-performing operator may move a similar volume of tokens but with a completely different structure — high input, low reuse, low output share, no iteration. The token counts can look similar. The architecture is completely different. The performative benchmark sees the architecture; the self-report survey sees neither.
This is why the measurement is called performative. It is derived from performance — from what the operator actually does — rather than from assertion. The operator does not self-classify. The telemetry classifies them. And because the telemetry is content-free, the measurement can run at enterprise scale without the privacy and governance concerns that come with reading prompt text. You measure how people operate, not what they say.
The result is a benchmark that is both more accurate and more scalable than self-report. More accurate, because it cannot be fooled by confidence. More scalable, because it does not require anyone to fill out a form. The telemetry is already being generated. The benchmark is a lens on data that already exists.
The five canonical metrics.
MO§ES™ derives five canonical metrics from content-free token telemetry. Each captures a different dimension of operator performance. Together they form a profile — not a single score, but a shape.
The four canonical telemetry signals — INPUT (I), OUTPUT (O), CACHE READ (R), and CACHE WRITE (W) — are combined into five derived metrics. No single metric is "the answer." Each one illuminates a different facet of how an operator runs a cascade. The canonical metric is Yield, but the full picture requires all five.
(R + W) / I
How much context the operator reuses and builds relative to new input. Higher leverage means the operator is getting more mileage out of existing context rather than feeding fresh input every turn. It captures context reuse efficiency — the operator's ability to compound prior work instead of restarting from scratch.
O / (I + O + R + W)
Productive output share of total token flow. Yield is the canonical metric: it asks what fraction of the entire token economy the operator converted into useful output. A high-yield operator turns a larger share of their token budget into results. A low-yield operator spends tokens on input and context management without converting them to output.
O / (I + O + R)
Signal-to-noise ratio for productive generation. Token SNR measures output relative to input and reused context, excluding newly written context. It isolates the efficiency of the generation step itself — how much signal the operator extracts per unit of input and reused context, independent of how much new context they are building.
turns / window
Iteration speed — how rapidly the operator cycles through turns within an evaluation window. Velocity captures the operator's cadence: how quickly they move from one turn to the next, how aggressively they iterate. High velocity without yield is churn; high velocity with yield is productive iteration.
W / R
Ratio of new context built to context reused. Construction captures context building quality — whether the operator is writing fresh context (high construction) or leaning on reused context (low construction). Neither extreme is universally better. The right construction ratio depends on the task, the model, and the workflow.
The important thing about these metrics is that none of them is a knowledge score. None of them asks whether the operator knows what a cascade is, or can define prompt engineering, or has completed a training module. They measure behavior. An operator who has never heard the word "leverage" but who naturally reuses context across turns will score well on Leverage. An operator who has read every paper on prompt engineering but who restarts every session from scratch will score poorly. The metrics see what the operator does, not what the operator knows about doing.
I = Input, O = Output, R = Cache Read, W = Cache Write. All metrics are computed from canonical telemetry. No prompt content is required. Velocity is computed from turn timestamps within the evaluation window.
Bespoke evaluations.
Generic benchmarks tell you how someone performs on a standardized task. Bespoke evals tell you how they perform on your tasks, with your models, in your workflows. The gap between generic and bespoke is where insight lives.
The five canonical metrics are general-purpose. They work for any operator, any model, any workflow, because they are derived from telemetry signals that every AI interaction produces. That generality is a strength — it means you can compare operators across teams, across organizations, and against a reference population without custom instrumentation. But generality is also a limit. A generic benchmark tells you how an operator performs in the abstract. It does not tell you how they perform on the specific tasks your enterprise actually runs.
This is the gap between standard and bespoke evaluation. A standard operator eval applies common metrics to common conditions. A bespoke enterprise eval is built around your workflows, your roles, your models, your tasks, and your operating conditions. It asks the questions your organization actually needs answered: which operators are most effective on the customer-support cascade? Who structures context best for the code-generation workflow? Where is the divergence between high usage and high performance on the research-summarization task?
The gap between generic and bespoke is where insight lives. An operator might score well on a generic benchmark — high yield, strong leverage, good construction — and still underperform on your specific workflow, because your workflow rewards a different operating pattern. Or the reverse: an operator with middling generic scores might be the strongest performer on your highest-value task, because the task plays to a structural strength that the generic benchmark does not weight. The generic benchmark gives you a baseline. The bespoke eval gives you the answer to the question you actually care about.
Building a bespoke eval is not a research project. MO§ES™ turns enterprise-defined conditions — roles, workflows, tasks, models, desired behaviors, constraints, operational goals, risk boundaries, performance questions — into repeatable, measurable evaluations. The pipeline is the same one used for standard evals: define the question, identify the signals, build the eval, validate, deploy. The difference is that the question and the signals are yours. Configurations are saveable as JSON and shareable across CLI, TUI, and MCP, so a bespoke eval built for one team can be reused across the organization.
The result is measurement that is both comparable and relevant. The canonical metrics give you a common language for operator performance across the enterprise. The bespoke evals give you performance answers for the workflows that actually drive your business. You need both. Generic without bespoke is academically interesting and operationally useless. Bespoke without generic is a pile of team-specific numbers with no reference frame. Together they form a measurement system that can answer the question every enterprise is actually asking: who is effective at the work that matters, and how do we help more operators get there?
See the full framework.
This article covers the argument for performative benchmarks over self-report. The complete methodology — operator evaluation definitions, benchmark construction, bespoke eval design, cohort analysis, intervention testing, governance labels, limitations, and falsifiability — lives on the methodology page.
Every measurement carries an evidence label. Outcome joins are labeled ASSOCIATION — never CAUSATION. Composite scores are DEVELOPMENTAL — they route workflows, not people. No punitive labels. No operator rankings. Telemetry is content-free.