> ## Documentation Index
> Fetch the complete documentation index at: https://docs.promptingcompany.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Benchmark

> Read your product's public TPC Benchmarks results, how they changed over time, and the suggestions for improving them from the TPC CLI.

# Benchmark

Benchmark commands are product-scoped and read-only. Pass `--scope <org-slug>/<product-slug>`.

```bash theme={null}
tpc benchmark summary --scope pro-org/my-product
tpc benchmark trend --scope pro-org/my-product
tpc benchmark suggestions --scope pro-org/my-product
```

These commands read your product's results on the public TPC Benchmarks leaderboard: whether AI coding agents can go from your documentation to a first working integration. They do not create experiments. To set up your own competitive benchmark experiments, see [Agent Experience](https://docs.promptingcompany.com/cli/commands/agent-experience.md) and `tpc sim ax setup --type benchmark`.

This is product data for signed-in TPC users. The public TPC Benchmarks plugin for ChatGPT is a separate product, described in [TPC Benchmarks in ChatGPT](https://docs.promptingcompany.com/mcp/benchmarks.md).

Every command accepts `--format json` and prints the raw response data instead of a table.

## How your product matches a vendor

A product shows results when it matches a benchmarked vendor in one of two ways.

| Match | How it happens | What you see |
| - | - | - |
| Linked | The product is linked to the vendor, for example after your team claims it on the Benchmarks page. | Results and suggestions. |
| Domain | The product's website domain matches the vendor's domain. | Results only. The CLI prints "Claim this product on its Benchmarks page to see suggestions". |

A product that matches no vendor prints `Not benchmarked` with a hint to request a benchmark from the product's Benchmarks page in TPC.

## Summary

```bash theme={null}
tpc benchmark summary --scope pro-org/my-product
```

Prints the vendor, how it matched (`linked` or `domain`), the completion rate as passed over total, the experience score, the readiness score, the date it was last tested, the public results link, and a table with one row per task.

| Field | Meaning |
| - | - |
| Completion rate | The share of benchmark runs where the agent finished the task. |
| Experience | A 0 to 100 score for how smooth the integration was. |
| Readiness | A 0 to 100 score from the vendor's latest site readiness scan. |

Tasks that TPC cannot score yet show the status `unsupported` and no completion rate.

## Trend

```bash theme={null}
tpc benchmark trend --scope pro-org/my-product
```

Prints the daily readiness score, then one table per harness and model setup with each day's runs, completion rate, experience score, and the titles of the tasks tested that day.

A day is flagged `tasks changed` when the set of tasks, or a task's version, differs from the previous day of the same setup. Use the flag to tell a real change in results from a change in the benchmark itself.

Run trends need runs on at least three different days. Until then the command prints `Not enough runs yet`.

## Suggestions

```bash theme={null}
tpc benchmark suggestions --scope pro-org/my-product
```

Suggestions appear only for linked products. Open suggestions are grouped by effort, quickest first: low, low-medium, and medium+.

| Type | Where it comes from | What it shows |
| - | - | - |
| Readiness | The vendor's latest site readiness scan. | The goal, the fix, links to the relevant specs, and `Still true as of` the scan date. |
| Experience | Friction that coding agents hit in benchmark runs. | The fix, the number of benchmark runs it is based on with a link to each run, `Still true as of` the last check that confirmed it or `Last evidence` when it was never rechecked, and your response if you accepted it. |

Each experience suggestion is labeled `Spec requirement` when a cited specification requires it, with the specifications listed under `Specs`, or `Optional · TPC readiness credit` when it goes beyond the specification and earns TPC readiness credit.

Experience suggestions are marked `AI-generated`. TPC drafts them from the failed or struggling benchmark runs and reviews them before publishing. A recheck confirms whether the problem still happens, so treat the evidence date as the freshness of the suggestion.

Closed suggestions follow the open ones, newest first. A suggestion closes when the issue stops showing up in scans or runs, or when your team marks it fixed or declines it. Responding to a suggestion, claiming a vendor, and requesting a benchmark are done in TPC, not from the CLI.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.