Skip to main content

Model Council v2

Use tpc council-v2 to ask a question, collect model responses, inspect or recluster narratives, assign share directions, and retrieve dashboard data. Every command is noninteractive.

Prerequisites

  • Authenticate with a TPC session token, such as one obtained through tpc auth login or tpc auth claim. Enterprise API keys are not supported.
  • Model Council v2 must be enabled for your organization.
  • Pass --scope <org-slug>/<product-slug> on every command. You can also pass the product’s dashboard URL as the scope.
  • Use --format json to emit one structured result object. Text mode emits indented JSON; CSV is unsupported.

Agent workflow

1. Discover a question (optional)

Returns suggested questions with text, topicId, and topicText. Missing suggestions are generated and cached. You can also supply your own question directly.

2. Create the question and start analysis

Generate and retain a UUID for this operation before issuing the command. The example UUID below is illustrative; use a new UUID for each new question.
Creation starts analysis and exits immediately. The idempotency key becomes the question ID. Retry an interrupted request with the same key and identical input, including models, prompts, scheduled iterations, and initial narratives. Reusing a key with different setup returns a conflict. Without a custom prompt set, setup targets 25 generated prompts and uses ChatGPT and Claude. Inspect the dashboard’s phrasings to confirm the saved prompt set before starting a repeated-run experiment. When selecting a suggestion, also pass --source generated, its --topic-text, and --topic when its topicId is non-null. Defaults for custom questions are --source user, no topic ID, and --topic-text "Custom question". If the question was saved but analysis could not start, the command prints status: "saved", the question ID, and recovery instructions, then exits with code 1. Recover with:
start also retries incomplete or failed analysis. An already-running workflow is reused; fully processed questions need no new run.

Advanced question setup

Choose models, supply reviewed prompts, set scheduled repetitions, and define initial narratives at creation:
setup.json:
Files are limited to 1 MB each. Only one file can read stdin. Set models and iterations in either JSON or flags, not both. Supply a prompt file or selectedPrompts in the setup file, not both; --prompt can append to either. JSON rejects unknown fields. Generated output’s scope and dashboardUrl metadata is accepted; scope must match the resolved target product, including when the command uses the legacy product/org scope order. Prompts must be distinct ignoring case and whitespace, and contain 5–2,000 characters. Optional prompt fields are persona (up to 200 characters), situation (up to 500), and sourcePromptId for a prompt from this product’s library. Edit or remove entries in selectedPrompts before creating the question. Omit selectedPrompts to generate the default set. Supply 1–199 initial narratives with unique labels (1–200 characters), definitions (1–2,000), and optional exclusions (up to 2,000). Others is reserved for unmatched claims. Supplied narratives classify claims from the first run; omit initialNarratives for automatic discovery. Empty arrays are rejected.

Generate and review prompts

--count accepts 1–100 and defaults to 25. Optional --topic-text supplies topic context. This generates a selectedPrompts setup file without creating a question or collecting answers. Review and edit it, then pass --file prompts.json or --prompts-file prompts.json to questions create. Retain that file for retries instead of generating a different set under the same idempotency key. An unmeasurable question exits nonzero with the reason.

Select existing library prompts

Results contain prompts (id, text, topicId), topics, page, and hasMore. Use --topic <id> to filter by topic, and increment --page while hasMore is true. Pages contain up to 25 prompts. Copy chosen entries into your setup file using text and sourcePromptId:
The server checks that each source prompt still belongs to this product and is available. You can edit its selected text without changing the library prompt.

Import prompts or write them directly

Use plain text with one prompt per line, or CSV with exactly one prompt header. Quote CSV prompts containing commas or newlines. Blank prompts are skipped; invalid or duplicate prompts reject the whole request.
JSON prompt imports use the same {"selectedPrompts":[...]} shape as generated output. Use --file when the JSON also contains model, iteration, or narrative settings.

3. Inspect or wait for analysis

get returns the question’s saved setup, current prompts and narratives, processing progress, state, and nextAction: phrasings and clusters can be empty while initial setup is still processing. The original selectedPrompts and initialNarratives remain available before generated rows exist and are distinct from the current prompts and narratives after later edits. question also retains status, errors, grouping state, ready/position timestamps, and phrase tracking. For narrative shares and answer evidence, use narratives list or dashboard. questions watch continues to poll the lightweight status response without fetching setup, prompts, or narratives on each iteration. watch polls without starting or retrying analysis. It stops at awaiting_positions, ready, or failed. The default timeout is five minutes, measured after authentication and scope resolution; the default polling interval is three seconds, with a minimum of one second. Both flags accept Go durations such as 30s or 5m. On timeout, watch prints the last observed state with timedOut: true and exits with code 1. If no status response arrived, the state is unknown. Resume with get or another watch invocation.

4. Inspect narratives and save positions

The response includes current narrative IDs, labels, definitions, shares, and sampled evidence. Unset share directions default to Maintain. Evidence is capped per model; the response includes evidencePerModelLimit. Others contains uncategorized claims and is excluded from narrative lists, share calculations, charts, and response coverage. In the dashboard, open the Others section below the narratives to browse its claims. Previously dismissed claims appear in Others and remain excluded from automatic reclustering. An admin can restore a processed claim by moving it to a named narrative. Create a positions file using the returned major narrative IDs:
Stances express the desired share direction: mine means increase, opposing means reduce, and neutral means maintain. Choose directions using the product’s goals.
Submit every current major narrative exactly once, excluding Others. The API saves the set atomically and rejects stale or incomplete sets. If the narrative set changes, fetch it again before retrying. Use {"marks":[]} when there are no major narratives. Input is limited to 1 MiB.

5. Get the dashboard

The result includes narrative shares and evidence, stance breakdown (framing), modelComparison, top cited URLs, current suggested actions, processing progress, and a dashboardUrl. Date filters are inclusive UTC dates and apply to the dashboard’s claim and citation data; suggested actions and processing progress describe the current question. Either date boundary can be omitted. Evidence and citation caps are included in the response. Phrase tracking is configured per question in the web dashboard and takes effect on the next collection. question.phraseTracking contains the current phrase list, user alternates, and generated grammatical variants. phraseCounts contains response-level counts for the selected range: explicit, implicit, checked, failed, and a variants breakdown. Add explicit + implicit for the combined count; alternate counts can overlap. A response is checked implicitly only when no explicit variant matches. Failed implicit checks and older, untracked responses are not counted as absent. Editing a phrase or its alternates starts a new phrase ID, keeping earlier results separate. To get the product question overview, omit --question:
Both return up to 100 active questions, newest first, with their status and summary metrics. Date flags require --question.

6. Collect a fresh batch (platform admins)

After initial processing finishes and you save positions, use collect to run the saved prompts again:
collect requires a platform-admin session, rather than just an organization-admin role. It uses the same collection operation as the web app’s Run same prompts action, reusing the saved prompts, models, and narrative definitions. Your API deployment must support POST /api/v1/products/{productId}/narrative-split/{questionId}/runs. Generate and retain a new UUID for each intended batch. That UUID becomes the collection’s run ID. Retry an interrupted request with the same key to retrieve or resume that same run. The result contains id, questionId, and a nextCommand to inspect history. A completed key returns its original run, even if a different run is now active; a failed collection can be resumed when its configuration still matches and no other run is active. Only one collection can be queued or running per question. A different request while a collection is active returns HTTP 409 Conflict, without creating another run. Wait for it to finish, then retry your new request. For sequential experiments, use a new UUID for each iteration and count distinct completed run IDs, not successful command invocations. collect returns without waiting. questions runs returns up to 20 collections per page, newest first, with status, errors, expected/completed/failed answer counts, total, page, pageSize, and activeRunId. Follow pagination to find older IDs. Wait until the returned collection ID has status completed before recording its result. questions watch tracks initial analysis, not collection completion. If collection dispatch fails, inspect history and retry with the same key.

7. Read per-run and cumulative results

These flags require --question. --run cannot be combined with dates or --view; latest and auto cannot be combined with dates. The response’s selection describes the chosen scope. Suggested actions and processing progress still describe the current question. For a convergence experiment, save each completed collection’s JSON locally and compare its runSets prompt and definition fingerprints before pooling results. --view all includes scheduled collections and any other answers for the question, so use per-run results when you need exactly your experimental batches. Shares use categorized claims as their denominator; answer coverage is reported separately. Keep the prompt set, model configuration, and narrative definitions unchanged across iterations.

Recluster saved claims

Reclustering replaces the current major narratives and reassigns existing saved claims. It does not collect new answers. New narratives have new IDs and default to Maintain; save positions again after reassignment finishes. Initial analysis must have finished, the question must be ready or failed with an initial ready timestamp, and no answer collection can be active.

Draft and review definitions

Export the current set to refine it, or request a new draft using a prompt:
Drafting does not apply changes. Edit the returned narratives array and retain expectedClusterIds, which captures the current major narrative IDs before draft generation. The output can be passed directly to start --file after review. You can also write your own set using this shape:
Include every current major narrative ID in expectedClusterIds, excluding Others. Use an empty array when there are none. Supply 1–199 narratives with unique labels (1–200 characters, excluding the reserved name Others) and definitions (1–2000 characters). Draft prompts accept 1–10000 characters. Files are limited to 1 MiB. The optional questionId, scope, and dashboardUrl metadata from draft is accepted; question and scope metadata must match the command. Each narrative has a title (label), a positive direction (definition, what to include), and a negative direction (exclusion, what to exclude, at most 2000 characters). Exclusions take precedence over inclusion during claim assignment. Generated drafts supply all three. Existing narratives and older files may omit exclusion or leave it empty, meaning no additional exclusion.

Start and inspect reclustering

Generate and retain a new UUID for each intended reclustering operation:
--file - reads reviewed JSON from stdin. Omit --file to generate narratives automatically using the default clustering instructions. Platform admins may supply --metaprompt (1–20000 characters) for automatic clustering; it cannot be combined with --file. Prompt-based drafting and reviewed sets do not require platform-admin access. start returns immediately with requestId and a request-aware nextCommand. Reviewed sets are checked for stale IDs both at submission and before replacement. If the set changed, export and review it again. --request-id is a workflow correlation ID, not a general idempotency guarantee. After an interrupted submission, inspect status before resubmitting. The API rejects a concurrent workflow; once preparation has applied, the stored request ID prevents that preparation from being repeated while it remains the latest request. Retain identical input when retrying that operation. status reads once; poll it with the same request ID: The response includes the current question and progress. The API retains only the latest applied request ID and latest rejection, so older or unknown IDs can remain pending. Failures before preparation may also remain pending; inspect question.status and question.error rather than polling indefinitely. Do not use questions watch to confirm a new reclustering request: before preparation, it can still see the previous completed state.

Stop reclustering

This targets the question’s current reclustering workflow. The CLI returns status: "stop_requested" to confirm the API accepted the stop request. It does not confirm that cancellation was enqueued or completed, or that work is still active. Stopping an already-finished workflow is a successful no-op; the API returns the same response for both cases. If work is active, it must drain before another workflow can start. Cancellation does not roll back changes already applied. Inspect the original request with recluster status before retrying.

Agent output and exit codes

Results include scope and dashboardUrl. When another step is available, nextAction names the command and nextCommand supplies an argv array including the question ID and scope. A positions command contains a <positions.json> placeholder that the agent must replace with its authored file path.
  • 0: The operation succeeded. watch reaching awaiting_positions is successful because it needs agent input next.
  • 1: Validation, authentication, API, or filesystem error; creation saved but did not start; recluster status reports rejection or failure; or watch encountered failed analysis, cancellation, or timeout.
For creation failures after saving and watch failures after a status response, read the JSON result even when the process exits nonzero. get and dashboard are reads: inspect their state rather than interpreting exit code 0 as analysis completion.