`POST /api/intelligence/agent-experiments` — create an experiment. Admin-only. Variant names must be unique within the experiment (case-insensitively) — including against each other inside this one payload; a collision is a `409`.

POST

/api/intelligence/agent-experiments

const url = 'https://example.com/api/intelligence/agent-experiments';
const options = {
  method: 'POST',
  headers: {'Content-Type': 'application/json'},
  body: '{"dataset":"example","evaluators":"example","name":"example","project_id":"2489E9AD-2EE2-8E00-8EC9-32D5F69181C0","purpose":"example","status":"example","stop_criteria":"example","variants":[{"budget_policy":"example","config":"example","endpoint":"example","harness":"example","is_control":true,"model":"example","name":"example","prompt_policy":"example","provider":"example","runner":"example","tool_profile":"example"}]}'
};

try {
  const response = await fetch(url, options);
  const data = await response.json();
  console.log(data);
} catch (error) {
  console.error(error);
}

curl --request POST \
  --url https://example.com/api/intelligence/agent-experiments \
  --header 'Content-Type: application/json' \
  --data '{ "dataset": "example", "evaluators": "example", "name": "example", "project_id": "2489E9AD-2EE2-8E00-8EC9-32D5F69181C0", "purpose": "example", "status": "example", "stop_criteria": "example", "variants": [ { "budget_policy": "example", "config": "example", "endpoint": "example", "harness": "example", "is_control": true, "model": "example", "name": "example", "prompt_policy": "example", "provider": "example", "runner": "example", "tool_profile": "example" } ] }'

Request Body^required

application/json

object

dataset

evaluators

name

required

string

project_id

string | null format: uuid

purpose

string | null

status

string | null

stop_criteria

variants

required

Array<object>

object

budget_policy

config

endpoint

string | null

harness

string | null

is_control

boolean | null

model

required

string

name

required

string

prompt_policy

provider

required

string

runner

required

string

tool_profile

string | null

Example^generated

{
  "dataset": "example",
  "evaluators": "example",
  "name": "example",
  "project_id": "2489E9AD-2EE2-8E00-8EC9-32D5F69181C0",
  "purpose": "example",
  "status": "example",
  "stop_criteria": "example",
  "variants": [
    {
      "budget_policy": "example",
      "config": "example",
      "endpoint": "example",
      "harness": "example",
      "is_control": true,
      "model": "example",
      "name": "example",
      "prompt_policy": "example",
      "provider": "example",
      "runner": "example",
      "tool_profile": "example"
    }
  ]
}

Responses

200

application/json

object

attempts

required

#838 E1 — the experiment’s live attempts, queued ones included.

Array<object>

One materialised (variant, dataset case) attempt that has not reached a terminal state (#838 E1). The board needs this because the recorded-run telemetry (AgentExperimentRun) is written from a task OUTCOME — an attempt that is queued, pending or still running appears in it nowhere.

object

dataset_case_id

required

string

run_id

string | null format: uuid

status

required

The RUN’s status: queued | pending | starting | running | waiting. Read from the run, not the task mirror — the mirror maps a queued run onto pending (issue #110), and that is exactly the distinction shown here.

string

task_class

required

string

task_id

required

string format: uuid

task_label

required

string

variant_id

string | null format: uuid

experiment

required

object

completed_at

string | null format: date-time

created_at

required

string format: date-time

created_by

string | null format: uuid

dataset

required

evaluators

required

string format: uuid

name

required

string

project_id

string | null format: uuid

purpose

required

string

recommendation

string | null

recommendation_confidence

number | null format: double

status

required

string

stop_criteria

required

updated_at

required

string format: date-time

workspace_id

required

string format: uuid

harness_comparison

required

Array<object>

object

accepted_runs

required

integer

cost_per_accepted_patch

number | null format: double

cost_usd

required

number format: double

cost_waste_usd

required

number format: double

harness

string | null

key

required

string

label

required

string

minutes_per_accepted_patch

number | null format: double

model

string | null

provider

string | null

regression_rate

required

number format: double

retry_rate

required

number format: double

review_acceptance_rate

number | null format: double

runner

string | null

sample_size

required

integer

scope_violation_rate

required

number format: double

score

required

number format: double

success_rate

required

number format: double

test_pass_rate

number | null format: double

timeout_rate

required

number format: double

tokens_per_accepted_patch

number | null format: double

tool_error_rate

required

number format: double

leaderboard

required

Array<object>

object

accepted_runs

required

integer

cost_per_accepted_patch

number | null format: double

cost_usd

required

number format: double

cost_waste_usd

required

number format: double

harness

string | null

key

required

string

label

required

string

minutes_per_accepted_patch

number | null format: double

model

string | null

provider

string | null

regression_rate

required

number format: double

retry_rate

required

number format: double

review_acceptance_rate

number | null format: double

runner

string | null

sample_size

required

integer

scope_violation_rate

required

number format: double

score

required

number format: double

success_rate

required

number format: double

test_pass_rate

number | null format: double

timeout_rate

required

number format: double

tokens_per_accepted_patch

number | null format: double

tool_error_rate

required

number format: double

model_comparison

required

Array<object>

object

accepted_runs

required

integer

cost_per_accepted_patch

number | null format: double

cost_usd

required

number format: double

cost_waste_usd

required

number format: double

harness

string | null

key

required

string

label

required

string

minutes_per_accepted_patch

number | null format: double

model

string | null

provider

string | null

regression_rate

required

number format: double

retry_rate

required

number format: double

review_acceptance_rate

number | null format: double

runner

string | null

sample_size

required

integer

scope_violation_rate

required

number format: double

score

required

number format: double

success_rate

required

number format: double

test_pass_rate

number | null format: double

timeout_rate

required

number format: double

tokens_per_accepted_patch

number | null format: double

tool_error_rate

required

number format: double

recommendations

required

Array<object>

object

basis

required

string

confidence

required

number format: double

recommendation

required

string

sample_size

required

integer

scope

required

string

runs

required

Array<object>

object

accepted

required

boolean

active_ms

integer | null format: int64

artifacts

required

bridge_events

required

cache_read_tokens

integer | null format: int64

cache_write_tokens

integer | null format: int64

cost_usd

number | null format: double

created_at

required

string format: date-time

dataset_case_id

string | null

experiment_id

required

string format: uuid

finished_at

string | null format: date-time

human_rating

number | null format: double

required

string format: uuid

input_tokens

integer | null format: int64

lint_passed

boolean | null

output_tokens

integer | null format: int64

reasoning_tokens

integer | null format: int64

regression

required

boolean

result

required

retry_count

required

integer format: int32

review_accepted

boolean | null

run_id

string | null format: uuid

scope_violation

required

boolean

started_at

string | null format: date-time

status

required

string

task_class

required

string

task_id

string | null format: uuid

task_label

required

string

tests_passed

boolean | null

timeout

required

boolean

tool_error_count

required

integer format: int32

turns

integer | null format: int32

variant_id

required

string format: uuid

wall_ms

integer | null format: int64

workspace_id

required

string format: uuid

task_class_matrix

required

Array<object>

object

task_class

required

string

variants

required

Array<object>

object

accepted_runs

required

integer

cost_per_accepted_patch

number | null format: double

cost_usd

required

number format: double

cost_waste_usd

required

number format: double

harness

string | null

key

required

string

label

required

string

minutes_per_accepted_patch

number | null format: double

model

string | null

provider

string | null

regression_rate

required

number format: double

retry_rate

required

number format: double

review_acceptance_rate

number | null format: double

runner

string | null

sample_size

required

integer

scope_violation_rate

required

number format: double

score

required

number format: double

success_rate

required

number format: double

test_pass_rate

number | null format: double

timeout_rate

required

number format: double

tokens_per_accepted_patch

number | null format: double

tool_error_rate

required

number format: double

winner_key

string | null

winner_label

string | null

variants

required

Array<object>

object

budget_policy

required

config

required

created_at

required

string format: date-time

endpoint

string | null

experiment_id

required

string format: uuid

harness

required

string

required

string format: uuid

is_control

required

boolean

model

required

string

name

required

string

prompt_policy

required

provider

required

string

runner

required

string

tool_profile

required

string

workspace_id

required

string format: uuid

Example^generated

{
  "attempts": [
    {
      "dataset_case_id": "example",
      "run_id": "2489E9AD-2EE2-8E00-8EC9-32D5F69181C0",
      "status": "example",
      "task_class": "example",
      "task_id": "2489E9AD-2EE2-8E00-8EC9-32D5F69181C0",
      "task_label": "example",
      "variant_id": "2489E9AD-2EE2-8E00-8EC9-32D5F69181C0"
    }
  ],
  "experiment": {
    "completed_at": "2026-04-15T12:00:00Z",
    "created_at": "2026-04-15T12:00:00Z",
    "created_by": "2489E9AD-2EE2-8E00-8EC9-32D5F69181C0",
    "dataset": "example",
    "evaluators": "example",
    "id": "2489E9AD-2EE2-8E00-8EC9-32D5F69181C0",
    "name": "example",
    "project_id": "2489E9AD-2EE2-8E00-8EC9-32D5F69181C0",
    "purpose": "example",
    "recommendation": "example",
    "recommendation_confidence": 1,
    "status": "example",
    "stop_criteria": "example",
    "updated_at": "2026-04-15T12:00:00Z",
    "workspace_id": "2489E9AD-2EE2-8E00-8EC9-32D5F69181C0"
  },
  "harness_comparison": [
    {
      "accepted_runs": 1,
      "cost_per_accepted_patch": 1,
      "cost_usd": 1,
      "cost_waste_usd": 1,
      "harness": "example",
      "key": "example",
      "label": "example",
      "minutes_per_accepted_patch": 1,
      "model": "example",
      "provider": "example",
      "regression_rate": 1,
      "retry_rate": 1,
      "review_acceptance_rate": 1,
      "runner": "example",
      "sample_size": 1,
      "scope_violation_rate": 1,
      "score": 1,
      "success_rate": 1,
      "test_pass_rate": 1,
      "timeout_rate": 1,
      "tokens_per_accepted_patch": 1,
      "tool_error_rate": 1
    }
  ],
  "leaderboard": [
    {
      "accepted_runs": 1,
      "cost_per_accepted_patch": 1,
      "cost_usd": 1,
      "cost_waste_usd": 1,
      "harness": "example",
      "key": "example",
      "label": "example",
      "minutes_per_accepted_patch": 1,
      "model": "example",
      "provider": "example",
      "regression_rate": 1,
      "retry_rate": 1,
      "review_acceptance_rate": 1,
      "runner": "example",
      "sample_size": 1,
      "scope_violation_rate": 1,
      "score": 1,
      "success_rate": 1,
      "test_pass_rate": 1,
      "timeout_rate": 1,
      "tokens_per_accepted_patch": 1,
      "tool_error_rate": 1
    }
  ],
  "model_comparison": [
    {
      "accepted_runs": 1,
      "cost_per_accepted_patch": 1,
      "cost_usd": 1,
      "cost_waste_usd": 1,
      "harness": "example",
      "key": "example",
      "label": "example",
      "minutes_per_accepted_patch": 1,
      "model": "example",
      "provider": "example",
      "regression_rate": 1,
      "retry_rate": 1,
      "review_acceptance_rate": 1,
      "runner": "example",
      "sample_size": 1,
      "scope_violation_rate": 1,
      "score": 1,
      "success_rate": 1,
      "test_pass_rate": 1,
      "timeout_rate": 1,
      "tokens_per_accepted_patch": 1,
      "tool_error_rate": 1
    }
  ],
  "recommendations": [
    {
      "basis": "example",
      "confidence": 1,
      "recommendation": "example",
      "sample_size": 1,
      "scope": "example"
    }
  ],
  "runs": [
    {
      "accepted": true,
      "active_ms": 1,
      "artifacts": "example",
      "bridge_events": "example",
      "cache_read_tokens": 1,
      "cache_write_tokens": 1,
      "cost_usd": 1,
      "created_at": "2026-04-15T12:00:00Z",
      "dataset_case_id": "example",
      "experiment_id": "2489E9AD-2EE2-8E00-8EC9-32D5F69181C0",
      "finished_at": "2026-04-15T12:00:00Z",
      "human_rating": 1,
      "id": "2489E9AD-2EE2-8E00-8EC9-32D5F69181C0",
      "input_tokens": 1,
      "lint_passed": true,
      "output_tokens": 1,
      "reasoning_tokens": 1,
      "regression": true,
      "result": "example",
      "retry_count": 1,
      "review_accepted": true,
      "run_id": "2489E9AD-2EE2-8E00-8EC9-32D5F69181C0",
      "scope_violation": true,
      "started_at": "2026-04-15T12:00:00Z",
      "status": "example",
      "task_class": "example",
      "task_id": "2489E9AD-2EE2-8E00-8EC9-32D5F69181C0",
      "task_label": "example",
      "tests_passed": true,
      "timeout": true,
      "tool_error_count": 1,
      "turns": 1,
      "variant_id": "2489E9AD-2EE2-8E00-8EC9-32D5F69181C0",
      "wall_ms": 1,
      "workspace_id": "2489E9AD-2EE2-8E00-8EC9-32D5F69181C0"
    }
  ],
  "task_class_matrix": [
    {
      "task_class": "example",
      "variants": [
        {
          "accepted_runs": 1,
          "cost_per_accepted_patch": 1,
          "cost_usd": 1,
          "cost_waste_usd": 1,
          "harness": "example",
          "key": "example",
          "label": "example",
          "minutes_per_accepted_patch": 1,
          "model": "example",
          "provider": "example",
          "regression_rate": 1,
          "retry_rate": 1,
          "review_acceptance_rate": 1,
          "runner": "example",
          "sample_size": 1,
          "scope_violation_rate": 1,
          "score": 1,
          "success_rate": 1,
          "test_pass_rate": 1,
          "timeout_rate": 1,
          "tokens_per_accepted_patch": 1,
          "tool_error_rate": 1
        }
      ],
      "winner_key": "example",
      "winner_label": "example"
    }
  ],
  "variants": [
    {
      "budget_policy": "example",
      "config": "example",
      "created_at": "2026-04-15T12:00:00Z",
      "endpoint": "example",
      "experiment_id": "2489E9AD-2EE2-8E00-8EC9-32D5F69181C0",
      "harness": "example",
      "id": "2489E9AD-2EE2-8E00-8EC9-32D5F69181C0",
      "is_control": true,
      "model": "example",
      "name": "example",
      "prompt_policy": "example",
      "provider": "example",
      "runner": "example",
      "tool_profile": "example",
      "workspace_id": "2489E9AD-2EE2-8E00-8EC9-32D5F69181C0"
    }
  ]
}

409

Two variants share a name

application/json

The canonical JSON body of every error response — the single source of truth the frontend binds to. Every AppError serializes as this exact shape, and the generated OpenAPI component ApiErrorBody (with its ErrorCode enum) is what the frontend error schema is generated from, so there is no hand-written error schema on either end.

object

code

required

Machine-readable, stable error code.

string

Allowed values: not_found unauthorized forbidden license_required license_expired bad_request unprocessable precondition_failed conflict method_not_allowed rate_limited too_many_requests quota_exceeded database_error docker_error vault_error internal_error

details

One of:

null
object

null

Present only on a quota-exceeded 403 — the inline upgrade-CTA payload.

object

key

required

The entitlement feature key that was hit, e.g. apps.max_count.

string

limit

required

The plan’s limit for this key.

integer format: int64

upgrade_url

required

Where to send the user to upgrade.

string

used

required

Current usage (count or bytes, per the key).

integer format: int64

error

required

Human-readable message (the server’s English text; the client may localize by code).

string

Example

{
  "code": "not_found"
}

`POST /api/intelligence/agent-experiments` — create an experiment. Admin-only. Variant names must be unique within the experiment (case-insensitively) — including against each other inside this one payload; a collision is a `409`.

Request Bodyrequired

Examplegenerated

Responses

200

Examplegenerated

409

Example

Request Body^required

Example^generated

Example^generated