A/B tests reference

The account's A/B tests, as the portal's A/B tests tab and an A/B test's page show them: one benchmark on two sides, each run with its scores, and a run's comparison. Reading spends no allowance. Starting and stopping a test through the API follow in a later release.

Every route below follows the conventions in Work with the BI Pixie API. You can also download the OpenAPI document.

List the account's A/B tests #

GET /v1/aiReadiness/abTests

The A/B tests tab, one row per test: its two sides, which questions it asks and where they came from, how many times it has run, its latest run with that run's scores, and the gap of every succeeded run, oldest first, which is what the consistency line states. Newest run first unless sort says otherwise. q searches the name, both semantic models and both data agents; workspaceId keeps the tests with either side in that workspace. Reading the tab spends nothing and asks Fabric nothing.

Scope: BIPixie.Read.All
Plan: Every plan
Counts against: Calls that read, 300 a minute

Parameters

Parameters for List the account's A/B tests
Name	In	Type	Required	Description
`q`	query	string	No	Keeps the tests whose name, semantic models or data agents contain this text.
`workspaceId`	query	string (uuid)	No	Keeps the tests with either side in this workspace.
`sort`	query	string	No	`latest` (the default, by the newest run's finish), `earliest`, `name` or `nameDesc`. One of `latest`, `earliest`, `name`, `nameDesc`.
`maxResults`	query	integer	No	The largest number of rows to return in one page.
`continuationToken`	query	string	No	The token the previous page answered with.

Responses

200. The tests.

Headers of the 200 response to List the account's A/B tests
Header	Description
`x-ms-request-id`	The id of this request. Quote it to support.

The 200 response to List the account's A/B tests, AbTestList
Field	Type	Always present	Description
`value`	array of AbTest	Yes
`abTestId`	string	Yes
`name`	string	Yes
`createdAtUtc`	string or null (date-time)	No
`sideA`	AbSide	Yes	One side of a test as it was set up.
`semanticModelId`	string or null	Yes
`workspaceId`	string or null	Yes
`semanticModelName`	string or null	No
`workspaceName`	string or null	No
`label`	string or null	No	The customer's short name for the side, when one was written.
`description`	string or null	No	The customer's own words about the side, when any were written.
`answeredBy`	AbAnsweredBy	Yes	What answers for a side, in the words a benchmark run is started with: the kind, the data agent when it is one and the workspace it lives in, and the saved AI provider when the side asks a generic AI agent.
`kind`	string or null	Yes	`DataAgent`, `BIPixieTestAgent`, `PowerBICopilot` or `GenericAgent`, as `agent.kind` on a benchmark run.
`dataAgentId`	string or null	No	The data agent that answers the side. Null for a `PowerBICopilot` or `GenericAgent` side, as are the other data agent fields.
`dataAgentName`	string or null	No
`dataAgentWorkspaceId`	string or null	No
`dataAgentWorkspaceName`	string or null	No	Where the data agent lives, when the test recorded it. A run's result reads it from the test, so it names the same workspace the test does.
`providerId`	string or null	No	The saved AI provider a generic AI agent side asks. On a run's result it is the one that answered the side, as its stored result recorded it.
`providerName`	string or null	No	The saved AI provider's name, when one was recorded.
`sideB`	AbSide	Yes	One side of a test as it was set up.
`semanticModelId`	string or null	Yes
`workspaceId`	string or null	Yes
`semanticModelName`	string or null	No
`workspaceName`	string or null	No
`label`	string or null	No	The customer's short name for the side, when one was written.
`description`	string or null	No	The customer's own words about the side, when any were written.
`answeredBy`	AbAnsweredBy	Yes	What answers for a side, in the words a benchmark run is started with: the kind, the data agent when it is one and the workspace it lives in, and the saved AI provider when the side asks a generic AI agent.
`kind`	string or null	Yes	`DataAgent`, `BIPixieTestAgent`, `PowerBICopilot` or `GenericAgent`, as `agent.kind` on a benchmark run.
`dataAgentId`	string or null	No	The data agent that answers the side. Null for a `PowerBICopilot` or `GenericAgent` side, as are the other data agent fields.
`dataAgentName`	string or null	No
`dataAgentWorkspaceId`	string or null	No
`dataAgentWorkspaceName`	string or null	No	Where the data agent lives, when the test recorded it. A run's result reads it from the test, so it names the same workspace the test does.
`providerId`	string or null	No	The saved AI provider a generic AI agent side asks. On a run's result it is the one that answered the side, as its stored result recorded it.
`providerName`	string or null	No	The saved AI provider's name, when one was recorded.
`questions`	object	Yes	Which questions the test asks, and where they came from.
`count`	integer	Yes
`questionSetHash`	string or null	No	The hash of the question set the test froze, which a run and the saved questions carry too.
`source`	object	Yes
`kind`	string	Yes	`Run` for a test made from a stored benchmark run, `Questions` for one made from the saved questions. One of `Run`, `Questions`.
`runId`	string or null	No	The stored benchmark run side A replays, when the test was made from one and that run still exists.
`runStoredAtUtc`	string or null (date-time)	No
`correctAnswers`	string	Yes	`Preserve` keeps the correct answers the test was made with on every run; `Update` computes them again from side A on each run. One of `Preserve`, `Update`.
`repeats`	integer	No	How many times each question is asked on each side.
`runCount`	integer	Yes
`latestRun`	one of AbLatestRun, null	No
`latestSucceededRunId`	string or null	No
`latestSucceededRunAggregate`	one of AbAggregate, null	No	The scores and response times of the newest run that succeeded, the run `latestSucceededRunId` names. Null when no run has succeeded.
`runGaps`	array of integer	No	Each succeeded run's gap, side B's correct answers minus side A's, oldest first, which is what the consistency line states.
`description`	string or null	No	The customer's own words about the test, when any were written.
`hypothesis`	object or null	No	What the customer expected, when they wrote it.
`text`	string	No
`kind`	string or null	No	`A`, `B` or `Same` when the hypothesis is one of the suggested sentences; null for the customer's own words.
`continuationToken`	string	No
`continuationUri`	string (uri)	No

Errors

Every refusal is an ErrorResponse with a stable errorCode. See Errors for what each HTTP status means and every code the API can return.

This is an example of a refusal. Every refusal has the same fields.

Example refusal

HTTP/1.1 404 Not Found
Content-Type: application/json
x-ms-request-id: cbf21b6d-b661-4b52-88cb-d79712e504fc

{
  "errorCode": "ItemNotFound",
  "message": "No item with this id is visible to the signed-in person in this workspace, or it is not of the type the route takes.",
  "isRetriable": false,
  "moreDetails": [],
  "relatedResource": {
    "resourceId": "d972b3f5-a2fa-47ec-94ff-0403a4d159fa",
    "resourceType": "SemanticModel"
  },
  "parameters": [
    {
      "name": "itemId",
      "value": "d972b3f5-a2fa-47ec-94ff-0403a4d159fa"
    }
  ],
  "requestId": "cbf21b6d-b661-4b52-88cb-d79712e504fc"
}

Example

Request

curl -i -X GET \
  -H "Authorization: Bearer $TOKEN" \
  "https://api.bipixie.com/v1/aiReadiness/abTests"

An example of what the call answers. The values are fictional.

Response, 200

HTTP/1.1 200 OK
Content-Type: application/json
x-ms-request-id: cbf21b6d-b661-4b52-88cb-d79712e504fc

{
  "value": [
    {
      "abTestId": "7c016a91-7eea-4f0c-b327-e32f68230272",
      "name": "Contoso Sales",
      "createdAtUtc": "2026-06-03T14:12:00Z",
      "sideA": {
        "semanticModelId": "f1fdc461-3ca1-4702-ae48-afc3522409fc",
        "workspaceId": "e17c1b59-7a2e-491e-acdd-11434854bbb0",
        "semanticModelName": "Contoso Sales",
        "workspaceName": "Contoso Sales Analytics",
        "label": "With AI instructions",
        "description": "Sales questions answered with and without the AI instructions.",
        "answeredBy": {
          "kind": "DataAgent",
          "dataAgentId": "05dd87d1-624f-4ffa-ab88-ee231a75a904",
          "dataAgentName": "Contoso Sales Agent",
          "dataAgentWorkspaceId": "3a287f9c-63af-48a1-8c06-61c2e8095e5f",
          "dataAgentWorkspaceName": "Contoso Sales Analytics",
          "providerId": "b2df6be3-7769-4222-90ea-9fd98f4592e0",
          "providerName": "Azure OpenAI"
        }
      },
      "sideB": {
        "semanticModelId": "f1fdc461-3ca1-4702-ae48-afc3522409fc",
        "workspaceId": "e17c1b59-7a2e-491e-acdd-11434854bbb0",
        "semanticModelName": "Contoso Sales",
        "workspaceName": "Contoso Sales Analytics",
        "label": "With AI instructions",
        "description": "Sales questions answered with and without the AI instructions.",
        "answeredBy": {
          "kind": "DataAgent",
          "dataAgentId": "05dd87d1-624f-4ffa-ab88-ee231a75a904",
          "dataAgentName": "Contoso Sales Agent",
          "dataAgentWorkspaceId": "3a287f9c-63af-48a1-8c06-61c2e8095e5f",
          "dataAgentWorkspaceName": "Contoso Sales Analytics",
          "providerId": "b2df6be3-7769-4222-90ea-9fd98f4592e0",
          "providerName": "Azure OpenAI"
        }
      },
      "questions": {
        "count": 12,
        "questionSetHash": "12056ac56f6dd1de",
        "source": {
          "kind": "Run",
          "runId": "f2e14a95-8025-41de-a48f-860f8cfd2e68",
          "runStoredAtUtc": "2026-09-14T09:42:17Z"
        },
        "correctAnswers": "Preserve"
      },
      "repeats": 3,
      "runCount": 12,
      "latestRun": null,
      "latestSucceededRunId": "404dca33-3888-443a-8788-5329a0546348",
      "latestSucceededRunAggregate": null,
      "runGaps": [
        3
      ],
      "description": "Sales questions answered with and without the AI instructions.",
      "hypothesis": {
        "text": "The side with AI instructions answers more questions correctly.",
        "kind": "A"
      }
    }
  ],
  "continuationToken": "eyJza2lwIjoxMDB9",
  "continuationUri": "https://api.bipixie.com/v1/aiReadiness/abTests?continuationToken=eyJza2lwIjoxMDB9"
}

Create an A/B test and start its first run #

POST /v1/aiReadiness/abTests

Creates an A/B test from its two sides and starts its first run, as the BI Pixie portal's New A/B test does, as one operation of type RunAbTest. Each side names a semantic model and what answers for it, in the words a benchmark run is started with. The questions are side A's, and never ride the request: source names a stored benchmark run of side A's semantic model ({ "kind": "Run", "runId" }), whose questions and correct answers side A replays, asked as many times as that run asked them unless repeats says otherwise; or its saved benchmark questions ({ "kind": "Questions" }), which side A asks for the first time, the ones that can run, as runBenchmark asks them. A semantic model with no saved benchmark questions that can run answers 400 BenchmarkQuestionsMissing, and a run it does not hold 404 RunNotFound, before anything is counted.

The run counts toward the account's benchmark runs, one for each side that asks, or the A/B run the plan includes; reuseSideA keeps the stored run's answers as side A and asks side B alone. Both semantic models and both data agents are looked up as the signed-in person before anything is counted, so an id that person cannot see answers 404 and spends nothing. A GenericAgent side names the saved AI provider it asks in providerId.

Side B starts when the person who started the operation reads it, once side A has finished: BI Pixie keeps nothing of the caller's that can read a dataset, so side B runs with the token that read carries. Reading the operation as another person reports where the run is and starts nothing. An operation nobody reads after side A finishes is failed with SideBNotStarted an hour before the operation expires, and the test can then be run again. The operation's result is the run's comparison, exactly as getAbRunResult answers it. cancelOperation stops the run, as the portal's Stop does.

Scope: BIPixie.ReadWrite.All
Plan: Every plan
Counts against: Calls that start work, 30 a minute

Request body

JSON, CreateAbTestRequest. The body is required.

Request body for Create an A/B test and start its first run
Field	Type	Required	Description
`name`	string	Yes
`sideA`	CreateAbTestSide	Yes	One side of a test to create.
`semanticModelId`	string (uuid)	Yes
`workspaceId`	string (uuid)	Yes
`answeredBy`	object	Yes	What answers for this side, as `agent` on a benchmark run.
`kind`	string	Yes	One of `DataAgent`, `BIPixieTestAgent`, `PowerBICopilot`, `GenericAgent`.
`dataAgentId`	string (uuid)	No	The data agent to ask. Required for `DataAgent` and `BIPixieTestAgent`.
`workspaceId`	string (uuid)	No	The data agent's workspace, when it is not the semantic model's. Defaults to the semantic model's workspace.
`providerId`	string	No	The id of the saved AI provider to answer with. Required for `GenericAgent`.
`label`	string	No	Your short name for the side.
`description`	string	No	Your own words about the side.
`sideB`	CreateAbTestSide	Yes	One side of a test to create.
`semanticModelId`	string (uuid)	Yes
`workspaceId`	string (uuid)	Yes
`answeredBy`	object	Yes	What answers for this side, as `agent` on a benchmark run.
`kind`	string	Yes	One of `DataAgent`, `BIPixieTestAgent`, `PowerBICopilot`, `GenericAgent`.
`dataAgentId`	string (uuid)	No	The data agent to ask. Required for `DataAgent` and `BIPixieTestAgent`.
`workspaceId`	string (uuid)	No	The data agent's workspace, when it is not the semantic model's. Defaults to the semantic model's workspace.
`providerId`	string	No	The id of the saved AI provider to answer with. Required for `GenericAgent`.
`label`	string	No	Your short name for the side.
`description`	string	No	Your own words about the side.
`source`	object	Yes	Where the questions come from. A test answers it back under `questions.source`.
`kind`	string	Yes	`Run` replays a stored benchmark run of side A's semantic model, named in `runId`; `Questions` asks its saved benchmark questions for the first time. One of `Run`, `Questions`.
`runId`	string (uuid)	No	The stored benchmark run to replay. Required with `Run`, and not given with `Questions`.
`repeats`	integer	No	How many times each question is asked on each side. Defaults to what the stored run or the saved benchmark questions were asked with.
`correctAnswers`	string	No	`Preserve`, the default, keeps the correct answers the test is made with on every run; `Update` computes them again from side A on each run. A test made from a stored benchmark run replays that run's answers, so `Update` is taken only with a `Questions` source. One of `Preserve`, `Update`.
`reuseSideA`	boolean	No	Keeps the stored run's answers as side A and asks side B alone, so the run counts one side. Only with a `Run` source.
`description`	string	No	Your own words about the test.
`hypothesis`	object	No	What you expect.
`text`	string	Yes
`kind`	string	No	Which side you expect to answer better, when the expectation is one of the suggested sentences. One of `A`, `B`, `Same`.

Responses

202. The test was created and its first run started. The body carries the operation's id and address.

Headers of the 202 response to Create an A/B test and start its first run
Header	Description
`x-ms-request-id`	The id of this request. Quote it to support.
`x-ms-operation-id`	The id of the operation.
`Location`	The address of the operation's state.
`Retry-After`	Seconds to wait before asking for the operation's state.

To follow the work this call started, see Follow a long-running operation.

The 202 response to Create an A/B test and start its first run, Accepted
Field	Type	Always present	Description
`operationId`	string (uuid)	Yes	The operation's id, which the operation's state and result take.
`location`	string (uri)	Yes	Where to ask about the operation.

400. The request is not well formed, and nothing was started, changed or counted. errorCode is InvalidRequest, or InvalidPreview for a preview that is not available. A body member the operation does not document is refused this way, and the message names it and the documented member closest to it.

Headers of the 400 response to Create an A/B test and start its first run
Header	Description
`x-ms-request-id`	The id of this request. Quote it to support.

The 400 response to Create an A/B test and start its first run, ErrorResponse
Field	Type	Always present	Description
`errorCode`	string	Yes	Each code keeps its meaning once published. New codes can be added over time, and a code is never reused. One of `InvalidRequest`, `InvalidPreview`, `PayloadTooLarge`, `BenchmarkQuestionsMissing`, `OperationNotSupportedForItem`, `Unauthorized`, `TokenExpired`, `InsufficientScopes`, `AppOnlyTokenNotSupported`, `UserNotRegistered`, `AccountNotActive`, `AdminRoleRequired`, `FeatureNotInPlan`, `FeatureTurnedOff`, `LicenseNotYours`, `TrackedItemLimitReached`, `LicenseTrackedItemLimitReached`, `FabricConsentRequired`, `FabricAccessDenied`, `WorkspaceNotFound`, `ItemNotFound`, `AssessmentNotFound`, `RunNotFound`, `AbTestNotFound`, `OperationNotFound`, `ScheduleNotFound`, `AiProviderNotFound`, `AiCredentialNotFound`, `OperationInProgress`, `OperationHasNoResult`, `ResultNotReady`, `ScheduleConflict`, `AiProviderConflict`, `ItemChanged`, `CapacityPaused`, `OperationExpired`, `RegionNotReady`, `CapacityRequired`, `CapacityUnpaid`, `StoreNotConnected`, `RequestBlocked`, `AssessmentDailyLimitReached`, `BenchmarkRunsExhausted`, `OptimizationApplyDailyLimitReached`, `FirstLookLimitReached`, `BenchmarkBuildLimitReached`, `FabricUnavailable`, `StoreUnavailable`, `ThrottleStoreUnavailable`, `UnknownError`.
`message`	string	Yes
`isRetriable`	boolean	No
`moreDetails`	array of object	No
`relatedResource`	object or null	No
`resourceId`	string	No
`resourceType`	string	No
`parameters`	array of object	No
`name`	string	Yes
`value`	string	Yes
`requestId`	string (uuid)	Yes

Errors

Every refusal is an ErrorResponse with a stable errorCode. See Errors for what each HTTP status means and every code the API can return.

This is an example of a refusal. Every refusal has the same fields.

Example refusal

HTTP/1.1 404 Not Found
Content-Type: application/json
x-ms-request-id: cbf21b6d-b661-4b52-88cb-d79712e504fc

{
  "errorCode": "ItemNotFound",
  "message": "No item with this id is visible to the signed-in person in this workspace, or it is not of the type the route takes.",
  "isRetriable": false,
  "moreDetails": [],
  "relatedResource": {
    "resourceId": "d972b3f5-a2fa-47ec-94ff-0403a4d159fa",
    "resourceType": "SemanticModel"
  },
  "parameters": [
    {
      "name": "itemId",
      "value": "d972b3f5-a2fa-47ec-94ff-0403a4d159fa"
    }
  ],
  "requestId": "cbf21b6d-b661-4b52-88cb-d79712e504fc"
}

Example

Request

curl -i -X POST \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
  "name": "<string>",
  "sideA": {
    "semanticModelId": "<uuid>",
    "workspaceId": "<uuid>",
    "answeredBy": {
      "kind": "DataAgent"
    }
  },
  "sideB": {
    "semanticModelId": "<uuid>",
    "workspaceId": "<uuid>",
    "answeredBy": {
      "kind": "DataAgent"
    }
  },
  "source": {
    "kind": "Run"
  }
}' \
  "https://api.bipixie.com/v1/aiReadiness/abTests"

An example of what the call answers. The values are fictional.

Response, 202

HTTP/1.1 202 Accepted
Content-Type: application/json
x-ms-request-id: cbf21b6d-b661-4b52-88cb-d79712e504fc
x-ms-operation-id: cd57b041-e44a-4dda-a4ff-02935ffb8c04
Location: https://api.bipixie.com/v1/operations/cd57b041-e44a-4dda-a4ff-02935ffb8c04
Retry-After: 5

{
  "operationId": "cd57b041-e44a-4dda-a4ff-02935ffb8c04",
  "location": "https://api.bipixie.com/v1/aiReadiness/abTests"
}

Get one A/B test with its runs #

GET /v1/aiReadiness/abTests/{abTestId}

One A/B test as its page opens it, with its runs under runs, newest first, at most fifty. An id that names no test of this account, or a test since deleted, answers 404 AbTestNotFound.

Scope: BIPixie.Read.All
Plan: Every plan
Counts against: Calls that read, 300 a minute

Parameters

Parameters for Get one A/B test with its runs
Name	In	Type	Required	Description
`abTestId`	path	string (uuid)	Yes	The A/B test's id, as the A/B tests list gives it.

Responses

200. The test and its runs.

Headers of the 200 response to Get one A/B test with its runs
Header	Description
`x-ms-request-id`	The id of this request. Quote it to support.

The 200 response to Get one A/B test with its runs, AbTestWithRuns
Field	Type	Always present	Description
`abTestId`	string	Yes
`name`	string	Yes
`createdAtUtc`	string or null (date-time)	No
`sideA`	AbSide	Yes	One side of a test as it was set up.
`semanticModelId`	string or null	Yes
`workspaceId`	string or null	Yes
`semanticModelName`	string or null	No
`workspaceName`	string or null	No
`label`	string or null	No	The customer's short name for the side, when one was written.
`description`	string or null	No	The customer's own words about the side, when any were written.
`answeredBy`	AbAnsweredBy	Yes	What answers for a side, in the words a benchmark run is started with: the kind, the data agent when it is one and the workspace it lives in, and the saved AI provider when the side asks a generic AI agent.
`kind`	string or null	Yes	`DataAgent`, `BIPixieTestAgent`, `PowerBICopilot` or `GenericAgent`, as `agent.kind` on a benchmark run.
`dataAgentId`	string or null	No	The data agent that answers the side. Null for a `PowerBICopilot` or `GenericAgent` side, as are the other data agent fields.
`dataAgentName`	string or null	No
`dataAgentWorkspaceId`	string or null	No
`dataAgentWorkspaceName`	string or null	No	Where the data agent lives, when the test recorded it. A run's result reads it from the test, so it names the same workspace the test does.
`providerId`	string or null	No	The saved AI provider a generic AI agent side asks. On a run's result it is the one that answered the side, as its stored result recorded it.
`providerName`	string or null	No	The saved AI provider's name, when one was recorded.
`sideB`	AbSide	Yes	One side of a test as it was set up.
`semanticModelId`	string or null	Yes
`workspaceId`	string or null	Yes
`semanticModelName`	string or null	No
`workspaceName`	string or null	No
`label`	string or null	No	The customer's short name for the side, when one was written.
`description`	string or null	No	The customer's own words about the side, when any were written.
`answeredBy`	AbAnsweredBy	Yes	What answers for a side, in the words a benchmark run is started with: the kind, the data agent when it is one and the workspace it lives in, and the saved AI provider when the side asks a generic AI agent.
`kind`	string or null	Yes	`DataAgent`, `BIPixieTestAgent`, `PowerBICopilot` or `GenericAgent`, as `agent.kind` on a benchmark run.
`dataAgentId`	string or null	No	The data agent that answers the side. Null for a `PowerBICopilot` or `GenericAgent` side, as are the other data agent fields.
`dataAgentName`	string or null	No
`dataAgentWorkspaceId`	string or null	No
`dataAgentWorkspaceName`	string or null	No	Where the data agent lives, when the test recorded it. A run's result reads it from the test, so it names the same workspace the test does.
`providerId`	string or null	No	The saved AI provider a generic AI agent side asks. On a run's result it is the one that answered the side, as its stored result recorded it.
`providerName`	string or null	No	The saved AI provider's name, when one was recorded.
`questions`	object	Yes	Which questions the test asks, and where they came from.
`count`	integer	Yes
`questionSetHash`	string or null	No	The hash of the question set the test froze, which a run and the saved questions carry too.
`source`	object	Yes
`kind`	string	Yes	`Run` for a test made from a stored benchmark run, `Questions` for one made from the saved questions. One of `Run`, `Questions`.
`runId`	string or null	No	The stored benchmark run side A replays, when the test was made from one and that run still exists.
`runStoredAtUtc`	string or null (date-time)	No
`correctAnswers`	string	Yes	`Preserve` keeps the correct answers the test was made with on every run; `Update` computes them again from side A on each run. One of `Preserve`, `Update`.
`repeats`	integer	No	How many times each question is asked on each side.
`runCount`	integer	Yes
`latestRun`	one of AbLatestRun, null	No
`latestSucceededRunId`	string or null	No
`latestSucceededRunAggregate`	one of AbAggregate, null	No	The scores and response times of the newest run that succeeded, the run `latestSucceededRunId` names. Null when no run has succeeded.
`runGaps`	array of integer	No	Each succeeded run's gap, side B's correct answers minus side A's, oldest first, which is what the consistency line states.
`description`	string or null	No	The customer's own words about the test, when any were written.
`hypothesis`	object or null	No	What the customer expected, when they wrote it.
`text`	string	No
`kind`	string or null	No	`A`, `B` or `Same` when the hypothesis is one of the suggested sentences; null for the customer's own words.
`runs`	array of AbTestRun	Yes	The test's runs, newest first, at most fifty.
`runId`	string	Yes
`abTestId`	string	Yes
`status`	string or null	Yes	`Running`, `Succeeded`, `Failed` or `Cancelled`.
`stage`	string or null	No	Where a run is or was: `SideA`, `SideB`, `Comparing` or `Done`.
`startedAtUtc`	string or null (date-time)	No
`finishedAtUtc`	string or null (date-time)	No
`sideA`	AbRunSide	Yes	One side of a run as it stands.
`status`	string	Yes	`Waiting`, `Reused`, `Pending`, `Running`, `Completed`, `Failed`, or `Cancelled` for the side that was running when the run was stopped. One of `Waiting`, `Reused`, `Pending`, `Running`, `Completed`, `Failed`, `Cancelled`.
`benchmarkRunId`	string or null	No	The side's own benchmark run, which that semantic model's benchmark runs route opens.
`reusedFromRunId`	string or null	No	The stored run a side reused instead of running, when it did.
`progress`	object or null	No
`answered`	integer	No
`total`	integer	No
`stalled`	boolean	No
`errorCode`	string or null	No	Why the side failed, as an error code, when it did.
`capacityWorkspaceId`	string or null	No	The workspace whose capacity refused the side, when one did.
`stable`	integer or null	No
`scored`	integer or null	No
`total`	integer or null	No
`storedAtUtc`	string or null (date-time)	No
`responseTime`	one of BenchmarkResponseTime, null	No
`sideB`	AbRunSide	Yes	One side of a run as it stands.
`status`	string	Yes	`Waiting`, `Reused`, `Pending`, `Running`, `Completed`, `Failed`, or `Cancelled` for the side that was running when the run was stopped. One of `Waiting`, `Reused`, `Pending`, `Running`, `Completed`, `Failed`, `Cancelled`.
`benchmarkRunId`	string or null	No	The side's own benchmark run, which that semantic model's benchmark runs route opens.
`reusedFromRunId`	string or null	No	The stored run a side reused instead of running, when it did.
`progress`	object or null	No
`answered`	integer	No
`total`	integer	No
`stalled`	boolean	No
`errorCode`	string or null	No	Why the side failed, as an error code, when it did.
`capacityWorkspaceId`	string or null	No	The workspace whose capacity refused the side, when one did.
`stable`	integer or null	No
`scored`	integer or null	No
`total`	integer or null	No
`storedAtUtc`	string or null (date-time)	No
`responseTime`	one of BenchmarkResponseTime, null	No
`aggregate`	one of AbAggregate, null	No
`agreement`	one of AbAgreement, null	No
`questionSetHash`	string or null	No	The hash of the question set this run asked.
`questionsCount`	integer	No
`counted`	boolean or null	No	Whether this run counts against the account's benchmark runs, and while it runs, whether it would if stopped now. Null when the run predates the record.
`asks`	object	No	How many questions the run has asked so far, and how many it will ask across the sides that ask.
`made`	integer	No
`total`	integer	No
`cancel`	object or null	No	Present once the run was stopped, frozen at the moment of the stop.
`side`	string or null	No
`counted`	boolean	No
`asked`	integer	No
`total`	integer	No
`error`	object or null	No	Why a failed run failed, with a code from the operation vocabulary.
`errorCode`	string	No
`message`	string	No

Errors

Every refusal is an ErrorResponse with a stable errorCode. See Errors for what each HTTP status means and every code the API can return.

This is an example of a refusal. Every refusal has the same fields.

Example refusal

HTTP/1.1 404 Not Found
Content-Type: application/json
x-ms-request-id: cbf21b6d-b661-4b52-88cb-d79712e504fc

{
  "errorCode": "ItemNotFound",
  "message": "No item with this id is visible to the signed-in person in this workspace, or it is not of the type the route takes.",
  "isRetriable": false,
  "moreDetails": [],
  "relatedResource": {
    "resourceId": "d972b3f5-a2fa-47ec-94ff-0403a4d159fa",
    "resourceType": "SemanticModel"
  },
  "parameters": [
    {
      "name": "itemId",
      "value": "d972b3f5-a2fa-47ec-94ff-0403a4d159fa"
    }
  ],
  "requestId": "cbf21b6d-b661-4b52-88cb-d79712e504fc"
}

Example

Request

curl -i -X GET \
  -H "Authorization: Bearer $TOKEN" \
  "https://api.bipixie.com/v1/aiReadiness/abTests/$AB_TEST_ID"

An example of what the call answers. The values are fictional.

Response, 200

HTTP/1.1 200 OK
Content-Type: application/json
x-ms-request-id: cbf21b6d-b661-4b52-88cb-d79712e504fc

{
  "abTestId": "7c016a91-7eea-4f0c-b327-e32f68230272",
  "name": "Contoso Sales",
  "createdAtUtc": "2026-06-03T14:12:00Z",
  "sideA": {
    "semanticModelId": "f1fdc461-3ca1-4702-ae48-afc3522409fc",
    "workspaceId": "e17c1b59-7a2e-491e-acdd-11434854bbb0",
    "semanticModelName": "Contoso Sales",
    "workspaceName": "Contoso Sales Analytics",
    "label": "With AI instructions",
    "description": "Sales questions answered with and without the AI instructions.",
    "answeredBy": {
      "kind": "DataAgent",
      "dataAgentId": "05dd87d1-624f-4ffa-ab88-ee231a75a904",
      "dataAgentName": "Contoso Sales Agent",
      "dataAgentWorkspaceId": "3a287f9c-63af-48a1-8c06-61c2e8095e5f",
      "dataAgentWorkspaceName": "Contoso Sales Analytics",
      "providerId": "b2df6be3-7769-4222-90ea-9fd98f4592e0",
      "providerName": "Azure OpenAI"
    }
  },
  "sideB": {
    "semanticModelId": "f1fdc461-3ca1-4702-ae48-afc3522409fc",
    "workspaceId": "e17c1b59-7a2e-491e-acdd-11434854bbb0",
    "semanticModelName": "Contoso Sales",
    "workspaceName": "Contoso Sales Analytics",
    "label": "With AI instructions",
    "description": "Sales questions answered with and without the AI instructions.",
    "answeredBy": {
      "kind": "DataAgent",
      "dataAgentId": "05dd87d1-624f-4ffa-ab88-ee231a75a904",
      "dataAgentName": "Contoso Sales Agent",
      "dataAgentWorkspaceId": "3a287f9c-63af-48a1-8c06-61c2e8095e5f",
      "dataAgentWorkspaceName": "Contoso Sales Analytics",
      "providerId": "b2df6be3-7769-4222-90ea-9fd98f4592e0",
      "providerName": "Azure OpenAI"
    }
  },
  "questions": {
    "count": 12,
    "questionSetHash": "12056ac56f6dd1de",
    "source": {
      "kind": "Run",
      "runId": "f2e14a95-8025-41de-a48f-860f8cfd2e68",
      "runStoredAtUtc": "2026-09-14T09:42:17Z"
    },
    "correctAnswers": "Preserve"
  },
  "repeats": 3,
  "runCount": 12,
  "latestRun": null,
  "latestSucceededRunId": "404dca33-3888-443a-8788-5329a0546348",
  "latestSucceededRunAggregate": null,
  "runGaps": [
    3
  ],
  "description": "Sales questions answered with and without the AI instructions.",
  "hypothesis": {
    "text": "The side with AI instructions answers more questions correctly.",
    "kind": "A"
  },
  "runs": [
    {
      "runId": "f2e14a95-8025-41de-a48f-860f8cfd2e68",
      "abTestId": "7c016a91-7eea-4f0c-b327-e32f68230272",
      "status": "Running",
      "stage": "SideA",
      "startedAtUtc": "2026-09-14T09:30:05Z",
      "finishedAtUtc": "2026-09-14T09:42:17Z",
      "sideA": {
        "status": "Waiting",
        "benchmarkRunId": "3385922a-8c4f-494b-ac72-3b84555e499d",
        "reusedFromRunId": "0bf62889-ac10-471e-b4cc-480fbf0444fc",
        "progress": {
          "answered": 3,
          "total": 12
        },
        "stalled": true,
        "errorCode": "ItemNotFound",
        "capacityWorkspaceId": "532cb9ad-8b0e-4e52-8cc1-e74f1ad2dfec",
        "stable": 3,
        "scored": 12,
        "total": 12,
        "storedAtUtc": "2026-09-14T09:42:17Z",
        "responseTime": null
      },
      "sideB": {
        "status": "Waiting",
        "benchmarkRunId": "3385922a-8c4f-494b-ac72-3b84555e499d",
        "reusedFromRunId": "0bf62889-ac10-471e-b4cc-480fbf0444fc",
        "progress": {
          "answered": 3,
          "total": 12
        },
        "stalled": true,
        "errorCode": "ItemNotFound",
        "capacityWorkspaceId": "532cb9ad-8b0e-4e52-8cc1-e74f1ad2dfec",
        "stable": 3,
        "scored": 12,
        "total": 12,
        "storedAtUtc": "2026-09-14T09:42:17Z",
        "responseTime": null
      },
      "aggregate": null,
      "agreement": null,
      "questionSetHash": "12056ac56f6dd1de",
      "questionsCount": 12,
      "counted": true,
      "asks": {
        "made": 3,
        "total": 12
      },
      "cancel": {
        "side": "SideB",
        "counted": true,
        "asked": 3,
        "total": 12
      },
      "error": {
        "errorCode": "ItemNotFound",
        "message": "The semantic model could not be read."
      }
    }
  ]
}

Change an A/B test's name, labels, descriptions or hypothesis #

PATCH /v1/aiReadiness/abTests/{abTestId}

Changes the words of an A/B test, as the BI Pixie portal's edit does: its name, each side's label and description, the test's description and its hypothesis. Only the fields the body names change, and a field sent as null or empty is cleared; the name can be changed but not cleared. The sides, the questions and the runs stay as they were, so any other field is refused with 400 InvalidRequest naming it. The answer is the test as getAbTest reads it, without its runs. Nothing runs and nothing is counted.

Scope: BIPixie.ReadWrite.All
Plan: Every plan
Counts against: Calls that start work, 30 a minute

Parameters

Parameters for Change an A/B test's name, labels, descriptions or hypothesis
Name	In	Type	Required	Description
`abTestId`	path	string (uuid)	Yes	The A/B test's id, as the A/B tests list gives it.

Request body

JSON, UpdateAbTestRequest. The body is required.

Request body for Change an A/B test's name, labels, descriptions or hypothesis
Field	Type	Required	Description
`name`	string	No
`description`	string or null	No	Your own words about the test.
`hypothesis`	object or null	No	What you expect. Null clears it.
`text`	string	Yes
`kind`	string	No	Which side you expect to answer better, when the expectation is one of the suggested sentences. One of `A`, `B`, `Same`.
`sideA`	UpdateAbTestSide	No	Your words about one side. The side itself cannot change.
`label`	string or null	No	Your short name for the side.
`description`	string or null	No	Your own words about the side.
`sideB`	UpdateAbTestSide	No	Your words about one side. The side itself cannot change.
`label`	string or null	No	Your short name for the side.
`description`	string or null	No	Your own words about the side.

Responses

200. The test, changed.

Headers of the 200 response to Change an A/B test's name, labels, descriptions or hypothesis
Header	Description
`x-ms-request-id`	The id of this request. Quote it to support.

The 200 response to Change an A/B test's name, labels, descriptions or hypothesis, AbTest
Field	Type	Always present	Description
`abTestId`	string	Yes
`name`	string	Yes
`createdAtUtc`	string or null (date-time)	No
`sideA`	AbSide	Yes	One side of a test as it was set up.
`semanticModelId`	string or null	Yes
`workspaceId`	string or null	Yes
`semanticModelName`	string or null	No
`workspaceName`	string or null	No
`label`	string or null	No	The customer's short name for the side, when one was written.
`description`	string or null	No	The customer's own words about the side, when any were written.
`answeredBy`	AbAnsweredBy	Yes	What answers for a side, in the words a benchmark run is started with: the kind, the data agent when it is one and the workspace it lives in, and the saved AI provider when the side asks a generic AI agent.
`kind`	string or null	Yes	`DataAgent`, `BIPixieTestAgent`, `PowerBICopilot` or `GenericAgent`, as `agent.kind` on a benchmark run.
`dataAgentId`	string or null	No	The data agent that answers the side. Null for a `PowerBICopilot` or `GenericAgent` side, as are the other data agent fields.
`dataAgentName`	string or null	No
`dataAgentWorkspaceId`	string or null	No
`dataAgentWorkspaceName`	string or null	No	Where the data agent lives, when the test recorded it. A run's result reads it from the test, so it names the same workspace the test does.
`providerId`	string or null	No	The saved AI provider a generic AI agent side asks. On a run's result it is the one that answered the side, as its stored result recorded it.
`providerName`	string or null	No	The saved AI provider's name, when one was recorded.
`sideB`	AbSide	Yes	One side of a test as it was set up.
`semanticModelId`	string or null	Yes
`workspaceId`	string or null	Yes
`semanticModelName`	string or null	No
`workspaceName`	string or null	No
`label`	string or null	No	The customer's short name for the side, when one was written.
`description`	string or null	No	The customer's own words about the side, when any were written.
`answeredBy`	AbAnsweredBy	Yes	What answers for a side, in the words a benchmark run is started with: the kind, the data agent when it is one and the workspace it lives in, and the saved AI provider when the side asks a generic AI agent.
`kind`	string or null	Yes	`DataAgent`, `BIPixieTestAgent`, `PowerBICopilot` or `GenericAgent`, as `agent.kind` on a benchmark run.
`dataAgentId`	string or null	No	The data agent that answers the side. Null for a `PowerBICopilot` or `GenericAgent` side, as are the other data agent fields.
`dataAgentName`	string or null	No
`dataAgentWorkspaceId`	string or null	No
`dataAgentWorkspaceName`	string or null	No	Where the data agent lives, when the test recorded it. A run's result reads it from the test, so it names the same workspace the test does.
`providerId`	string or null	No	The saved AI provider a generic AI agent side asks. On a run's result it is the one that answered the side, as its stored result recorded it.
`providerName`	string or null	No	The saved AI provider's name, when one was recorded.
`questions`	object	Yes	Which questions the test asks, and where they came from.
`count`	integer	Yes
`questionSetHash`	string or null	No	The hash of the question set the test froze, which a run and the saved questions carry too.
`source`	object	Yes
`kind`	string	Yes	`Run` for a test made from a stored benchmark run, `Questions` for one made from the saved questions. One of `Run`, `Questions`.
`runId`	string or null	No	The stored benchmark run side A replays, when the test was made from one and that run still exists.
`runStoredAtUtc`	string or null (date-time)	No
`correctAnswers`	string	Yes	`Preserve` keeps the correct answers the test was made with on every run; `Update` computes them again from side A on each run. One of `Preserve`, `Update`.
`repeats`	integer	No	How many times each question is asked on each side.
`runCount`	integer	Yes
`latestRun`	one of AbLatestRun, null	No
`latestSucceededRunId`	string or null	No
`latestSucceededRunAggregate`	one of AbAggregate, null	No	The scores and response times of the newest run that succeeded, the run `latestSucceededRunId` names. Null when no run has succeeded.
`runGaps`	array of integer	No	Each succeeded run's gap, side B's correct answers minus side A's, oldest first, which is what the consistency line states.
`description`	string or null	No	The customer's own words about the test, when any were written.
`hypothesis`	object or null	No	What the customer expected, when they wrote it.
`text`	string	No
`kind`	string or null	No	`A`, `B` or `Same` when the hypothesis is one of the suggested sentences; null for the customer's own words.

Headers of the 400 response to Change an A/B test's name, labels, descriptions or hypothesis
Header	Description
`x-ms-request-id`	The id of this request. Quote it to support.

The 400 response to Change an A/B test's name, labels, descriptions or hypothesis, ErrorResponse
Field	Type	Always present	Description
`errorCode`	string	Yes	Each code keeps its meaning once published. New codes can be added over time, and a code is never reused. One of `InvalidRequest`, `InvalidPreview`, `PayloadTooLarge`, `BenchmarkQuestionsMissing`, `OperationNotSupportedForItem`, `Unauthorized`, `TokenExpired`, `InsufficientScopes`, `AppOnlyTokenNotSupported`, `UserNotRegistered`, `AccountNotActive`, `AdminRoleRequired`, `FeatureNotInPlan`, `FeatureTurnedOff`, `LicenseNotYours`, `TrackedItemLimitReached`, `LicenseTrackedItemLimitReached`, `FabricConsentRequired`, `FabricAccessDenied`, `WorkspaceNotFound`, `ItemNotFound`, `AssessmentNotFound`, `RunNotFound`, `AbTestNotFound`, `OperationNotFound`, `ScheduleNotFound`, `AiProviderNotFound`, `AiCredentialNotFound`, `OperationInProgress`, `OperationHasNoResult`, `ResultNotReady`, `ScheduleConflict`, `AiProviderConflict`, `ItemChanged`, `CapacityPaused`, `OperationExpired`, `RegionNotReady`, `CapacityRequired`, `CapacityUnpaid`, `StoreNotConnected`, `RequestBlocked`, `AssessmentDailyLimitReached`, `BenchmarkRunsExhausted`, `OptimizationApplyDailyLimitReached`, `FirstLookLimitReached`, `BenchmarkBuildLimitReached`, `FabricUnavailable`, `StoreUnavailable`, `ThrottleStoreUnavailable`, `UnknownError`.
`message`	string	Yes
`isRetriable`	boolean	No
`moreDetails`	array of object	No
`relatedResource`	object or null	No
`resourceId`	string	No
`resourceType`	string	No
`parameters`	array of object	No
`name`	string	Yes
`value`	string	Yes
`requestId`	string (uuid)	Yes

Errors

Every refusal is an ErrorResponse with a stable errorCode. See Errors for what each HTTP status means and every code the API can return.

This is an example of a refusal. Every refusal has the same fields.

Example refusal

HTTP/1.1 404 Not Found
Content-Type: application/json
x-ms-request-id: cbf21b6d-b661-4b52-88cb-d79712e504fc

{
  "errorCode": "ItemNotFound",
  "message": "No item with this id is visible to the signed-in person in this workspace, or it is not of the type the route takes.",
  "isRetriable": false,
  "moreDetails": [],
  "relatedResource": {
    "resourceId": "d972b3f5-a2fa-47ec-94ff-0403a4d159fa",
    "resourceType": "SemanticModel"
  },
  "parameters": [
    {
      "name": "itemId",
      "value": "d972b3f5-a2fa-47ec-94ff-0403a4d159fa"
    }
  ],
  "requestId": "cbf21b6d-b661-4b52-88cb-d79712e504fc"
}

Example

Request

curl -i -X PATCH \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
  "name": "<string>",
  "description": "<string>",
  "hypothesis": {
    "text": "<string>"
  },
  "sideA": {},
  "sideB": {}
}' \
  "https://api.bipixie.com/v1/aiReadiness/abTests/$AB_TEST_ID"

An example of what the call answers. The values are fictional.

Response, 200

HTTP/1.1 200 OK
Content-Type: application/json
x-ms-request-id: cbf21b6d-b661-4b52-88cb-d79712e504fc

{
  "abTestId": "7c016a91-7eea-4f0c-b327-e32f68230272",
  "name": "Contoso Sales",
  "createdAtUtc": "2026-06-03T14:12:00Z",
  "sideA": {
    "semanticModelId": "f1fdc461-3ca1-4702-ae48-afc3522409fc",
    "workspaceId": "e17c1b59-7a2e-491e-acdd-11434854bbb0",
    "semanticModelName": "Contoso Sales",
    "workspaceName": "Contoso Sales Analytics",
    "label": "With AI instructions",
    "description": "Sales questions answered with and without the AI instructions.",
    "answeredBy": {
      "kind": "DataAgent",
      "dataAgentId": "05dd87d1-624f-4ffa-ab88-ee231a75a904",
      "dataAgentName": "Contoso Sales Agent",
      "dataAgentWorkspaceId": "3a287f9c-63af-48a1-8c06-61c2e8095e5f",
      "dataAgentWorkspaceName": "Contoso Sales Analytics",
      "providerId": "b2df6be3-7769-4222-90ea-9fd98f4592e0",
      "providerName": "Azure OpenAI"
    }
  },
  "sideB": {
    "semanticModelId": "f1fdc461-3ca1-4702-ae48-afc3522409fc",
    "workspaceId": "e17c1b59-7a2e-491e-acdd-11434854bbb0",
    "semanticModelName": "Contoso Sales",
    "workspaceName": "Contoso Sales Analytics",
    "label": "With AI instructions",
    "description": "Sales questions answered with and without the AI instructions.",
    "answeredBy": {
      "kind": "DataAgent",
      "dataAgentId": "05dd87d1-624f-4ffa-ab88-ee231a75a904",
      "dataAgentName": "Contoso Sales Agent",
      "dataAgentWorkspaceId": "3a287f9c-63af-48a1-8c06-61c2e8095e5f",
      "dataAgentWorkspaceName": "Contoso Sales Analytics",
      "providerId": "b2df6be3-7769-4222-90ea-9fd98f4592e0",
      "providerName": "Azure OpenAI"
    }
  },
  "questions": {
    "count": 12,
    "questionSetHash": "12056ac56f6dd1de",
    "source": {
      "kind": "Run",
      "runId": "f2e14a95-8025-41de-a48f-860f8cfd2e68",
      "runStoredAtUtc": "2026-09-14T09:42:17Z"
    },
    "correctAnswers": "Preserve"
  },
  "repeats": 3,
  "runCount": 12,
  "latestRun": null,
  "latestSucceededRunId": "404dca33-3888-443a-8788-5329a0546348",
  "latestSucceededRunAggregate": null,
  "runGaps": [
    3
  ],
  "description": "Sales questions answered with and without the AI instructions.",
  "hypothesis": {
    "text": "The side with AI instructions answers more questions correctly.",
    "kind": "A"
  }
}

Delete an A/B test #

DELETE /v1/aiReadiness/abTests/{abTestId}

Deletes the A/B test, every one of its runs and every comparison, as the BI Pixie portal's delete does. The benchmark runs each side produced are ordinary benchmark runs and are kept. A run that is still going is deleted too, and its operation then reads Failed. Nothing a run spent is given back. An id that names no test of this account answers 404 AbTestNotFound.

Scope: BIPixie.ReadWrite.All
Plan: Every plan
Counts against: Calls that start work, 30 a minute

Parameters

Parameters for Delete an A/B test
Name	In	Type	Required	Description
`abTestId`	path	string (uuid)	Yes	The A/B test's id, as the A/B tests list gives it.

Responses

200. Deleted, with the number of runs deleted with it.

Headers of the 200 response to Delete an A/B test
Header	Description
`x-ms-request-id`	The id of this request. Quote it to support.

The 200 response to Delete an A/B test, DeletedAbTest
Field	Type	Always present	Description
`abTestId`	string (uuid)	Yes	The test deleted.
`deletedRuns`	integer	Yes	How many of the test's runs were deleted with it.

Headers of the 400 response to Delete an A/B test
Header	Description
`x-ms-request-id`	The id of this request. Quote it to support.

The 400 response to Delete an A/B test, ErrorResponse
Field	Type	Always present	Description
`errorCode`	string	Yes	Each code keeps its meaning once published. New codes can be added over time, and a code is never reused. One of `InvalidRequest`, `InvalidPreview`, `PayloadTooLarge`, `BenchmarkQuestionsMissing`, `OperationNotSupportedForItem`, `Unauthorized`, `TokenExpired`, `InsufficientScopes`, `AppOnlyTokenNotSupported`, `UserNotRegistered`, `AccountNotActive`, `AdminRoleRequired`, `FeatureNotInPlan`, `FeatureTurnedOff`, `LicenseNotYours`, `TrackedItemLimitReached`, `LicenseTrackedItemLimitReached`, `FabricConsentRequired`, `FabricAccessDenied`, `WorkspaceNotFound`, `ItemNotFound`, `AssessmentNotFound`, `RunNotFound`, `AbTestNotFound`, `OperationNotFound`, `ScheduleNotFound`, `AiProviderNotFound`, `AiCredentialNotFound`, `OperationInProgress`, `OperationHasNoResult`, `ResultNotReady`, `ScheduleConflict`, `AiProviderConflict`, `ItemChanged`, `CapacityPaused`, `OperationExpired`, `RegionNotReady`, `CapacityRequired`, `CapacityUnpaid`, `StoreNotConnected`, `RequestBlocked`, `AssessmentDailyLimitReached`, `BenchmarkRunsExhausted`, `OptimizationApplyDailyLimitReached`, `FirstLookLimitReached`, `BenchmarkBuildLimitReached`, `FabricUnavailable`, `StoreUnavailable`, `ThrottleStoreUnavailable`, `UnknownError`.
`message`	string	Yes
`isRetriable`	boolean	No
`moreDetails`	array of object	No
`relatedResource`	object or null	No
`resourceId`	string	No
`resourceType`	string	No
`parameters`	array of object	No
`name`	string	Yes
`value`	string	Yes
`requestId`	string (uuid)	Yes

Errors

Every refusal is an ErrorResponse with a stable errorCode. See Errors for what each HTTP status means and every code the API can return.

This is an example of a refusal. Every refusal has the same fields.

Example refusal

HTTP/1.1 404 Not Found
Content-Type: application/json
x-ms-request-id: cbf21b6d-b661-4b52-88cb-d79712e504fc

{
  "errorCode": "ItemNotFound",
  "message": "No item with this id is visible to the signed-in person in this workspace, or it is not of the type the route takes.",
  "isRetriable": false,
  "moreDetails": [],
  "relatedResource": {
    "resourceId": "d972b3f5-a2fa-47ec-94ff-0403a4d159fa",
    "resourceType": "SemanticModel"
  },
  "parameters": [
    {
      "name": "itemId",
      "value": "d972b3f5-a2fa-47ec-94ff-0403a4d159fa"
    }
  ],
  "requestId": "cbf21b6d-b661-4b52-88cb-d79712e504fc"
}

Example

Request

curl -i -X DELETE \
  -H "Authorization: Bearer $TOKEN" \
  "https://api.bipixie.com/v1/aiReadiness/abTests/$AB_TEST_ID"

An example of what the call answers. The values are fictional.

Response, 200

HTTP/1.1 200 OK
Content-Type: application/json
x-ms-request-id: cbf21b6d-b661-4b52-88cb-d79712e504fc

{
  "abTestId": "7c016a91-7eea-4f0c-b327-e32f68230272",
  "deletedRuns": 3
}

Run an A/B test again #

POST /v1/aiReadiness/abTests/{abTestId}/runs

Starts another run of the test, as the BI Pixie portal's Run again does, as one operation of type RunAbTest. Both sides ask the questions the test froze, or, with questionSet: Current, the saved benchmark questions side A's semantic model holds now, the ones that can run, as runBenchmark asks them, over the reporting period the test fixed; every run records which questions it asked. reuseRunId keeps a stored benchmark run of side A's semantic model as side A, which must have asked the frozen questions, and asks side B alone with this request's token, so the run counts one side. The run counts toward the account's benchmark runs as the first run did. Both semantic models are looked up as the signed-in person before anything is counted. Side B starts on the operation owner's read, as createAbTest describes, and the operation's result is the run's comparison. A test whose side A has not finished its first run from the saved benchmark questions has nothing frozen yet and answers 409 ResultNotReady; a test whose source run was deleted answers 404 RunNotFound. The body may be empty.

Scope: BIPixie.ReadWrite.All
Plan: Every plan
Counts against: Calls that start work, 30 a minute

Parameters

Parameters for Run an A/B test again
Name	In	Type	Required	Description
`abTestId`	path	string (uuid)	Yes	The A/B test's id, as the A/B tests list gives it.

Request body

JSON, RunAbTestAgainRequest. The body is optional.

Request body for Run an A/B test again
Field	Type	Required	Description
`questionSet`	string	No	`Frozen`, the default, asks the questions the test froze; `Current` asks the saved benchmark questions side A's semantic model holds now. One of `Frozen`, `Current`.
`reuseRunId`	string (uuid)	No	A stored benchmark run of side A's semantic model to keep as side A, so only side B asks. Not with `Current`.

Responses

202. The run was started. The body carries the operation's id and address.

Headers of the 202 response to Run an A/B test again
Header	Description
`x-ms-request-id`	The id of this request. Quote it to support.
`x-ms-operation-id`	The id of the operation.
`Location`	The address of the operation's state.
`Retry-After`	Seconds to wait before asking for the operation's state.

To follow the work this call started, see Follow a long-running operation.

The 202 response to Run an A/B test again, Accepted
Field	Type	Always present	Description
`operationId`	string (uuid)	Yes	The operation's id, which the operation's state and result take.
`location`	string (uri)	Yes	Where to ask about the operation.

Headers of the 400 response to Run an A/B test again
Header	Description
`x-ms-request-id`	The id of this request. Quote it to support.

The 400 response to Run an A/B test again, ErrorResponse
Field	Type	Always present	Description
`errorCode`	string	Yes	Each code keeps its meaning once published. New codes can be added over time, and a code is never reused. One of `InvalidRequest`, `InvalidPreview`, `PayloadTooLarge`, `BenchmarkQuestionsMissing`, `OperationNotSupportedForItem`, `Unauthorized`, `TokenExpired`, `InsufficientScopes`, `AppOnlyTokenNotSupported`, `UserNotRegistered`, `AccountNotActive`, `AdminRoleRequired`, `FeatureNotInPlan`, `FeatureTurnedOff`, `LicenseNotYours`, `TrackedItemLimitReached`, `LicenseTrackedItemLimitReached`, `FabricConsentRequired`, `FabricAccessDenied`, `WorkspaceNotFound`, `ItemNotFound`, `AssessmentNotFound`, `RunNotFound`, `AbTestNotFound`, `OperationNotFound`, `ScheduleNotFound`, `AiProviderNotFound`, `AiCredentialNotFound`, `OperationInProgress`, `OperationHasNoResult`, `ResultNotReady`, `ScheduleConflict`, `AiProviderConflict`, `ItemChanged`, `CapacityPaused`, `OperationExpired`, `RegionNotReady`, `CapacityRequired`, `CapacityUnpaid`, `StoreNotConnected`, `RequestBlocked`, `AssessmentDailyLimitReached`, `BenchmarkRunsExhausted`, `OptimizationApplyDailyLimitReached`, `FirstLookLimitReached`, `BenchmarkBuildLimitReached`, `FabricUnavailable`, `StoreUnavailable`, `ThrottleStoreUnavailable`, `UnknownError`.
`message`	string	Yes
`isRetriable`	boolean	No
`moreDetails`	array of object	No
`relatedResource`	object or null	No
`resourceId`	string	No
`resourceType`	string	No
`parameters`	array of object	No
`name`	string	Yes
`value`	string	Yes
`requestId`	string (uuid)	Yes

Errors

Every refusal is an ErrorResponse with a stable errorCode. See Errors for what each HTTP status means and every code the API can return.

This is an example of a refusal. Every refusal has the same fields.

Example refusal

HTTP/1.1 404 Not Found
Content-Type: application/json
x-ms-request-id: cbf21b6d-b661-4b52-88cb-d79712e504fc

{
  "errorCode": "ItemNotFound",
  "message": "No item with this id is visible to the signed-in person in this workspace, or it is not of the type the route takes.",
  "isRetriable": false,
  "moreDetails": [],
  "relatedResource": {
    "resourceId": "d972b3f5-a2fa-47ec-94ff-0403a4d159fa",
    "resourceType": "SemanticModel"
  },
  "parameters": [
    {
      "name": "itemId",
      "value": "d972b3f5-a2fa-47ec-94ff-0403a4d159fa"
    }
  ],
  "requestId": "cbf21b6d-b661-4b52-88cb-d79712e504fc"
}

Example

Request

curl -i -X POST \
  -H "Authorization: Bearer $TOKEN" \
  "https://api.bipixie.com/v1/aiReadiness/abTests/$AB_TEST_ID/runs"

An example of what the call answers. The values are fictional.

Response, 202

HTTP/1.1 202 Accepted
Content-Type: application/json
x-ms-request-id: cbf21b6d-b661-4b52-88cb-d79712e504fc
x-ms-operation-id: cd57b041-e44a-4dda-a4ff-02935ffb8c04
Location: https://api.bipixie.com/v1/operations/cd57b041-e44a-4dda-a4ff-02935ffb8c04
Retry-After: 5

{
  "operationId": "cd57b041-e44a-4dda-a4ff-02935ffb8c04",
  "location": "https://api.bipixie.com/v1/aiReadiness/abTests/7c016a91-7eea-4f0c-b327-e32f68230272/runs"
}

Get one run of an A/B test #

GET /v1/aiReadiness/abTests/{abTestId}/runs/{runId}

One run as it stands: its status in the operation vocabulary and the stage it is at, each side with its own benchmark run and its progress, the totals once both sides have been scored, and the agreement check made before the run. Reading a run moves it forward exactly as the portal's poll does: side B starts once side A has landed, and the comparison once side B has. That spends nothing, because a run is charged when it starts. An id that names no run of this test answers 404 RunNotFound.

Scope: BIPixie.Read.All
Plan: Every plan
Counts against: Calls that read, 300 a minute

Parameters

Parameters for Get one run of an A/B test
Name	In	Type	Required	Description
`abTestId`	path	string (uuid)	Yes	The A/B test's id, as the A/B tests list gives it.
`runId`	path	string (uuid)	Yes	The run's id, as the runs list gives it.

Responses

200. The run.

Headers of the 200 response to Get one run of an A/B test
Header	Description
`x-ms-request-id`	The id of this request. Quote it to support.

The 200 response to Get one run of an A/B test, AbTestRun
Field	Type	Always present	Description
`runId`	string	Yes
`abTestId`	string	Yes
`status`	string or null	Yes	`Running`, `Succeeded`, `Failed` or `Cancelled`.
`stage`	string or null	No	Where a run is or was: `SideA`, `SideB`, `Comparing` or `Done`.
`startedAtUtc`	string or null (date-time)	No
`finishedAtUtc`	string or null (date-time)	No
`sideA`	AbRunSide	Yes	One side of a run as it stands.
`status`	string	Yes	`Waiting`, `Reused`, `Pending`, `Running`, `Completed`, `Failed`, or `Cancelled` for the side that was running when the run was stopped. One of `Waiting`, `Reused`, `Pending`, `Running`, `Completed`, `Failed`, `Cancelled`.
`benchmarkRunId`	string or null	No	The side's own benchmark run, which that semantic model's benchmark runs route opens.
`reusedFromRunId`	string or null	No	The stored run a side reused instead of running, when it did.
`progress`	object or null	No
`answered`	integer	No
`total`	integer	No
`stalled`	boolean	No
`errorCode`	string or null	No	Why the side failed, as an error code, when it did.
`capacityWorkspaceId`	string or null	No	The workspace whose capacity refused the side, when one did.
`stable`	integer or null	No
`scored`	integer or null	No
`total`	integer or null	No
`storedAtUtc`	string or null (date-time)	No
`responseTime`	one of BenchmarkResponseTime, null	No
`sideB`	AbRunSide	Yes	One side of a run as it stands.
`status`	string	Yes	`Waiting`, `Reused`, `Pending`, `Running`, `Completed`, `Failed`, or `Cancelled` for the side that was running when the run was stopped. One of `Waiting`, `Reused`, `Pending`, `Running`, `Completed`, `Failed`, `Cancelled`.
`benchmarkRunId`	string or null	No	The side's own benchmark run, which that semantic model's benchmark runs route opens.
`reusedFromRunId`	string or null	No	The stored run a side reused instead of running, when it did.
`progress`	object or null	No
`answered`	integer	No
`total`	integer	No
`stalled`	boolean	No
`errorCode`	string or null	No	Why the side failed, as an error code, when it did.
`capacityWorkspaceId`	string or null	No	The workspace whose capacity refused the side, when one did.
`stable`	integer or null	No
`scored`	integer or null	No
`total`	integer or null	No
`storedAtUtc`	string or null (date-time)	No
`responseTime`	one of BenchmarkResponseTime, null	No
`aggregate`	one of AbAggregate, null	No
`agreement`	one of AbAgreement, null	No
`questionSetHash`	string or null	No	The hash of the question set this run asked.
`questionsCount`	integer	No
`counted`	boolean or null	No	Whether this run counts against the account's benchmark runs, and while it runs, whether it would if stopped now. Null when the run predates the record.
`asks`	object	No	How many questions the run has asked so far, and how many it will ask across the sides that ask.
`made`	integer	No
`total`	integer	No
`cancel`	object or null	No	Present once the run was stopped, frozen at the moment of the stop.
`side`	string or null	No
`counted`	boolean	No
`asked`	integer	No
`total`	integer	No
`error`	object or null	No	Why a failed run failed, with a code from the operation vocabulary.
`errorCode`	string	No
`message`	string	No

Errors

Every refusal is an ErrorResponse with a stable errorCode. See Errors for what each HTTP status means and every code the API can return.

This is an example of a refusal. Every refusal has the same fields.

Example refusal

HTTP/1.1 404 Not Found
Content-Type: application/json
x-ms-request-id: cbf21b6d-b661-4b52-88cb-d79712e504fc

{
  "errorCode": "ItemNotFound",
  "message": "No item with this id is visible to the signed-in person in this workspace, or it is not of the type the route takes.",
  "isRetriable": false,
  "moreDetails": [],
  "relatedResource": {
    "resourceId": "d972b3f5-a2fa-47ec-94ff-0403a4d159fa",
    "resourceType": "SemanticModel"
  },
  "parameters": [
    {
      "name": "itemId",
      "value": "d972b3f5-a2fa-47ec-94ff-0403a4d159fa"
    }
  ],
  "requestId": "cbf21b6d-b661-4b52-88cb-d79712e504fc"
}

Example

Request

curl -i -X GET \
  -H "Authorization: Bearer $TOKEN" \
  "https://api.bipixie.com/v1/aiReadiness/abTests/$AB_TEST_ID/runs/$RUN_ID"

An example of what the call answers. The values are fictional.

Response, 200

HTTP/1.1 200 OK
Content-Type: application/json
x-ms-request-id: cbf21b6d-b661-4b52-88cb-d79712e504fc

{
  "runId": "f2e14a95-8025-41de-a48f-860f8cfd2e68",
  "abTestId": "7c016a91-7eea-4f0c-b327-e32f68230272",
  "status": "Running",
  "stage": "SideA",
  "startedAtUtc": "2026-09-14T09:30:05Z",
  "finishedAtUtc": "2026-09-14T09:42:17Z",
  "sideA": {
    "status": "Waiting",
    "benchmarkRunId": "3385922a-8c4f-494b-ac72-3b84555e499d",
    "reusedFromRunId": "0bf62889-ac10-471e-b4cc-480fbf0444fc",
    "progress": {
      "answered": 3,
      "total": 12
    },
    "stalled": true,
    "errorCode": "ItemNotFound",
    "capacityWorkspaceId": "532cb9ad-8b0e-4e52-8cc1-e74f1ad2dfec",
    "stable": 3,
    "scored": 12,
    "total": 12,
    "storedAtUtc": "2026-09-14T09:42:17Z",
    "responseTime": null
  },
  "sideB": {
    "status": "Waiting",
    "benchmarkRunId": "3385922a-8c4f-494b-ac72-3b84555e499d",
    "reusedFromRunId": "0bf62889-ac10-471e-b4cc-480fbf0444fc",
    "progress": {
      "answered": 3,
      "total": 12
    },
    "stalled": true,
    "errorCode": "ItemNotFound",
    "capacityWorkspaceId": "532cb9ad-8b0e-4e52-8cc1-e74f1ad2dfec",
    "stable": 3,
    "scored": 12,
    "total": 12,
    "storedAtUtc": "2026-09-14T09:42:17Z",
    "responseTime": null
  },
  "aggregate": null,
  "agreement": null,
  "questionSetHash": "12056ac56f6dd1de",
  "questionsCount": 12,
  "counted": true,
  "asks": {
    "made": 3,
    "total": 12
  },
  "cancel": {
    "side": "SideB",
    "counted": true,
    "asked": 3,
    "total": 12
  },
  "error": {
    "errorCode": "ItemNotFound",
    "message": "The semantic model could not be read."
  }
}

Delete one run of an A/B test #

DELETE /v1/aiReadiness/abTests/{abTestId}/runs/{runId}

Deletes one run and its comparison, as the BI Pixie portal's delete does. The test stays, and its latest run becomes the newest one left. The benchmark runs each side produced are kept. A run that is still going is deleted too, and its operation then reads Failed. Nothing the run spent is given back. An id that names no run of this test answers 404 RunNotFound, and one that names no test 404 AbTestNotFound.

Scope: BIPixie.ReadWrite.All
Plan: Every plan
Counts against: Calls that start work, 30 a minute

Parameters

Parameters for Delete one run of an A/B test
Name	In	Type	Required	Description
`abTestId`	path	string (uuid)	Yes	The A/B test's id, as the A/B tests list gives it.
`runId`	path	string (uuid)	Yes	The run's id, as the runs list gives it.

Responses

200. Deleted.

Headers of the 200 response to Delete one run of an A/B test
Header	Description
`x-ms-request-id`	The id of this request. Quote it to support.

The 200 response to Delete one run of an A/B test, DeletedAbTestRun
Field	Type	Always present	Description
`abTestId`	string (uuid)	Yes	The test the run belonged to, which stays.
`runId`	string (uuid)	Yes	The run deleted.
`deleted`	boolean	Yes	Always true; a run that could not be deleted is refused instead.

Headers of the 400 response to Delete one run of an A/B test
Header	Description
`x-ms-request-id`	The id of this request. Quote it to support.

The 400 response to Delete one run of an A/B test, ErrorResponse
Field	Type	Always present	Description
`errorCode`	string	Yes	Each code keeps its meaning once published. New codes can be added over time, and a code is never reused. One of `InvalidRequest`, `InvalidPreview`, `PayloadTooLarge`, `BenchmarkQuestionsMissing`, `OperationNotSupportedForItem`, `Unauthorized`, `TokenExpired`, `InsufficientScopes`, `AppOnlyTokenNotSupported`, `UserNotRegistered`, `AccountNotActive`, `AdminRoleRequired`, `FeatureNotInPlan`, `FeatureTurnedOff`, `LicenseNotYours`, `TrackedItemLimitReached`, `LicenseTrackedItemLimitReached`, `FabricConsentRequired`, `FabricAccessDenied`, `WorkspaceNotFound`, `ItemNotFound`, `AssessmentNotFound`, `RunNotFound`, `AbTestNotFound`, `OperationNotFound`, `ScheduleNotFound`, `AiProviderNotFound`, `AiCredentialNotFound`, `OperationInProgress`, `OperationHasNoResult`, `ResultNotReady`, `ScheduleConflict`, `AiProviderConflict`, `ItemChanged`, `CapacityPaused`, `OperationExpired`, `RegionNotReady`, `CapacityRequired`, `CapacityUnpaid`, `StoreNotConnected`, `RequestBlocked`, `AssessmentDailyLimitReached`, `BenchmarkRunsExhausted`, `OptimizationApplyDailyLimitReached`, `FirstLookLimitReached`, `BenchmarkBuildLimitReached`, `FabricUnavailable`, `StoreUnavailable`, `ThrottleStoreUnavailable`, `UnknownError`.
`message`	string	Yes
`isRetriable`	boolean	No
`moreDetails`	array of object	No
`relatedResource`	object or null	No
`resourceId`	string	No
`resourceType`	string	No
`parameters`	array of object	No
`name`	string	Yes
`value`	string	Yes
`requestId`	string (uuid)	Yes

Errors

Every refusal is an ErrorResponse with a stable errorCode. See Errors for what each HTTP status means and every code the API can return.

This is an example of a refusal. Every refusal has the same fields.

Example refusal

HTTP/1.1 404 Not Found
Content-Type: application/json
x-ms-request-id: cbf21b6d-b661-4b52-88cb-d79712e504fc

{
  "errorCode": "ItemNotFound",
  "message": "No item with this id is visible to the signed-in person in this workspace, or it is not of the type the route takes.",
  "isRetriable": false,
  "moreDetails": [],
  "relatedResource": {
    "resourceId": "d972b3f5-a2fa-47ec-94ff-0403a4d159fa",
    "resourceType": "SemanticModel"
  },
  "parameters": [
    {
      "name": "itemId",
      "value": "d972b3f5-a2fa-47ec-94ff-0403a4d159fa"
    }
  ],
  "requestId": "cbf21b6d-b661-4b52-88cb-d79712e504fc"
}

Example

Request

curl -i -X DELETE \
  -H "Authorization: Bearer $TOKEN" \
  "https://api.bipixie.com/v1/aiReadiness/abTests/$AB_TEST_ID/runs/$RUN_ID"

An example of what the call answers. The values are fictional.

Response, 200

HTTP/1.1 200 OK
Content-Type: application/json
x-ms-request-id: cbf21b6d-b661-4b52-88cb-d79712e504fc

{
  "abTestId": "7c016a91-7eea-4f0c-b327-e32f68230272",
  "runId": "f2e14a95-8025-41de-a48f-860f8cfd2e68",
  "deleted": true
}

Get a run's comparison #

GET /v1/aiReadiness/abTests/{abTestId}/runs/{runId}/result

The comparison of a run that succeeded: the headline the result page is computed from, both sides' scores and the totals over every question asked, and under document the comparison as BI Pixie stores it, with every question and both sides' verdicts, relayed from the account's store and never reshaped. While the run is still going the answer is 409 ResultNotReady with Retry-After; once it failed or was stopped it is 409 OperationHasNoResult, and reading the run says why. A store that could not answer is 502 StoreUnavailable, and one behind a paused capacity is 409 CapacityPaused.

Scope: BIPixie.Read.All
Plan: Every plan
Counts against: Calls that read, 300 a minute

Parameters

Parameters for Get a run's comparison
Name	In	Type	Required	Description
`abTestId`	path	string (uuid)	Yes	The A/B test's id, as the A/B tests list gives it.
`runId`	path	string (uuid)	Yes	The run's id, as the runs list gives it.

Responses

200. The comparison.

Headers of the 200 response to Get a run's comparison
Header	Description
`x-ms-request-id`	The id of this request. Quote it to support.

The 200 response to Get a run's comparison, AbTestResult
Field	Type	Always present	Description
`abTestId`	string	Yes
`runId`	string	Yes
`storedAtUtc`	string or null (date-time)	No
`sideA`	one of AbComparisonSide, null	Yes
`sideB`	one of AbComparisonSide, null	Yes
`aggregate`	one of AbAggregate, null	Yes
`agreement`	one of AbAgreement, null	No
`correctAnswers`	object	No	How the correct answers were kept, and when they were computed.
`mode`	string or null	No	One of `Preserve`, `Update`.
`computedAtUtc`	string or null (date-time)	No
`document`	object	Yes	The comparison as stored, except that a `PowerBICopilot` or `GenericAgent` side's `agent_ref.data_agent_id` is null, as `dataAgentId` is, because no data agent answered that side.

Errors

Every refusal is an ErrorResponse with a stable errorCode. See Errors for what each HTTP status means and every code the API can return.

This is an example of a refusal. Every refusal has the same fields.

Example refusal

HTTP/1.1 404 Not Found
Content-Type: application/json
x-ms-request-id: cbf21b6d-b661-4b52-88cb-d79712e504fc

{
  "errorCode": "ItemNotFound",
  "message": "No item with this id is visible to the signed-in person in this workspace, or it is not of the type the route takes.",
  "isRetriable": false,
  "moreDetails": [],
  "relatedResource": {
    "resourceId": "d972b3f5-a2fa-47ec-94ff-0403a4d159fa",
    "resourceType": "SemanticModel"
  },
  "parameters": [
    {
      "name": "itemId",
      "value": "d972b3f5-a2fa-47ec-94ff-0403a4d159fa"
    }
  ],
  "requestId": "cbf21b6d-b661-4b52-88cb-d79712e504fc"
}

Example

Request

curl -i -X GET \
  -H "Authorization: Bearer $TOKEN" \
  "https://api.bipixie.com/v1/aiReadiness/abTests/$AB_TEST_ID/runs/$RUN_ID/result"

An example of what the call answers. The values are fictional.

Response, 200

HTTP/1.1 200 OK
Content-Type: application/json
x-ms-request-id: cbf21b6d-b661-4b52-88cb-d79712e504fc

{
  "abTestId": "7c016a91-7eea-4f0c-b327-e32f68230272",
  "runId": "f2e14a95-8025-41de-a48f-860f8cfd2e68",
  "storedAtUtc": "2026-09-14T09:42:17Z",
  "sideA": null,
  "sideB": null,
  "aggregate": null,
  "agreement": null,
  "correctAnswers": {
    "mode": "Preserve",
    "computedAtUtc": "2026-09-14T09:42:17Z"
  },
  "document": {}
}

List the A/B tests a semantic model is on #

GET /v1/workspaces/{workspaceId}/semanticModels/{semanticModelId}/abTests

The A/B tests this semantic model is a side of, either side, newest first, as its page lists them. A semantic model with none answers an empty list, never a refusal. The list is never paged.

Scope: BIPixie.Read.All
Plan: Every plan
Counts against: Calls that read, 300 a minute

Parameters

Parameters for List the A/B tests a semantic model is on
Name	In	Type	Required	Description
`workspaceId`	path	string (uuid)	Yes	The id of the Fabric workspace.
`semanticModelId`	path	string (uuid)	Yes	The id of the semantic model in that workspace.

Responses

200. The tests.

Headers of the 200 response to List the A/B tests a semantic model is on
Header	Description
`x-ms-request-id`	The id of this request. Quote it to support.

The 200 response to List the A/B tests a semantic model is on, AbTestsOfItem
Field	Type	Always present	Description
`value`	array of AbTest	Yes
`abTestId`	string	Yes
`name`	string	Yes
`createdAtUtc`	string or null (date-time)	No
`sideA`	AbSide	Yes	One side of a test as it was set up.
`semanticModelId`	string or null	Yes
`workspaceId`	string or null	Yes
`semanticModelName`	string or null	No
`workspaceName`	string or null	No
`label`	string or null	No	The customer's short name for the side, when one was written.
`description`	string or null	No	The customer's own words about the side, when any were written.
`answeredBy`	AbAnsweredBy	Yes	What answers for a side, in the words a benchmark run is started with: the kind, the data agent when it is one and the workspace it lives in, and the saved AI provider when the side asks a generic AI agent.
`kind`	string or null	Yes	`DataAgent`, `BIPixieTestAgent`, `PowerBICopilot` or `GenericAgent`, as `agent.kind` on a benchmark run.
`dataAgentId`	string or null	No	The data agent that answers the side. Null for a `PowerBICopilot` or `GenericAgent` side, as are the other data agent fields.
`dataAgentName`	string or null	No
`dataAgentWorkspaceId`	string or null	No
`dataAgentWorkspaceName`	string or null	No	Where the data agent lives, when the test recorded it. A run's result reads it from the test, so it names the same workspace the test does.
`providerId`	string or null	No	The saved AI provider a generic AI agent side asks. On a run's result it is the one that answered the side, as its stored result recorded it.
`providerName`	string or null	No	The saved AI provider's name, when one was recorded.
`sideB`	AbSide	Yes	One side of a test as it was set up.
`semanticModelId`	string or null	Yes
`workspaceId`	string or null	Yes
`semanticModelName`	string or null	No
`workspaceName`	string or null	No
`label`	string or null	No	The customer's short name for the side, when one was written.
`description`	string or null	No	The customer's own words about the side, when any were written.
`answeredBy`	AbAnsweredBy	Yes	What answers for a side, in the words a benchmark run is started with: the kind, the data agent when it is one and the workspace it lives in, and the saved AI provider when the side asks a generic AI agent.
`kind`	string or null	Yes	`DataAgent`, `BIPixieTestAgent`, `PowerBICopilot` or `GenericAgent`, as `agent.kind` on a benchmark run.
`dataAgentId`	string or null	No	The data agent that answers the side. Null for a `PowerBICopilot` or `GenericAgent` side, as are the other data agent fields.
`dataAgentName`	string or null	No
`dataAgentWorkspaceId`	string or null	No
`dataAgentWorkspaceName`	string or null	No	Where the data agent lives, when the test recorded it. A run's result reads it from the test, so it names the same workspace the test does.
`providerId`	string or null	No	The saved AI provider a generic AI agent side asks. On a run's result it is the one that answered the side, as its stored result recorded it.
`providerName`	string or null	No	The saved AI provider's name, when one was recorded.
`questions`	object	Yes	Which questions the test asks, and where they came from.
`count`	integer	Yes
`questionSetHash`	string or null	No	The hash of the question set the test froze, which a run and the saved questions carry too.
`source`	object	Yes
`kind`	string	Yes	`Run` for a test made from a stored benchmark run, `Questions` for one made from the saved questions. One of `Run`, `Questions`.
`runId`	string or null	No	The stored benchmark run side A replays, when the test was made from one and that run still exists.
`runStoredAtUtc`	string or null (date-time)	No
`correctAnswers`	string	Yes	`Preserve` keeps the correct answers the test was made with on every run; `Update` computes them again from side A on each run. One of `Preserve`, `Update`.
`repeats`	integer	No	How many times each question is asked on each side.
`runCount`	integer	Yes
`latestRun`	one of AbLatestRun, null	No
`latestSucceededRunId`	string or null	No
`latestSucceededRunAggregate`	one of AbAggregate, null	No	The scores and response times of the newest run that succeeded, the run `latestSucceededRunId` names. Null when no run has succeeded.
`runGaps`	array of integer	No	Each succeeded run's gap, side B's correct answers minus side A's, oldest first, which is what the consistency line states.
`description`	string or null	No	The customer's own words about the test, when any were written.
`hypothesis`	object or null	No	What the customer expected, when they wrote it.
`text`	string	No
`kind`	string or null	No	`A`, `B` or `Same` when the hypothesis is one of the suggested sentences; null for the customer's own words.

Errors

Every refusal is an ErrorResponse with a stable errorCode. See Errors for what each HTTP status means and every code the API can return.

This is an example of a refusal. Every refusal has the same fields.

Example refusal

HTTP/1.1 404 Not Found
Content-Type: application/json
x-ms-request-id: cbf21b6d-b661-4b52-88cb-d79712e504fc

{
  "errorCode": "ItemNotFound",
  "message": "No item with this id is visible to the signed-in person in this workspace, or it is not of the type the route takes.",
  "isRetriable": false,
  "moreDetails": [],
  "relatedResource": {
    "resourceId": "d972b3f5-a2fa-47ec-94ff-0403a4d159fa",
    "resourceType": "SemanticModel"
  },
  "parameters": [
    {
      "name": "itemId",
      "value": "d972b3f5-a2fa-47ec-94ff-0403a4d159fa"
    }
  ],
  "requestId": "cbf21b6d-b661-4b52-88cb-d79712e504fc"
}

Example

Request

curl -i -X GET \
  -H "Authorization: Bearer $TOKEN" \
  "https://api.bipixie.com/v1/workspaces/$WORKSPACE_ID/semanticModels/$SEMANTIC_MODEL_ID/abTests"

An example of what the call answers. The values are fictional.

Response, 200

HTTP/1.1 200 OK
Content-Type: application/json
x-ms-request-id: cbf21b6d-b661-4b52-88cb-d79712e504fc

{
  "value": [
    {
      "abTestId": "7c016a91-7eea-4f0c-b327-e32f68230272",
      "name": "Contoso Sales",
      "createdAtUtc": "2026-06-03T14:12:00Z",
      "sideA": {
        "semanticModelId": "f1fdc461-3ca1-4702-ae48-afc3522409fc",
        "workspaceId": "e17c1b59-7a2e-491e-acdd-11434854bbb0",
        "semanticModelName": "Contoso Sales",
        "workspaceName": "Contoso Sales Analytics",
        "label": "With AI instructions",
        "description": "Sales questions answered with and without the AI instructions.",
        "answeredBy": {
          "kind": "DataAgent",
          "dataAgentId": "05dd87d1-624f-4ffa-ab88-ee231a75a904",
          "dataAgentName": "Contoso Sales Agent",
          "dataAgentWorkspaceId": "3a287f9c-63af-48a1-8c06-61c2e8095e5f",
          "dataAgentWorkspaceName": "Contoso Sales Analytics",
          "providerId": "b2df6be3-7769-4222-90ea-9fd98f4592e0",
          "providerName": "Azure OpenAI"
        }
      },
      "sideB": {
        "semanticModelId": "f1fdc461-3ca1-4702-ae48-afc3522409fc",
        "workspaceId": "e17c1b59-7a2e-491e-acdd-11434854bbb0",
        "semanticModelName": "Contoso Sales",
        "workspaceName": "Contoso Sales Analytics",
        "label": "With AI instructions",
        "description": "Sales questions answered with and without the AI instructions.",
        "answeredBy": {
          "kind": "DataAgent",
          "dataAgentId": "05dd87d1-624f-4ffa-ab88-ee231a75a904",
          "dataAgentName": "Contoso Sales Agent",
          "dataAgentWorkspaceId": "3a287f9c-63af-48a1-8c06-61c2e8095e5f",
          "dataAgentWorkspaceName": "Contoso Sales Analytics",
          "providerId": "b2df6be3-7769-4222-90ea-9fd98f4592e0",
          "providerName": "Azure OpenAI"
        }
      },
      "questions": {
        "count": 12,
        "questionSetHash": "12056ac56f6dd1de",
        "source": {
          "kind": "Run",
          "runId": "f2e14a95-8025-41de-a48f-860f8cfd2e68",
          "runStoredAtUtc": "2026-09-14T09:42:17Z"
        },
        "correctAnswers": "Preserve"
      },
      "repeats": 3,
      "runCount": 12,
      "latestRun": null,
      "latestSucceededRunId": "404dca33-3888-443a-8788-5329a0546348",
      "latestSucceededRunAggregate": null,
      "runGaps": [
        3
      ],
      "description": "Sales questions answered with and without the AI instructions.",
      "hypothesis": {
        "text": "The side with AI instructions answers more questions correctly.",
        "kind": "A"
      }
    }
  ]
}

List the A/B tests a data agent answers #

GET /v1/workspaces/{workspaceId}/dataAgents/{dataAgentId}/abTests

The A/B tests this data agent answers a side of, newest first, as its page lists them. A data agent that answers none answers an empty list, never a refusal. The list is never paged.

Scope: BIPixie.Read.All
Plan: Every plan
Counts against: Calls that read, 300 a minute

Parameters

Parameters for List the A/B tests a data agent answers
Name	In	Type	Required	Description
`workspaceId`	path	string (uuid)	Yes	The id of the Fabric workspace.
`dataAgentId`	path	string (uuid)	Yes	The id of the data agent in that workspace.

Responses

200. The tests.

Headers of the 200 response to List the A/B tests a data agent answers
Header	Description
`x-ms-request-id`	The id of this request. Quote it to support.

The 200 response to List the A/B tests a data agent answers, AbTestsOfItem
Field	Type	Always present	Description
`value`	array of AbTest	Yes
`abTestId`	string	Yes
`name`	string	Yes
`createdAtUtc`	string or null (date-time)	No
`sideA`	AbSide	Yes	One side of a test as it was set up.
`semanticModelId`	string or null	Yes
`workspaceId`	string or null	Yes
`semanticModelName`	string or null	No
`workspaceName`	string or null	No
`label`	string or null	No	The customer's short name for the side, when one was written.
`description`	string or null	No	The customer's own words about the side, when any were written.
`answeredBy`	AbAnsweredBy	Yes	What answers for a side, in the words a benchmark run is started with: the kind, the data agent when it is one and the workspace it lives in, and the saved AI provider when the side asks a generic AI agent.
`kind`	string or null	Yes	`DataAgent`, `BIPixieTestAgent`, `PowerBICopilot` or `GenericAgent`, as `agent.kind` on a benchmark run.
`dataAgentId`	string or null	No	The data agent that answers the side. Null for a `PowerBICopilot` or `GenericAgent` side, as are the other data agent fields.
`dataAgentName`	string or null	No
`dataAgentWorkspaceId`	string or null	No
`dataAgentWorkspaceName`	string or null	No	Where the data agent lives, when the test recorded it. A run's result reads it from the test, so it names the same workspace the test does.
`providerId`	string or null	No	The saved AI provider a generic AI agent side asks. On a run's result it is the one that answered the side, as its stored result recorded it.
`providerName`	string or null	No	The saved AI provider's name, when one was recorded.
`sideB`	AbSide	Yes	One side of a test as it was set up.
`semanticModelId`	string or null	Yes
`workspaceId`	string or null	Yes
`semanticModelName`	string or null	No
`workspaceName`	string or null	No
`label`	string or null	No	The customer's short name for the side, when one was written.
`description`	string or null	No	The customer's own words about the side, when any were written.
`answeredBy`	AbAnsweredBy	Yes	What answers for a side, in the words a benchmark run is started with: the kind, the data agent when it is one and the workspace it lives in, and the saved AI provider when the side asks a generic AI agent.
`kind`	string or null	Yes	`DataAgent`, `BIPixieTestAgent`, `PowerBICopilot` or `GenericAgent`, as `agent.kind` on a benchmark run.
`dataAgentId`	string or null	No	The data agent that answers the side. Null for a `PowerBICopilot` or `GenericAgent` side, as are the other data agent fields.
`dataAgentName`	string or null	No
`dataAgentWorkspaceId`	string or null	No
`dataAgentWorkspaceName`	string or null	No	Where the data agent lives, when the test recorded it. A run's result reads it from the test, so it names the same workspace the test does.
`providerId`	string or null	No	The saved AI provider a generic AI agent side asks. On a run's result it is the one that answered the side, as its stored result recorded it.
`providerName`	string or null	No	The saved AI provider's name, when one was recorded.
`questions`	object	Yes	Which questions the test asks, and where they came from.
`count`	integer	Yes
`questionSetHash`	string or null	No	The hash of the question set the test froze, which a run and the saved questions carry too.
`source`	object	Yes
`kind`	string	Yes	`Run` for a test made from a stored benchmark run, `Questions` for one made from the saved questions. One of `Run`, `Questions`.
`runId`	string or null	No	The stored benchmark run side A replays, when the test was made from one and that run still exists.
`runStoredAtUtc`	string or null (date-time)	No
`correctAnswers`	string	Yes	`Preserve` keeps the correct answers the test was made with on every run; `Update` computes them again from side A on each run. One of `Preserve`, `Update`.
`repeats`	integer	No	How many times each question is asked on each side.
`runCount`	integer	Yes
`latestRun`	one of AbLatestRun, null	No
`latestSucceededRunId`	string or null	No
`latestSucceededRunAggregate`	one of AbAggregate, null	No	The scores and response times of the newest run that succeeded, the run `latestSucceededRunId` names. Null when no run has succeeded.
`runGaps`	array of integer	No	Each succeeded run's gap, side B's correct answers minus side A's, oldest first, which is what the consistency line states.
`description`	string or null	No	The customer's own words about the test, when any were written.
`hypothesis`	object or null	No	What the customer expected, when they wrote it.
`text`	string	No
`kind`	string or null	No	`A`, `B` or `Same` when the hypothesis is one of the suggested sentences; null for the customer's own words.

Errors

Every refusal is an ErrorResponse with a stable errorCode. See Errors for what each HTTP status means and every code the API can return.

This is an example of a refusal. Every refusal has the same fields.

Example refusal

HTTP/1.1 404 Not Found
Content-Type: application/json
x-ms-request-id: cbf21b6d-b661-4b52-88cb-d79712e504fc

{
  "errorCode": "ItemNotFound",
  "message": "No item with this id is visible to the signed-in person in this workspace, or it is not of the type the route takes.",
  "isRetriable": false,
  "moreDetails": [],
  "relatedResource": {
    "resourceId": "d972b3f5-a2fa-47ec-94ff-0403a4d159fa",
    "resourceType": "SemanticModel"
  },
  "parameters": [
    {
      "name": "itemId",
      "value": "d972b3f5-a2fa-47ec-94ff-0403a4d159fa"
    }
  ],
  "requestId": "cbf21b6d-b661-4b52-88cb-d79712e504fc"
}

Example

Request

curl -i -X GET \
  -H "Authorization: Bearer $TOKEN" \
  "https://api.bipixie.com/v1/workspaces/$WORKSPACE_ID/dataAgents/$DATA_AGENT_ID/abTests"

An example of what the call answers. The values are fictional.

Response, 200

HTTP/1.1 200 OK
Content-Type: application/json
x-ms-request-id: cbf21b6d-b661-4b52-88cb-d79712e504fc

{
  "value": [
    {
      "abTestId": "7c016a91-7eea-4f0c-b327-e32f68230272",
      "name": "Contoso Sales",
      "createdAtUtc": "2026-06-03T14:12:00Z",
      "sideA": {
        "semanticModelId": "f1fdc461-3ca1-4702-ae48-afc3522409fc",
        "workspaceId": "e17c1b59-7a2e-491e-acdd-11434854bbb0",
        "semanticModelName": "Contoso Sales",
        "workspaceName": "Contoso Sales Analytics",
        "label": "With AI instructions",
        "description": "Sales questions answered with and without the AI instructions.",
        "answeredBy": {
          "kind": "DataAgent",
          "dataAgentId": "05dd87d1-624f-4ffa-ab88-ee231a75a904",
          "dataAgentName": "Contoso Sales Agent",
          "dataAgentWorkspaceId": "3a287f9c-63af-48a1-8c06-61c2e8095e5f",
          "dataAgentWorkspaceName": "Contoso Sales Analytics",
          "providerId": "b2df6be3-7769-4222-90ea-9fd98f4592e0",
          "providerName": "Azure OpenAI"
        }
      },
      "sideB": {
        "semanticModelId": "f1fdc461-3ca1-4702-ae48-afc3522409fc",
        "workspaceId": "e17c1b59-7a2e-491e-acdd-11434854bbb0",
        "semanticModelName": "Contoso Sales",
        "workspaceName": "Contoso Sales Analytics",
        "label": "With AI instructions",
        "description": "Sales questions answered with and without the AI instructions.",
        "answeredBy": {
          "kind": "DataAgent",
          "dataAgentId": "05dd87d1-624f-4ffa-ab88-ee231a75a904",
          "dataAgentName": "Contoso Sales Agent",
          "dataAgentWorkspaceId": "3a287f9c-63af-48a1-8c06-61c2e8095e5f",
          "dataAgentWorkspaceName": "Contoso Sales Analytics",
          "providerId": "b2df6be3-7769-4222-90ea-9fd98f4592e0",
          "providerName": "Azure OpenAI"
        }
      },
      "questions": {
        "count": 12,
        "questionSetHash": "12056ac56f6dd1de",
        "source": {
          "kind": "Run",
          "runId": "f2e14a95-8025-41de-a48f-860f8cfd2e68",
          "runStoredAtUtc": "2026-09-14T09:42:17Z"
        },
        "correctAnswers": "Preserve"
      },
      "repeats": 3,
      "runCount": 12,
      "latestRun": null,
      "latestSucceededRunId": "404dca33-3888-443a-8788-5329a0546348",
      "latestSucceededRunAggregate": null,
      "runGaps": [
        3
      ],
      "description": "Sales questions answered with and without the AI instructions.",
      "hypothesis": {
        "text": "The side with AI instructions answers more questions correctly.",
        "kind": "A"
      }
    }
  ]
}

A/B tests reference

List the account's A/B tests #

Parameters

Responses

Errors

Example

Create an A/B test and start its first run #

Request body

Responses

Errors

Example

Get one A/B test with its runs #

Parameters

Responses

Errors

Example

Change an A/B test's name, labels, descriptions or hypothesis #

Parameters

Request body

Responses

Errors

Example

Delete an A/B test #

Parameters

Responses

Errors

Example

Run an A/B test again #

Parameters

Request body

Responses

Errors

Example

Get one run of an A/B test #

Parameters

Responses

Errors

Example

Delete one run of an A/B test #

Parameters

Responses

Errors

Example

Get a run's comparison #

Parameters

Responses

Errors

Example

List the A/B tests a semantic model is on #

Parameters

Responses

Errors

Example

List the A/B tests a data agent answers #

Parameters

Responses

Errors

Example

What's next