Braintrust Data API v1.0.0
API specification for the backend data server. The API is hosted globally at https://api.braintrustdata.com or in your own environment. The v1 API is currently in preview mode and unstable until June 1, 2024. We may make backwards incompatible changes before then, as we learn from our users.
You can access the OpenAPI spec for this API at https://github.com/braintrustdata/braintrust-openapi.
Base URLs:
License: Apache 2.0
Authentication
- HTTP Authentication, scheme: bearer
Most Braintrust endpoints are authenticated by providing your API key as a headerAuthorization: Bearer [api_key]to your HTTP request. You can create an API key in the Braintrust organization settings page.
Projects
Create project
Code samples
const inputBody = JSON.stringify({
name: "string",
org_name: "string",
});
const headers = {
"Content-Type": "application/json",
Accept: "application/json",
Authorization: "Bearer {access-token}",
};
fetch("https://api.braintrustdata.com/v1/project", {
method: "POST",
body: inputBody,
headers: headers,
})
.then(function (res) {
return res.json();
})
.then(function (body) {
console.log(body);
});POST /v1/project
Create a new project. If there is an existing project with the same name as the one specified in the request, will return the existing project unmodified
Body parameter
{
"name": "string",
"org_name": "string"
}Parameters
| Name | In | Type | Required | Description |
|---|---|---|---|---|
| body | body | CreateProject | true | Any desired information about the new project object |
Example responses
200 Response
{
"id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
"org_id": "a40f5d1f-d889-42e9-94ea-b9b33585fc6b",
"name": "string",
"created": "2019-08-24T14:15:22Z",
"deleted_at": "2019-08-24T14:15:22Z",
"user_id": "a169451c-8525-4352-b8ca-070dd449a1a5"
}400 Response
"string"Responses
| Status | Meaning | Description | Schema |
|---|---|---|---|
| 200 | OK | Returns the new project object | Project |
| 400 | Bad Request | The request was unacceptable, often due to missing a required parameter | Inline |
| 401 | Unauthorized | No valid API key provided | Inline |
| 403 | Forbidden | The API key doesn’t have permissions to perform the request | Inline |
| 429 | Too Many Requests | Too many requests hit the API too quickly. We recommend an exponential backoff of your requests | Inline |
| 500 | Internal Server Error | Something went wrong on Braintrust's end. (These are rare.) | Inline |
Response Schema
Status Code 400
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 401
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 403
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 429
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 500
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
🔒
This endpoint requires authentication.
DEPRECATED. Create or replace project
Code samples
const inputBody = JSON.stringify({
name: "string",
org_name: "string",
});
const headers = {
"Content-Type": "application/json",
Accept: "application/json",
Authorization: "Bearer {access-token}",
};
fetch("https://api.braintrustdata.com/v1/project", {
method: "PUT",
body: inputBody,
headers: headers,
})
.then(function (res) {
return res.json();
})
.then(function (body) {
console.log(body);
});PUT /v1/project
NOTE: This operation is deprecated and will be removed in a future revision of the API. Create or replace a new project. If there is an existing project with the same name as the one specified in the request, will return the existing project unmodified, will replace the existing project with the provided fields
Body parameter
{
"name": "string",
"org_name": "string"
}Parameters
| Name | In | Type | Required | Description |
|---|---|---|---|---|
| body | body | CreateProject | true | Any desired information about the new project object |
Example responses
200 Response
{
"id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
"org_id": "a40f5d1f-d889-42e9-94ea-b9b33585fc6b",
"name": "string",
"created": "2019-08-24T14:15:22Z",
"deleted_at": "2019-08-24T14:15:22Z",
"user_id": "a169451c-8525-4352-b8ca-070dd449a1a5"
}400 Response
"string"Responses
| Status | Meaning | Description | Schema |
|---|---|---|---|
| 200 | OK | Returns the new project object | Project |
| 400 | Bad Request | The request was unacceptable, often due to missing a required parameter | Inline |
| 401 | Unauthorized | No valid API key provided | Inline |
| 403 | Forbidden | The API key doesn’t have permissions to perform the request | Inline |
| 429 | Too Many Requests | Too many requests hit the API too quickly. We recommend an exponential backoff of your requests | Inline |
| 500 | Internal Server Error | Something went wrong on Braintrust's end. (These are rare.) | Inline |
Response Schema
Status Code 400
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 401
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 403
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 429
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 500
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
🔒
This endpoint requires authentication.
List projects
Code samples
const headers = {
Accept: "application/json",
Authorization: "Bearer {access-token}",
};
fetch("https://api.braintrustdata.com/v1/project", {
method: "GET",
headers: headers,
})
.then(function (res) {
return res.json();
})
.then(function (body) {
console.log(body);
});GET /v1/project
List out all projects. The projects are sorted by creation date, with the most recently-created projects coming first
Parameters
| Name | In | Type | Required | Description |
|---|---|---|---|---|
| limit | query | AppLimitParam | false | Limit the number of objects to return |
| starting_after | query | StartingAfter | false | Pagination cursor id. |
| ending_before | query | EndingBefore | false | Pagination cursor id. |
| project_name | query | ProjectName | false | Name of the project to search for |
| org_name | query | OrgName | false | Filter search results to within a particular organization |
| ids | query | Ids | false | Filter search results to a particular set of object IDs. To specify a list of IDs, include the query param multiple times |
Detailed descriptions
starting_after: Pagination cursor id.
For example, if the final item in the last page you fetched had an id of foo, pass starting_after=foo to fetch the next page. Note: you may only pass one of starting_after and ending_before
ending_before: Pagination cursor id.
For example, if the initial item in the last page you fetched had an id of foo, pass ending_before=foo to fetch the previous page. Note: you may only pass one of starting_after and ending_before
Example responses
200 Response
{
"objects": [
{
"id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
"org_id": "a40f5d1f-d889-42e9-94ea-b9b33585fc6b",
"name": "string",
"created": "2019-08-24T14:15:22Z",
"deleted_at": "2019-08-24T14:15:22Z",
"user_id": "a169451c-8525-4352-b8ca-070dd449a1a5"
}
]
}400 Response
"string"Responses
| Status | Meaning | Description | Schema |
|---|---|---|---|
| 200 | OK | Returns a list of project objects | Inline |
| 400 | Bad Request | The request was unacceptable, often due to missing a required parameter | Inline |
| 401 | Unauthorized | No valid API key provided | Inline |
| 403 | Forbidden | The API key doesn’t have permissions to perform the request | Inline |
| 429 | Too Many Requests | Too many requests hit the API too quickly. We recommend an exponential backoff of your requests | Inline |
| 500 | Internal Server Error | Something went wrong on Braintrust's end. (These are rare.) | Inline |
Response Schema
Status Code 200
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| » objects | [Project] | true | none | A list of project objects |
| »» id | string(uuid) | true | none | Unique identifier for the project |
| »» org_id | string(uuid) | true | none | Unique id for the organization that the project belongs under |
| »» name | string | true | none | Name of the project |
| »» created | string(date-time)¦null | false | none | Date of project creation |
| »» deleted_at | string(date-time)¦null | false | none | Date of project deletion, or null if the project is still active |
| »» user_id | string(uuid)¦null | false | none | Identifies the user who created the project |
Status Code 400
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 401
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 403
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 429
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 500
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
🔒
This endpoint requires authentication.
Get project
Code samples
const headers = {
Accept: "application/json",
Authorization: "Bearer {access-token}",
};
fetch("https://api.braintrustdata.com/v1/project/{project_id}", {
method: "GET",
headers: headers,
})
.then(function (res) {
return res.json();
})
.then(function (body) {
console.log(body);
});GET /v1/project/{project_id}
Get a project object by its id
Parameters
| Name | In | Type | Required | Description |
|---|---|---|---|---|
| project_id | path | ProjectId | true | Project id |
Example responses
200 Response
{
"id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
"org_id": "a40f5d1f-d889-42e9-94ea-b9b33585fc6b",
"name": "string",
"created": "2019-08-24T14:15:22Z",
"deleted_at": "2019-08-24T14:15:22Z",
"user_id": "a169451c-8525-4352-b8ca-070dd449a1a5"
}400 Response
"string"Responses
| Status | Meaning | Description | Schema |
|---|---|---|---|
| 200 | OK | Returns the project object | Project |
| 400 | Bad Request | The request was unacceptable, often due to missing a required parameter | Inline |
| 401 | Unauthorized | No valid API key provided | Inline |
| 403 | Forbidden | The API key doesn’t have permissions to perform the request | Inline |
| 429 | Too Many Requests | Too many requests hit the API too quickly. We recommend an exponential backoff of your requests | Inline |
| 500 | Internal Server Error | Something went wrong on Braintrust's end. (These are rare.) | Inline |
Response Schema
Status Code 400
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 401
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 403
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 429
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 500
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
🔒
This endpoint requires authentication.
Partially update project
Code samples
const inputBody = JSON.stringify({
name: "string",
});
const headers = {
"Content-Type": "application/json",
Accept: "application/json",
Authorization: "Bearer {access-token}",
};
fetch("https://api.braintrustdata.com/v1/project/{project_id}", {
method: "PATCH",
body: inputBody,
headers: headers,
})
.then(function (res) {
return res.json();
})
.then(function (body) {
console.log(body);
});PATCH /v1/project/{project_id}
Partially update a project object. Specify the fields to update in the payload. Any object-type fields will be deep-merged with existing content. Currently we do not support removing fields or setting them to null.
Body parameter
{
"name": "string"
}Parameters
| Name | In | Type | Required | Description |
|---|---|---|---|---|
| project_id | path | ProjectId | true | Project id |
| body | body | PatchProject | false | Fields to update |
Example responses
200 Response
{
"id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
"org_id": "a40f5d1f-d889-42e9-94ea-b9b33585fc6b",
"name": "string",
"created": "2019-08-24T14:15:22Z",
"deleted_at": "2019-08-24T14:15:22Z",
"user_id": "a169451c-8525-4352-b8ca-070dd449a1a5"
}400 Response
"string"Responses
| Status | Meaning | Description | Schema |
|---|---|---|---|
| 200 | OK | Returns the project object | Project |
| 400 | Bad Request | The request was unacceptable, often due to missing a required parameter | Inline |
| 401 | Unauthorized | No valid API key provided | Inline |
| 403 | Forbidden | The API key doesn’t have permissions to perform the request | Inline |
| 429 | Too Many Requests | Too many requests hit the API too quickly. We recommend an exponential backoff of your requests | Inline |
| 500 | Internal Server Error | Something went wrong on Braintrust's end. (These are rare.) | Inline |
Response Schema
Status Code 400
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 401
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 403
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 429
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 500
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
🔒
This endpoint requires authentication.
Delete project
Code samples
const headers = {
Accept: "application/json",
Authorization: "Bearer {access-token}",
};
fetch("https://api.braintrustdata.com/v1/project/{project_id}", {
method: "DELETE",
headers: headers,
})
.then(function (res) {
return res.json();
})
.then(function (body) {
console.log(body);
});DELETE /v1/project/{project_id}
Delete a project object by its id
Parameters
| Name | In | Type | Required | Description |
|---|---|---|---|---|
| project_id | path | ProjectId | true | Project id |
Example responses
200 Response
{
"id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
"org_id": "a40f5d1f-d889-42e9-94ea-b9b33585fc6b",
"name": "string",
"created": "2019-08-24T14:15:22Z",
"deleted_at": "2019-08-24T14:15:22Z",
"user_id": "a169451c-8525-4352-b8ca-070dd449a1a5"
}400 Response
"string"Responses
| Status | Meaning | Description | Schema |
|---|---|---|---|
| 200 | OK | Returns the deleted project object | Project |
| 400 | Bad Request | The request was unacceptable, often due to missing a required parameter | Inline |
| 401 | Unauthorized | No valid API key provided | Inline |
| 403 | Forbidden | The API key doesn’t have permissions to perform the request | Inline |
| 429 | Too Many Requests | Too many requests hit the API too quickly. We recommend an exponential backoff of your requests | Inline |
| 500 | Internal Server Error | Something went wrong on Braintrust's end. (These are rare.) | Inline |
Response Schema
Status Code 400
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 401
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 403
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 429
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 500
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
🔒
This endpoint requires authentication.
Insert project logs events
Code samples
const inputBody = JSON.stringify({
events: [
{
input: null,
output: null,
expected: null,
scores: {
property1: 1,
property2: 1,
},
metadata: {
property1: null,
property2: null,
},
tags: ["string"],
metrics: {
start: 0,
end: 0,
prompt_tokens: 0,
completion_tokens: 0,
tokens: 0,
property1: null,
property2: null,
},
context: {
caller_functionname: "string",
caller_filename: "string",
caller_lineno: 0,
property1: null,
property2: null,
},
span_attributes: {
name: "string",
type: "llm",
property1: null,
property2: null,
},
id: "string",
_object_delete: true,
_is_merge: false,
_parent_id: "string",
},
],
});
const headers = {
"Content-Type": "application/json",
Accept: "application/json",
Authorization: "Bearer {access-token}",
};
fetch("https://api.braintrustdata.com/v1/project_logs/{project_id}/insert", {
method: "POST",
body: inputBody,
headers: headers,
})
.then(function (res) {
return res.json();
})
.then(function (body) {
console.log(body);
});POST /v1/project_logs/{project_id}/insert
Insert a set of events into the project logs
Body parameter
{
"events": [
{
"input": null,
"output": null,
"expected": null,
"scores": {
"property1": 1,
"property2": 1
},
"metadata": {
"property1": null,
"property2": null
},
"tags": ["string"],
"metrics": {
"start": 0,
"end": 0,
"prompt_tokens": 0,
"completion_tokens": 0,
"tokens": 0,
"property1": null,
"property2": null
},
"context": {
"caller_functionname": "string",
"caller_filename": "string",
"caller_lineno": 0,
"property1": null,
"property2": null
},
"span_attributes": {
"name": "string",
"type": "llm",
"property1": null,
"property2": null
},
"id": "string",
"_object_delete": true,
"_is_merge": false,
"_parent_id": "string"
}
]
}Parameters
| Name | In | Type | Required | Description |
|---|---|---|---|---|
| project_id | path | ProjectId | true | Project id |
| body | body | InsertProjectLogsEventRequest | true | An array of project logs events to insert |
Example responses
200 Response
{
"row_ids": ["string"]
}400 Response
"string"Responses
| Status | Meaning | Description | Schema |
|---|---|---|---|
| 200 | OK | Returns the inserted row ids | InsertEventsResponse |
| 400 | Bad Request | The request was unacceptable, often due to missing a required parameter | Inline |
| 401 | Unauthorized | No valid API key provided | Inline |
| 403 | Forbidden | The API key doesn’t have permissions to perform the request | Inline |
| 429 | Too Many Requests | Too many requests hit the API too quickly. We recommend an exponential backoff of your requests | Inline |
| 500 | Internal Server Error | Something went wrong on Braintrust's end. (These are rare.) | Inline |
Response Schema
Status Code 400
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 401
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 403
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 429
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 500
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
🔒
This endpoint requires authentication.
Fetch project logs (POST form)
Code samples
const inputBody = JSON.stringify({
limit: 0,
cursor: "string",
max_xact_id: "string",
max_root_span_id: "string",
filters: [
{
type: "path_lookup",
path: ["string"],
value: null,
},
],
version: "string",
});
const headers = {
"Content-Type": "application/json",
Accept: "application/json",
Authorization: "Bearer {access-token}",
};
fetch("https://api.braintrustdata.com/v1/project_logs/{project_id}/fetch", {
method: "POST",
body: inputBody,
headers: headers,
})
.then(function (res) {
return res.json();
})
.then(function (body) {
console.log(body);
});POST /v1/project_logs/{project_id}/fetch
Fetch the events in a project logs. Equivalent to the GET form of the same path, but with the parameters in the request body rather than in the URL query
Body parameter
{
"limit": 0,
"cursor": "string",
"max_xact_id": "string",
"max_root_span_id": "string",
"filters": [
{
"type": "path_lookup",
"path": ["string"],
"value": null
}
],
"version": "string"
}Parameters
| Name | In | Type | Required | Description |
|---|---|---|---|---|
| project_id | path | ProjectId | true | Project id |
| body | body | FetchEventsRequest | false | Filters for the fetch query |
Example responses
200 Response
{
"events": [
{
"id": "string",
"_xact_id": "string",
"created": "2019-08-24T14:15:22Z",
"org_id": "a40f5d1f-d889-42e9-94ea-b9b33585fc6b",
"project_id": "405d8375-3514-403b-8c43-83ae74cfe0e9",
"log_id": "g",
"input": null,
"output": null,
"expected": null,
"scores": {
"property1": 1,
"property2": 1
},
"metadata": {
"property1": null,
"property2": null
},
"tags": ["string"],
"metrics": {
"start": 0,
"end": 0,
"prompt_tokens": 0,
"completion_tokens": 0,
"tokens": 0,
"property1": null,
"property2": null
},
"context": {
"caller_functionname": "string",
"caller_filename": "string",
"caller_lineno": 0,
"property1": null,
"property2": null
},
"span_id": "string",
"span_parents": ["string"],
"root_span_id": "string",
"span_attributes": {
"name": "string",
"type": "llm",
"property1": null,
"property2": null
}
}
],
"cursor": "string"
}400 Response
"string"Responses
| Status | Meaning | Description | Schema |
|---|---|---|---|
| 200 | OK | Returns the fetched rows | FetchProjectLogsEventsResponse |
| 400 | Bad Request | The request was unacceptable, often due to missing a required parameter | Inline |
| 401 | Unauthorized | No valid API key provided | Inline |
| 403 | Forbidden | The API key doesn’t have permissions to perform the request | Inline |
| 429 | Too Many Requests | Too many requests hit the API too quickly. We recommend an exponential backoff of your requests | Inline |
| 500 | Internal Server Error | Something went wrong on Braintrust's end. (These are rare.) | Inline |
Response Schema
Status Code 400
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 401
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 403
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 429
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 500
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
🔒
This endpoint requires authentication.
Fetch project logs (GET form)
Code samples
const headers = {
Accept: "application/json",
Authorization: "Bearer {access-token}",
};
fetch("https://api.braintrustdata.com/v1/project_logs/{project_id}/fetch", {
method: "GET",
headers: headers,
})
.then(function (res) {
return res.json();
})
.then(function (body) {
console.log(body);
});GET /v1/project_logs/{project_id}/fetch
Fetch the events in a project logs. Equivalent to the POST form of the same path, but with the parameters in the URL query rather than in the request body
Parameters
| Name | In | Type | Required | Description |
|---|---|---|---|---|
| project_id | path | ProjectId | true | Project id |
| limit | query | FetchLimitParam | false | limit the number of traces fetched |
| max_xact_id | query | MaxXactId | false | DEPRECATION NOTICE: The manually-constructed pagination cursor is deprecated in favor of the explicit 'cursor' returned by object fetch requests. Please prefer the 'cursor' argument going forwards. |
| max_root_span_id | query | MaxRootSpanId | false | DEPRECATION NOTICE: The manually-constructed pagination cursor is deprecated in favor of the explicit 'cursor' returned by object fetch requests. Please prefer the 'cursor' argument going forwards. |
| version | query | any | false | Retrieve prompt at a specific version. |
Detailed descriptions
limit: limit the number of traces fetched
Fetch queries may be paginated if the total result size is expected to be large (e.g. project_logs which accumulate over a long time). Note that fetch queries only support pagination in descending time order (from latest to earliest _xact_id. Furthermore, later pages may return rows which showed up in earlier pages, except with an earlier _xact_id. This happens because pagination occurs over the whole version history of the event log. You will most likely want to exclude any such duplicate, outdated rows (by id) from your combined result set.
The limit parameter controls the number of full traces to return. So you may end up with more individual rows than the specified limit if you are fetching events containing traces.
max_xact_id: DEPRECATION NOTICE: The manually-constructed pagination cursor is deprecated in favor of the explicit 'cursor' returned by object fetch requests. Please prefer the 'cursor' argument going forwards.
Together, max_xact_id and max_root_span_id form a pagination cursor
Since a paginated fetch query returns results in order from latest to earliest, the cursor for the next page can be found as the row with the minimum (earliest) value of the tuple (_xact_id, root_span_id). See the documentation of limit for an overview of paginating fetch queries.
max_root_span_id: DEPRECATION NOTICE: The manually-constructed pagination cursor is deprecated in favor of the explicit 'cursor' returned by object fetch requests. Please prefer the 'cursor' argument going forwards.
Together, max_xact_id and max_root_span_id form a pagination cursor
Since a paginated fetch query returns results in order from latest to earliest, the cursor for the next page can be found as the row with the minimum (earliest) value of the tuple (_xact_id, root_span_id). See the documentation of limit for an overview of paginating fetch queries.
version: Retrieve prompt at a specific version.
The version id can either be a transaction id (e.g. '1000192656880881099') or a version identifier (e.g. '81cd05ee665fdfb3').
Example responses
200 Response
{
"events": [
{
"id": "string",
"_xact_id": "string",
"created": "2019-08-24T14:15:22Z",
"org_id": "a40f5d1f-d889-42e9-94ea-b9b33585fc6b",
"project_id": "405d8375-3514-403b-8c43-83ae74cfe0e9",
"log_id": "g",
"input": null,
"output": null,
"expected": null,
"scores": {
"property1": 1,
"property2": 1
},
"metadata": {
"property1": null,
"property2": null
},
"tags": ["string"],
"metrics": {
"start": 0,
"end": 0,
"prompt_tokens": 0,
"completion_tokens": 0,
"tokens": 0,
"property1": null,
"property2": null
},
"context": {
"caller_functionname": "string",
"caller_filename": "string",
"caller_lineno": 0,
"property1": null,
"property2": null
},
"span_id": "string",
"span_parents": ["string"],
"root_span_id": "string",
"span_attributes": {
"name": "string",
"type": "llm",
"property1": null,
"property2": null
}
}
],
"cursor": "string"
}400 Response
"string"Responses
| Status | Meaning | Description | Schema |
|---|---|---|---|
| 200 | OK | Returns the fetched rows | FetchProjectLogsEventsResponse |
| 400 | Bad Request | The request was unacceptable, often due to missing a required parameter | Inline |
| 401 | Unauthorized | No valid API key provided | Inline |
| 403 | Forbidden | The API key doesn’t have permissions to perform the request | Inline |
| 429 | Too Many Requests | Too many requests hit the API too quickly. We recommend an exponential backoff of your requests | Inline |
| 500 | Internal Server Error | Something went wrong on Braintrust's end. (These are rare.) | Inline |
Response Schema
Status Code 400
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 401
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 403
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 429
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 500
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
🔒
This endpoint requires authentication.
Feedback for project logs events
Code samples
const inputBody = JSON.stringify({
feedback: [
{
id: "string",
scores: {
property1: 1,
property2: 1,
},
expected: null,
comment: "string",
metadata: {
property1: null,
property2: null,
},
source: "app",
},
],
});
const headers = {
"Content-Type": "application/json",
Accept: "text/plain",
Authorization: "Bearer {access-token}",
};
fetch("https://api.braintrustdata.com/v1/project_logs/{project_id}/feedback", {
method: "POST",
body: inputBody,
headers: headers,
})
.then(function (res) {
return res.json();
})
.then(function (body) {
console.log(body);
});POST /v1/project_logs/{project_id}/feedback
Log feedback for a set of project logs events
Body parameter
{
"feedback": [
{
"id": "string",
"scores": {
"property1": 1,
"property2": 1
},
"expected": null,
"comment": "string",
"metadata": {
"property1": null,
"property2": null
},
"source": "app"
}
]
}Parameters
| Name | In | Type | Required | Description |
|---|---|---|---|---|
| project_id | path | ProjectId | true | Project id |
| body | body | FeedbackProjectLogsEventRequest | true | An array of feedback objects |
Example responses
400 Response
"string"nullResponses
| Status | Meaning | Description | Schema |
|---|---|---|---|
| 200 | OK | No return value | None |
| 400 | Bad Request | The request was unacceptable, often due to missing a required parameter | Inline |
| 401 | Unauthorized | No valid API key provided | Inline |
| 403 | Forbidden | The API key doesn’t have permissions to perform the request | Inline |
| 429 | Too Many Requests | Too many requests hit the API too quickly. We recommend an exponential backoff of your requests | Inline |
| 500 | Internal Server Error | Something went wrong on Braintrust's end. (These are rare.) | Inline |
Response Schema
Status Code 400
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 401
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 403
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 429
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 500
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
🔒
This endpoint requires authentication.
Experiments
Create experiment
Code samples
const inputBody = JSON.stringify({
project_id: "405d8375-3514-403b-8c43-83ae74cfe0e9",
name: "string",
description: "string",
repo_info: {
commit: "string",
branch: "string",
tag: "string",
dirty: true,
author_name: "string",
author_email: "string",
commit_message: "string",
commit_time: "string",
git_diff: "string",
},
base_exp_id: "4838cee2-a665-4545-aa9f-483678c01a6b",
dataset_id: "8c4c51f1-f6f3-43bc-b65d-7415e8ef22c0",
dataset_version: "string",
public: true,
metadata: {
property1: null,
property2: null,
},
ensure_new: true,
});
const headers = {
"Content-Type": "application/json",
Accept: "application/json",
Authorization: "Bearer {access-token}",
};
fetch("https://api.braintrustdata.com/v1/experiment", {
method: "POST",
body: inputBody,
headers: headers,
})
.then(function (res) {
return res.json();
})
.then(function (body) {
console.log(body);
});POST /v1/experiment
Create a new experiment. If there is an existing experiment in the project with the same name as the one specified in the request, will return the existing experiment unmodified
Body parameter
{
"project_id": "405d8375-3514-403b-8c43-83ae74cfe0e9",
"name": "string",
"description": "string",
"repo_info": {
"commit": "string",
"branch": "string",
"tag": "string",
"dirty": true,
"author_name": "string",
"author_email": "string",
"commit_message": "string",
"commit_time": "string",
"git_diff": "string"
},
"base_exp_id": "4838cee2-a665-4545-aa9f-483678c01a6b",
"dataset_id": "8c4c51f1-f6f3-43bc-b65d-7415e8ef22c0",
"dataset_version": "string",
"public": true,
"metadata": {
"property1": null,
"property2": null
},
"ensure_new": true
}Parameters
| Name | In | Type | Required | Description |
|---|---|---|---|---|
| body | body | CreateExperiment | true | Any desired information about the new experiment object |
Example responses
200 Response
{
"id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
"project_id": "405d8375-3514-403b-8c43-83ae74cfe0e9",
"name": "string",
"description": "string",
"created": "2019-08-24T14:15:22Z",
"repo_info": {
"commit": "string",
"branch": "string",
"tag": "string",
"dirty": true,
"author_name": "string",
"author_email": "string",
"commit_message": "string",
"commit_time": "string",
"git_diff": "string"
},
"commit": "string",
"base_exp_id": "4838cee2-a665-4545-aa9f-483678c01a6b",
"deleted_at": "2019-08-24T14:15:22Z",
"dataset_id": "8c4c51f1-f6f3-43bc-b65d-7415e8ef22c0",
"dataset_version": "string",
"public": true,
"user_id": "a169451c-8525-4352-b8ca-070dd449a1a5",
"metadata": {
"property1": null,
"property2": null
}
}400 Response
"string"Responses
| Status | Meaning | Description | Schema |
|---|---|---|---|
| 200 | OK | Returns the new experiment object | Experiment |
| 400 | Bad Request | The request was unacceptable, often due to missing a required parameter | Inline |
| 401 | Unauthorized | No valid API key provided | Inline |
| 403 | Forbidden | The API key doesn’t have permissions to perform the request | Inline |
| 429 | Too Many Requests | Too many requests hit the API too quickly. We recommend an exponential backoff of your requests | Inline |
| 500 | Internal Server Error | Something went wrong on Braintrust's end. (These are rare.) | Inline |
Response Schema
Status Code 400
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 401
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 403
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 429
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 500
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
🔒
This endpoint requires authentication.
DEPRECATED. Create or replace experiment
Code samples
const inputBody = JSON.stringify({
project_id: "405d8375-3514-403b-8c43-83ae74cfe0e9",
name: "string",
description: "string",
repo_info: {
commit: "string",
branch: "string",
tag: "string",
dirty: true,
author_name: "string",
author_email: "string",
commit_message: "string",
commit_time: "string",
git_diff: "string",
},
base_exp_id: "4838cee2-a665-4545-aa9f-483678c01a6b",
dataset_id: "8c4c51f1-f6f3-43bc-b65d-7415e8ef22c0",
dataset_version: "string",
public: true,
metadata: {
property1: null,
property2: null,
},
ensure_new: true,
});
const headers = {
"Content-Type": "application/json",
Accept: "application/json",
Authorization: "Bearer {access-token}",
};
fetch("https://api.braintrustdata.com/v1/experiment", {
method: "PUT",
body: inputBody,
headers: headers,
})
.then(function (res) {
return res.json();
})
.then(function (body) {
console.log(body);
});PUT /v1/experiment
NOTE: This operation is deprecated and will be removed in a future revision of the API. Create or replace a new experiment. If there is an existing experiment in the project with the same name as the one specified in the request, will return the existing experiment unmodified, will replace the existing experiment with the provided fields
Body parameter
{
"project_id": "405d8375-3514-403b-8c43-83ae74cfe0e9",
"name": "string",
"description": "string",
"repo_info": {
"commit": "string",
"branch": "string",
"tag": "string",
"dirty": true,
"author_name": "string",
"author_email": "string",
"commit_message": "string",
"commit_time": "string",
"git_diff": "string"
},
"base_exp_id": "4838cee2-a665-4545-aa9f-483678c01a6b",
"dataset_id": "8c4c51f1-f6f3-43bc-b65d-7415e8ef22c0",
"dataset_version": "string",
"public": true,
"metadata": {
"property1": null,
"property2": null
},
"ensure_new": true
}Parameters
| Name | In | Type | Required | Description |
|---|---|---|---|---|
| body | body | CreateExperiment | true | Any desired information about the new experiment object |
Example responses
200 Response
{
"id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
"project_id": "405d8375-3514-403b-8c43-83ae74cfe0e9",
"name": "string",
"description": "string",
"created": "2019-08-24T14:15:22Z",
"repo_info": {
"commit": "string",
"branch": "string",
"tag": "string",
"dirty": true,
"author_name": "string",
"author_email": "string",
"commit_message": "string",
"commit_time": "string",
"git_diff": "string"
},
"commit": "string",
"base_exp_id": "4838cee2-a665-4545-aa9f-483678c01a6b",
"deleted_at": "2019-08-24T14:15:22Z",
"dataset_id": "8c4c51f1-f6f3-43bc-b65d-7415e8ef22c0",
"dataset_version": "string",
"public": true,
"user_id": "a169451c-8525-4352-b8ca-070dd449a1a5",
"metadata": {
"property1": null,
"property2": null
}
}400 Response
"string"Responses
| Status | Meaning | Description | Schema |
|---|---|---|---|
| 200 | OK | Returns the new experiment object | Experiment |
| 400 | Bad Request | The request was unacceptable, often due to missing a required parameter | Inline |
| 401 | Unauthorized | No valid API key provided | Inline |
| 403 | Forbidden | The API key doesn’t have permissions to perform the request | Inline |
| 429 | Too Many Requests | Too many requests hit the API too quickly. We recommend an exponential backoff of your requests | Inline |
| 500 | Internal Server Error | Something went wrong on Braintrust's end. (These are rare.) | Inline |
Response Schema
Status Code 400
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 401
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 403
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 429
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 500
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
🔒
This endpoint requires authentication.
List experiments
Code samples
const headers = {
Accept: "application/json",
Authorization: "Bearer {access-token}",
};
fetch("https://api.braintrustdata.com/v1/experiment", {
method: "GET",
headers: headers,
})
.then(function (res) {
return res.json();
})
.then(function (body) {
console.log(body);
});GET /v1/experiment
List out all experiments. The experiments are sorted by creation date, with the most recently-created experiments coming first
Parameters
| Name | In | Type | Required | Description |
|---|---|---|---|---|
| limit | query | AppLimitParam | false | Limit the number of objects to return |
| starting_after | query | StartingAfter | false | Pagination cursor id. |
| ending_before | query | EndingBefore | false | Pagination cursor id. |
| experiment_name | query | ExperimentName | false | Name of the experiment to search for |
| project_name | query | ProjectName | false | Name of the project to search for |
| org_name | query | OrgName | false | Filter search results to within a particular organization |
| ids | query | Ids | false | Filter search results to a particular set of object IDs. To specify a list of IDs, include the query param multiple times |
Detailed descriptions
starting_after: Pagination cursor id.
For example, if the final item in the last page you fetched had an id of foo, pass starting_after=foo to fetch the next page. Note: you may only pass one of starting_after and ending_before
ending_before: Pagination cursor id.
For example, if the initial item in the last page you fetched had an id of foo, pass ending_before=foo to fetch the previous page. Note: you may only pass one of starting_after and ending_before
Example responses
200 Response
{
"objects": [
{
"id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
"project_id": "405d8375-3514-403b-8c43-83ae74cfe0e9",
"name": "string",
"description": "string",
"created": "2019-08-24T14:15:22Z",
"repo_info": {
"commit": "string",
"branch": "string",
"tag": "string",
"dirty": true,
"author_name": "string",
"author_email": "string",
"commit_message": "string",
"commit_time": "string",
"git_diff": "string"
},
"commit": "string",
"base_exp_id": "4838cee2-a665-4545-aa9f-483678c01a6b",
"deleted_at": "2019-08-24T14:15:22Z",
"dataset_id": "8c4c51f1-f6f3-43bc-b65d-7415e8ef22c0",
"dataset_version": "string",
"public": true,
"user_id": "a169451c-8525-4352-b8ca-070dd449a1a5",
"metadata": {
"property1": null,
"property2": null
}
}
]
}400 Response
"string"Responses
| Status | Meaning | Description | Schema |
|---|---|---|---|
| 200 | OK | Returns a list of experiment objects | Inline |
| 400 | Bad Request | The request was unacceptable, often due to missing a required parameter | Inline |
| 401 | Unauthorized | No valid API key provided | Inline |
| 403 | Forbidden | The API key doesn’t have permissions to perform the request | Inline |
| 429 | Too Many Requests | Too many requests hit the API too quickly. We recommend an exponential backoff of your requests | Inline |
| 500 | Internal Server Error | Something went wrong on Braintrust's end. (These are rare.) | Inline |
Response Schema
Status Code 200
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| » objects | [Experiment] | true | none | A list of experiment objects |
| »» id | string(uuid) | true | none | Unique identifier for the experiment |
| »» project_id | string(uuid) | true | none | Unique identifier for the project that the experiment belongs under |
| »» name | string | true | none | Name of the experiment. Within a project, experiment names are unique |
| »» description | string¦null | false | none | Textual description of the experiment |
| »» created | string(date-time)¦null | false | none | Date of experiment creation |
| »» repo_info | RepoInfo¦null | false | none | Metadata about the state of the repo when the experiment was created |
| »»» commit | string¦null | false | none | SHA of most recent commit |
| »»» branch | string¦null | false | none | Name of the branch the most recent commit belongs to |
| »»» tag | string¦null | false | none | Name of the tag on the most recent commit |
| »»» dirty | boolean¦null | false | none | Whether or not the repo had uncommitted changes when snapshotted |
| »»» author_name | string¦null | false | none | Name of the author of the most recent commit |
| »»» author_email | string¦null | false | none | Email of the author of the most recent commit |
| »»» commit_message | string¦null | false | none | Most recent commit message |
| »»» commit_time | string¦null | false | none | Time of the most recent commit |
| »»» git_diff | string¦null | false | none | If the repo was dirty when run, this includes the diff between the current state of the repo and the most recent commit. |
| »» commit | string¦null | false | none | Commit, taken directly from repo_info.commit |
| »» base_exp_id | string(uuid)¦null | false | none | Id of default base experiment to compare against when viewing this experiment |
| »» deleted_at | string(date-time)¦null | false | none | Date of experiment deletion, or null if the experiment is still active |
| »» dataset_id | string(uuid)¦null | false | none | Identifier of the linked dataset, or null if the experiment is not linked to a dataset |
| »» dataset_version | string¦null | false | none | Version number of the linked dataset the experiment was run against. This can be used to reproduce the experiment after the dataset has been modified. |
| »» public | boolean | true | none | Whether or not the experiment is public. Public experiments can be viewed by anybody inside or outside the organization |
| »» user_id | string(uuid)¦null | false | none | Identifies the user who created the experiment |
| »» metadata | object¦null | false | none | User-controlled metadata about the experiment |
| »»» additionalProperties | any | false | none | none |
Status Code 400
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 401
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 403
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 429
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 500
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
🔒
This endpoint requires authentication.
Get experiment
Code samples
const headers = {
Accept: "application/json",
Authorization: "Bearer {access-token}",
};
fetch("https://api.braintrustdata.com/v1/experiment/{experiment_id}", {
method: "GET",
headers: headers,
})
.then(function (res) {
return res.json();
})
.then(function (body) {
console.log(body);
});GET /v1/experiment/{experiment_id}
Get an experiment object by its id
Parameters
| Name | In | Type | Required | Description |
|---|---|---|---|---|
| experiment_id | path | ExperimentId | true | Experiment id |
Example responses
200 Response
{
"id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
"project_id": "405d8375-3514-403b-8c43-83ae74cfe0e9",
"name": "string",
"description": "string",
"created": "2019-08-24T14:15:22Z",
"repo_info": {
"commit": "string",
"branch": "string",
"tag": "string",
"dirty": true,
"author_name": "string",
"author_email": "string",
"commit_message": "string",
"commit_time": "string",
"git_diff": "string"
},
"commit": "string",
"base_exp_id": "4838cee2-a665-4545-aa9f-483678c01a6b",
"deleted_at": "2019-08-24T14:15:22Z",
"dataset_id": "8c4c51f1-f6f3-43bc-b65d-7415e8ef22c0",
"dataset_version": "string",
"public": true,
"user_id": "a169451c-8525-4352-b8ca-070dd449a1a5",
"metadata": {
"property1": null,
"property2": null
}
}400 Response
"string"Responses
| Status | Meaning | Description | Schema |
|---|---|---|---|
| 200 | OK | Returns the experiment object | Experiment |
| 400 | Bad Request | The request was unacceptable, often due to missing a required parameter | Inline |
| 401 | Unauthorized | No valid API key provided | Inline |
| 403 | Forbidden | The API key doesn’t have permissions to perform the request | Inline |
| 429 | Too Many Requests | Too many requests hit the API too quickly. We recommend an exponential backoff of your requests | Inline |
| 500 | Internal Server Error | Something went wrong on Braintrust's end. (These are rare.) | Inline |
Response Schema
Status Code 400
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 401
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 403
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 429
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 500
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
🔒
This endpoint requires authentication.
Partially update experiment
Code samples
const inputBody = JSON.stringify({
name: "string",
description: "string",
repo_info: {
commit: "string",
branch: "string",
tag: "string",
dirty: true,
author_name: "string",
author_email: "string",
commit_message: "string",
commit_time: "string",
git_diff: "string",
},
base_exp_id: "4838cee2-a665-4545-aa9f-483678c01a6b",
dataset_id: "8c4c51f1-f6f3-43bc-b65d-7415e8ef22c0",
dataset_version: "string",
public: true,
metadata: {
property1: null,
property2: null,
},
});
const headers = {
"Content-Type": "application/json",
Accept: "application/json",
Authorization: "Bearer {access-token}",
};
fetch("https://api.braintrustdata.com/v1/experiment/{experiment_id}", {
method: "PATCH",
body: inputBody,
headers: headers,
})
.then(function (res) {
return res.json();
})
.then(function (body) {
console.log(body);
});PATCH /v1/experiment/{experiment_id}
Partially update an experiment object. Specify the fields to update in the payload. Any object-type fields will be deep-merged with existing content. Currently we do not support removing fields or setting them to null.
Body parameter
{
"name": "string",
"description": "string",
"repo_info": {
"commit": "string",
"branch": "string",
"tag": "string",
"dirty": true,
"author_name": "string",
"author_email": "string",
"commit_message": "string",
"commit_time": "string",
"git_diff": "string"
},
"base_exp_id": "4838cee2-a665-4545-aa9f-483678c01a6b",
"dataset_id": "8c4c51f1-f6f3-43bc-b65d-7415e8ef22c0",
"dataset_version": "string",
"public": true,
"metadata": {
"property1": null,
"property2": null
}
}Parameters
| Name | In | Type | Required | Description |
|---|---|---|---|---|
| experiment_id | path | ExperimentId | true | Experiment id |
| body | body | PatchExperiment | false | Fields to update |
Example responses
200 Response
{
"id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
"project_id": "405d8375-3514-403b-8c43-83ae74cfe0e9",
"name": "string",
"description": "string",
"created": "2019-08-24T14:15:22Z",
"repo_info": {
"commit": "string",
"branch": "string",
"tag": "string",
"dirty": true,
"author_name": "string",
"author_email": "string",
"commit_message": "string",
"commit_time": "string",
"git_diff": "string"
},
"commit": "string",
"base_exp_id": "4838cee2-a665-4545-aa9f-483678c01a6b",
"deleted_at": "2019-08-24T14:15:22Z",
"dataset_id": "8c4c51f1-f6f3-43bc-b65d-7415e8ef22c0",
"dataset_version": "string",
"public": true,
"user_id": "a169451c-8525-4352-b8ca-070dd449a1a5",
"metadata": {
"property1": null,
"property2": null
}
}400 Response
"string"Responses
| Status | Meaning | Description | Schema |
|---|---|---|---|
| 200 | OK | Returns the experiment object | Experiment |
| 400 | Bad Request | The request was unacceptable, often due to missing a required parameter | Inline |
| 401 | Unauthorized | No valid API key provided | Inline |
| 403 | Forbidden | The API key doesn’t have permissions to perform the request | Inline |
| 429 | Too Many Requests | Too many requests hit the API too quickly. We recommend an exponential backoff of your requests | Inline |
| 500 | Internal Server Error | Something went wrong on Braintrust's end. (These are rare.) | Inline |
Response Schema
Status Code 400
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 401
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 403
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 429
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 500
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
🔒
This endpoint requires authentication.
Delete experiment
Code samples
const headers = {
Accept: "application/json",
Authorization: "Bearer {access-token}",
};
fetch("https://api.braintrustdata.com/v1/experiment/{experiment_id}", {
method: "DELETE",
headers: headers,
})
.then(function (res) {
return res.json();
})
.then(function (body) {
console.log(body);
});DELETE /v1/experiment/{experiment_id}
Delete an experiment object by its id
Parameters
| Name | In | Type | Required | Description |
|---|---|---|---|---|
| experiment_id | path | ExperimentId | true | Experiment id |
Example responses
200 Response
{
"id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
"project_id": "405d8375-3514-403b-8c43-83ae74cfe0e9",
"name": "string",
"description": "string",
"created": "2019-08-24T14:15:22Z",
"repo_info": {
"commit": "string",
"branch": "string",
"tag": "string",
"dirty": true,
"author_name": "string",
"author_email": "string",
"commit_message": "string",
"commit_time": "string",
"git_diff": "string"
},
"commit": "string",
"base_exp_id": "4838cee2-a665-4545-aa9f-483678c01a6b",
"deleted_at": "2019-08-24T14:15:22Z",
"dataset_id": "8c4c51f1-f6f3-43bc-b65d-7415e8ef22c0",
"dataset_version": "string",
"public": true,
"user_id": "a169451c-8525-4352-b8ca-070dd449a1a5",
"metadata": {
"property1": null,
"property2": null
}
}400 Response
"string"Responses
| Status | Meaning | Description | Schema |
|---|---|---|---|
| 200 | OK | Returns the deleted experiment object | Experiment |
| 400 | Bad Request | The request was unacceptable, often due to missing a required parameter | Inline |
| 401 | Unauthorized | No valid API key provided | Inline |
| 403 | Forbidden | The API key doesn’t have permissions to perform the request | Inline |
| 429 | Too Many Requests | Too many requests hit the API too quickly. We recommend an exponential backoff of your requests | Inline |
| 500 | Internal Server Error | Something went wrong on Braintrust's end. (These are rare.) | Inline |
Response Schema
Status Code 400
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 401
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 403
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 429
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 500
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
🔒
This endpoint requires authentication.
Insert experiment events
Code samples
const inputBody = JSON.stringify({
events: [
{
input: null,
output: null,
expected: null,
scores: {
property1: 1,
property2: 1,
},
metadata: {
property1: null,
property2: null,
},
tags: ["string"],
metrics: {
start: 0,
end: 0,
prompt_tokens: 0,
completion_tokens: 0,
tokens: 0,
property1: null,
property2: null,
},
context: {
caller_functionname: "string",
caller_filename: "string",
caller_lineno: 0,
property1: null,
property2: null,
},
span_attributes: {
name: "string",
type: "llm",
property1: null,
property2: null,
},
id: "string",
dataset_record_id: "string",
_object_delete: true,
_is_merge: false,
_parent_id: "string",
},
],
});
const headers = {
"Content-Type": "application/json",
Accept: "application/json",
Authorization: "Bearer {access-token}",
};
fetch("https://api.braintrustdata.com/v1/experiment/{experiment_id}/insert", {
method: "POST",
body: inputBody,
headers: headers,
})
.then(function (res) {
return res.json();
})
.then(function (body) {
console.log(body);
});POST /v1/experiment/{experiment_id}/insert
Insert a set of events into the experiment
Body parameter
{
"events": [
{
"input": null,
"output": null,
"expected": null,
"scores": {
"property1": 1,
"property2": 1
},
"metadata": {
"property1": null,
"property2": null
},
"tags": ["string"],
"metrics": {
"start": 0,
"end": 0,
"prompt_tokens": 0,
"completion_tokens": 0,
"tokens": 0,
"property1": null,
"property2": null
},
"context": {
"caller_functionname": "string",
"caller_filename": "string",
"caller_lineno": 0,
"property1": null,
"property2": null
},
"span_attributes": {
"name": "string",
"type": "llm",
"property1": null,
"property2": null
},
"id": "string",
"dataset_record_id": "string",
"_object_delete": true,
"_is_merge": false,
"_parent_id": "string"
}
]
}Parameters
| Name | In | Type | Required | Description |
|---|---|---|---|---|
| experiment_id | path | ExperimentId | true | Experiment id |
| body | body | InsertExperimentEventRequest | true | An array of experiment events to insert |
Example responses
200 Response
{
"row_ids": ["string"]
}400 Response
"string"Responses
| Status | Meaning | Description | Schema |
|---|---|---|---|
| 200 | OK | Returns the inserted row ids | InsertEventsResponse |
| 400 | Bad Request | The request was unacceptable, often due to missing a required parameter | Inline |
| 401 | Unauthorized | No valid API key provided | Inline |
| 403 | Forbidden | The API key doesn’t have permissions to perform the request | Inline |
| 429 | Too Many Requests | Too many requests hit the API too quickly. We recommend an exponential backoff of your requests | Inline |
| 500 | Internal Server Error | Something went wrong on Braintrust's end. (These are rare.) | Inline |
Response Schema
Status Code 400
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 401
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 403
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 429
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 500
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
🔒
This endpoint requires authentication.
Fetch experiment (POST form)
Code samples
const inputBody = JSON.stringify({
limit: 0,
cursor: "string",
max_xact_id: "string",
max_root_span_id: "string",
filters: [
{
type: "path_lookup",
path: ["string"],
value: null,
},
],
version: "string",
});
const headers = {
"Content-Type": "application/json",
Accept: "application/json",
Authorization: "Bearer {access-token}",
};
fetch("https://api.braintrustdata.com/v1/experiment/{experiment_id}/fetch", {
method: "POST",
body: inputBody,
headers: headers,
})
.then(function (res) {
return res.json();
})
.then(function (body) {
console.log(body);
});POST /v1/experiment/{experiment_id}/fetch
Fetch the events in an experiment. Equivalent to the GET form of the same path, but with the parameters in the request body rather than in the URL query
Body parameter
{
"limit": 0,
"cursor": "string",
"max_xact_id": "string",
"max_root_span_id": "string",
"filters": [
{
"type": "path_lookup",
"path": ["string"],
"value": null
}
],
"version": "string"
}Parameters
| Name | In | Type | Required | Description |
|---|---|---|---|---|
| experiment_id | path | ExperimentId | true | Experiment id |
| body | body | FetchEventsRequest | false | Filters for the fetch query |
Example responses
200 Response
{
"events": [
{
"id": "string",
"dataset_record_id": "string",
"_xact_id": "string",
"created": "2019-08-24T14:15:22Z",
"project_id": "405d8375-3514-403b-8c43-83ae74cfe0e9",
"experiment_id": "916afd89-cac5-4339-9c59-dd068abdfa69",
"input": null,
"output": null,
"expected": null,
"scores": {
"property1": 1,
"property2": 1
},
"metadata": {
"property1": null,
"property2": null
},
"tags": ["string"],
"metrics": {
"start": 0,
"end": 0,
"prompt_tokens": 0,
"completion_tokens": 0,
"tokens": 0,
"property1": null,
"property2": null
},
"context": {
"caller_functionname": "string",
"caller_filename": "string",
"caller_lineno": 0,
"property1": null,
"property2": null
},
"span_id": "string",
"span_parents": ["string"],
"root_span_id": "string",
"span_attributes": {
"name": "string",
"type": "llm",
"property1": null,
"property2": null
}
}
],
"cursor": "string"
}400 Response
"string"Responses
| Status | Meaning | Description | Schema |
|---|---|---|---|
| 200 | OK | Returns the fetched rows | FetchExperimentEventsResponse |
| 400 | Bad Request | The request was unacceptable, often due to missing a required parameter | Inline |
| 401 | Unauthorized | No valid API key provided | Inline |
| 403 | Forbidden | The API key doesn’t have permissions to perform the request | Inline |
| 429 | Too Many Requests | Too many requests hit the API too quickly. We recommend an exponential backoff of your requests | Inline |
| 500 | Internal Server Error | Something went wrong on Braintrust's end. (These are rare.) | Inline |
Response Schema
Status Code 400
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 401
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 403
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 429
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 500
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
🔒
This endpoint requires authentication.
Fetch experiment (GET form)
Code samples
const headers = {
Accept: "application/json",
Authorization: "Bearer {access-token}",
};
fetch("https://api.braintrustdata.com/v1/experiment/{experiment_id}/fetch", {
method: "GET",
headers: headers,
})
.then(function (res) {
return res.json();
})
.then(function (body) {
console.log(body);
});GET /v1/experiment/{experiment_id}/fetch
Fetch the events in an experiment. Equivalent to the POST form of the same path, but with the parameters in the URL query rather than in the request body
Parameters
| Name | In | Type | Required | Description |
|---|---|---|---|---|
| experiment_id | path | ExperimentId | true | Experiment id |
| limit | query | FetchLimitParam | false | limit the number of traces fetched |
| max_xact_id | query | MaxXactId | false | DEPRECATION NOTICE: The manually-constructed pagination cursor is deprecated in favor of the explicit 'cursor' returned by object fetch requests. Please prefer the 'cursor' argument going forwards. |
| max_root_span_id | query | MaxRootSpanId | false | DEPRECATION NOTICE: The manually-constructed pagination cursor is deprecated in favor of the explicit 'cursor' returned by object fetch requests. Please prefer the 'cursor' argument going forwards. |
| version | query | any | false | Retrieve prompt at a specific version. |
Detailed descriptions
limit: limit the number of traces fetched
Fetch queries may be paginated if the total result size is expected to be large (e.g. project_logs which accumulate over a long time). Note that fetch queries only support pagination in descending time order (from latest to earliest _xact_id. Furthermore, later pages may return rows which showed up in earlier pages, except with an earlier _xact_id. This happens because pagination occurs over the whole version history of the event log. You will most likely want to exclude any such duplicate, outdated rows (by id) from your combined result set.
The limit parameter controls the number of full traces to return. So you may end up with more individual rows than the specified limit if you are fetching events containing traces.
max_xact_id: DEPRECATION NOTICE: The manually-constructed pagination cursor is deprecated in favor of the explicit 'cursor' returned by object fetch requests. Please prefer the 'cursor' argument going forwards.
Together, max_xact_id and max_root_span_id form a pagination cursor
Since a paginated fetch query returns results in order from latest to earliest, the cursor for the next page can be found as the row with the minimum (earliest) value of the tuple (_xact_id, root_span_id). See the documentation of limit for an overview of paginating fetch queries.
max_root_span_id: DEPRECATION NOTICE: The manually-constructed pagination cursor is deprecated in favor of the explicit 'cursor' returned by object fetch requests. Please prefer the 'cursor' argument going forwards.
Together, max_xact_id and max_root_span_id form a pagination cursor
Since a paginated fetch query returns results in order from latest to earliest, the cursor for the next page can be found as the row with the minimum (earliest) value of the tuple (_xact_id, root_span_id). See the documentation of limit for an overview of paginating fetch queries.
version: Retrieve prompt at a specific version.
The version id can either be a transaction id (e.g. '1000192656880881099') or a version identifier (e.g. '81cd05ee665fdfb3').
Example responses
200 Response
{
"events": [
{
"id": "string",
"dataset_record_id": "string",
"_xact_id": "string",
"created": "2019-08-24T14:15:22Z",
"project_id": "405d8375-3514-403b-8c43-83ae74cfe0e9",
"experiment_id": "916afd89-cac5-4339-9c59-dd068abdfa69",
"input": null,
"output": null,
"expected": null,
"scores": {
"property1": 1,
"property2": 1
},
"metadata": {
"property1": null,
"property2": null
},
"tags": ["string"],
"metrics": {
"start": 0,
"end": 0,
"prompt_tokens": 0,
"completion_tokens": 0,
"tokens": 0,
"property1": null,
"property2": null
},
"context": {
"caller_functionname": "string",
"caller_filename": "string",
"caller_lineno": 0,
"property1": null,
"property2": null
},
"span_id": "string",
"span_parents": ["string"],
"root_span_id": "string",
"span_attributes": {
"name": "string",
"type": "llm",
"property1": null,
"property2": null
}
}
],
"cursor": "string"
}400 Response
"string"Responses
| Status | Meaning | Description | Schema |
|---|---|---|---|
| 200 | OK | Returns the fetched rows | FetchExperimentEventsResponse |
| 400 | Bad Request | The request was unacceptable, often due to missing a required parameter | Inline |
| 401 | Unauthorized | No valid API key provided | Inline |
| 403 | Forbidden | The API key doesn’t have permissions to perform the request | Inline |
| 429 | Too Many Requests | Too many requests hit the API too quickly. We recommend an exponential backoff of your requests | Inline |
| 500 | Internal Server Error | Something went wrong on Braintrust's end. (These are rare.) | Inline |
Response Schema
Status Code 400
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 401
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 403
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 429
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 500
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
🔒
This endpoint requires authentication.
Feedback for experiment events
Code samples
const inputBody = JSON.stringify({
feedback: [
{
id: "string",
scores: {
property1: 1,
property2: 1,
},
expected: null,
comment: "string",
metadata: {
property1: null,
property2: null,
},
source: "app",
},
],
});
const headers = {
"Content-Type": "application/json",
Accept: "text/plain",
Authorization: "Bearer {access-token}",
};
fetch("https://api.braintrustdata.com/v1/experiment/{experiment_id}/feedback", {
method: "POST",
body: inputBody,
headers: headers,
})
.then(function (res) {
return res.json();
})
.then(function (body) {
console.log(body);
});POST /v1/experiment/{experiment_id}/feedback
Log feedback for a set of experiment events
Body parameter
{
"feedback": [
{
"id": "string",
"scores": {
"property1": 1,
"property2": 1
},
"expected": null,
"comment": "string",
"metadata": {
"property1": null,
"property2": null
},
"source": "app"
}
]
}Parameters
| Name | In | Type | Required | Description |
|---|---|---|---|---|
| experiment_id | path | ExperimentId | true | Experiment id |
| body | body | FeedbackExperimentEventRequest | true | An array of feedback objects |
Example responses
400 Response
"string"nullResponses
| Status | Meaning | Description | Schema |
|---|---|---|---|
| 200 | OK | No return value | None |
| 400 | Bad Request | The request was unacceptable, often due to missing a required parameter | Inline |
| 401 | Unauthorized | No valid API key provided | Inline |
| 403 | Forbidden | The API key doesn’t have permissions to perform the request | Inline |
| 429 | Too Many Requests | Too many requests hit the API too quickly. We recommend an exponential backoff of your requests | Inline |
| 500 | Internal Server Error | Something went wrong on Braintrust's end. (These are rare.) | Inline |
Response Schema
Status Code 400
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 401
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 403
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 429
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 500
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
🔒
This endpoint requires authentication.
Summarize experiment
Code samples
const headers = {
Accept: "application/json",
Authorization: "Bearer {access-token}",
};
fetch(
"https://api.braintrustdata.com/v1/experiment/{experiment_id}/summarize",
{
method: "GET",
headers: headers,
},
)
.then(function (res) {
return res.json();
})
.then(function (body) {
console.log(body);
});GET /v1/experiment/{experiment_id}/summarize
Summarize experiment
Parameters
| Name | In | Type | Required | Description |
|---|---|---|---|---|
| experiment_id | path | ExperimentId | true | Experiment id |
| summarize_scores | query | SummarizeScores | false | Whether to summarize the scores and metrics. If false (or omitted), only the metadata will be returned. |
| comparison_experiment_id | query | ComparisonExperimentId | false | The experiment to compare against, if summarizing scores and metrics. If omitted, will fall back to the base_exp_id stored in the experiment metadata, and then to the most recent experiment run in the same project. Must pass summarize_scores=true for this id to be used |
Example responses
200 Response
{
"project_name": "string",
"experiment_name": "string",
"project_url": "http://example.com",
"experiment_url": "http://example.com",
"comparison_experiment_name": "string",
"scores": {
"property1": {
"name": "string",
"score": 1,
"diff": -1,
"improvements": 0,
"regressions": 0
},
"property2": {
"name": "string",
"score": 1,
"diff": -1,
"improvements": 0,
"regressions": 0
}
},
"metrics": {
"property1": {
"name": "string",
"metric": 0,
"unit": "string",
"diff": 0,
"improvements": 0,
"regressions": 0
},
"property2": {
"name": "string",
"metric": 0,
"unit": "string",
"diff": 0,
"improvements": 0,
"regressions": 0
}
}
}400 Response
"string"Responses
| Status | Meaning | Description | Schema |
|---|---|---|---|
| 200 | OK | Experiment summary | SummarizeExperimentResponse |
| 400 | Bad Request | The request was unacceptable, often due to missing a required parameter | Inline |
| 401 | Unauthorized | No valid API key provided | Inline |
| 403 | Forbidden | The API key doesn’t have permissions to perform the request | Inline |
| 429 | Too Many Requests | Too many requests hit the API too quickly. We recommend an exponential backoff of your requests | Inline |
| 500 | Internal Server Error | Something went wrong on Braintrust's end. (These are rare.) | Inline |
Response Schema
Status Code 400
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 401
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 403
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 429
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 500
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
🔒
This endpoint requires authentication.
Datasets
Create dataset
Code samples
const inputBody = JSON.stringify({
project_id: "405d8375-3514-403b-8c43-83ae74cfe0e9",
name: "string",
description: "string",
});
const headers = {
"Content-Type": "application/json",
Accept: "application/json",
Authorization: "Bearer {access-token}",
};
fetch("https://api.braintrustdata.com/v1/dataset", {
method: "POST",
body: inputBody,
headers: headers,
})
.then(function (res) {
return res.json();
})
.then(function (body) {
console.log(body);
});POST /v1/dataset
Create a new dataset. If there is an existing dataset in the project with the same name as the one specified in the request, will return the existing dataset unmodified
Body parameter
{
"project_id": "405d8375-3514-403b-8c43-83ae74cfe0e9",
"name": "string",
"description": "string"
}Parameters
| Name | In | Type | Required | Description |
|---|---|---|---|---|
| body | body | CreateDataset | true | Any desired information about the new dataset object |
Example responses
200 Response
{
"id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
"project_id": "405d8375-3514-403b-8c43-83ae74cfe0e9",
"name": "string",
"description": "string",
"created": "2019-08-24T14:15:22Z",
"deleted_at": "2019-08-24T14:15:22Z",
"user_id": "a169451c-8525-4352-b8ca-070dd449a1a5"
}400 Response
"string"Responses
| Status | Meaning | Description | Schema |
|---|---|---|---|
| 200 | OK | Returns the new dataset object | Dataset |
| 400 | Bad Request | The request was unacceptable, often due to missing a required parameter | Inline |
| 401 | Unauthorized | No valid API key provided | Inline |
| 403 | Forbidden | The API key doesn’t have permissions to perform the request | Inline |
| 429 | Too Many Requests | Too many requests hit the API too quickly. We recommend an exponential backoff of your requests | Inline |
| 500 | Internal Server Error | Something went wrong on Braintrust's end. (These are rare.) | Inline |
Response Schema
Status Code 400
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 401
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 403
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 429
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 500
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
🔒
This endpoint requires authentication.
DEPRECATED. Create or replace dataset
Code samples
const inputBody = JSON.stringify({
project_id: "405d8375-3514-403b-8c43-83ae74cfe0e9",
name: "string",
description: "string",
});
const headers = {
"Content-Type": "application/json",
Accept: "application/json",
Authorization: "Bearer {access-token}",
};
fetch("https://api.braintrustdata.com/v1/dataset", {
method: "PUT",
body: inputBody,
headers: headers,
})
.then(function (res) {
return res.json();
})
.then(function (body) {
console.log(body);
});PUT /v1/dataset
NOTE: This operation is deprecated and will be removed in a future revision of the API. Create or replace a new dataset. If there is an existing dataset in the project with the same name as the one specified in the request, will return the existing dataset unmodified, will replace the existing dataset with the provided fields
Body parameter
{
"project_id": "405d8375-3514-403b-8c43-83ae74cfe0e9",
"name": "string",
"description": "string"
}Parameters
| Name | In | Type | Required | Description |
|---|---|---|---|---|
| body | body | CreateDataset | true | Any desired information about the new dataset object |
Example responses
200 Response
{
"id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
"project_id": "405d8375-3514-403b-8c43-83ae74cfe0e9",
"name": "string",
"description": "string",
"created": "2019-08-24T14:15:22Z",
"deleted_at": "2019-08-24T14:15:22Z",
"user_id": "a169451c-8525-4352-b8ca-070dd449a1a5"
}400 Response
"string"Responses
| Status | Meaning | Description | Schema |
|---|---|---|---|
| 200 | OK | Returns the new dataset object | Dataset |
| 400 | Bad Request | The request was unacceptable, often due to missing a required parameter | Inline |
| 401 | Unauthorized | No valid API key provided | Inline |
| 403 | Forbidden | The API key doesn’t have permissions to perform the request | Inline |
| 429 | Too Many Requests | Too many requests hit the API too quickly. We recommend an exponential backoff of your requests | Inline |
| 500 | Internal Server Error | Something went wrong on Braintrust's end. (These are rare.) | Inline |
Response Schema
Status Code 400
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 401
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 403
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 429
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 500
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
🔒
This endpoint requires authentication.
List datasets
Code samples
const headers = {
Accept: "application/json",
Authorization: "Bearer {access-token}",
};
fetch("https://api.braintrustdata.com/v1/dataset", {
method: "GET",
headers: headers,
})
.then(function (res) {
return res.json();
})
.then(function (body) {
console.log(body);
});GET /v1/dataset
List out all datasets. The datasets are sorted by creation date, with the most recently-created datasets coming first
Parameters
| Name | In | Type | Required | Description |
|---|---|---|---|---|
| limit | query | AppLimitParam | false | Limit the number of objects to return |
| starting_after | query | StartingAfter | false | Pagination cursor id. |
| ending_before | query | EndingBefore | false | Pagination cursor id. |
| dataset_name | query | DatasetName | false | Name of the dataset to search for |
| project_name | query | ProjectName | false | Name of the project to search for |
| org_name | query | OrgName | false | Filter search results to within a particular organization |
| ids | query | Ids | false | Filter search results to a particular set of object IDs. To specify a list of IDs, include the query param multiple times |
Detailed descriptions
starting_after: Pagination cursor id.
For example, if the final item in the last page you fetched had an id of foo, pass starting_after=foo to fetch the next page. Note: you may only pass one of starting_after and ending_before
ending_before: Pagination cursor id.
For example, if the initial item in the last page you fetched had an id of foo, pass ending_before=foo to fetch the previous page. Note: you may only pass one of starting_after and ending_before
Example responses
200 Response
{
"objects": [
{
"id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
"project_id": "405d8375-3514-403b-8c43-83ae74cfe0e9",
"name": "string",
"description": "string",
"created": "2019-08-24T14:15:22Z",
"deleted_at": "2019-08-24T14:15:22Z",
"user_id": "a169451c-8525-4352-b8ca-070dd449a1a5"
}
]
}400 Response
"string"Responses
| Status | Meaning | Description | Schema |
|---|---|---|---|
| 200 | OK | Returns a list of dataset objects | Inline |
| 400 | Bad Request | The request was unacceptable, often due to missing a required parameter | Inline |
| 401 | Unauthorized | No valid API key provided | Inline |
| 403 | Forbidden | The API key doesn’t have permissions to perform the request | Inline |
| 429 | Too Many Requests | Too many requests hit the API too quickly. We recommend an exponential backoff of your requests | Inline |
| 500 | Internal Server Error | Something went wrong on Braintrust's end. (These are rare.) | Inline |
Response Schema
Status Code 200
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| » objects | [Dataset] | true | none | A list of dataset objects |
| »» id | string(uuid) | true | none | Unique identifier for the dataset |
| »» project_id | string(uuid)¦null | false | none | Unique identifier for the project that the dataset belongs under |
| »» name | string | true | none | Name of the dataset. Within a project, dataset names are unique |
| »» description | string¦null | false | none | Textual description of the dataset |
| »» created | string(date-time)¦null | false | none | Date of dataset creation |
| »» deleted_at | string(date-time)¦null | false | none | Date of dataset deletion, or null if the dataset is still active |
| »» user_id | string(uuid)¦null | false | none | Identifies the user who created the dataset |
Status Code 400
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 401
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 403
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 429
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 500
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
🔒
This endpoint requires authentication.
Get dataset
Code samples
const headers = {
Accept: "application/json",
Authorization: "Bearer {access-token}",
};
fetch("https://api.braintrustdata.com/v1/dataset/{dataset_id}", {
method: "GET",
headers: headers,
})
.then(function (res) {
return res.json();
})
.then(function (body) {
console.log(body);
});GET /v1/dataset/{dataset_id}
Get a dataset object by its id
Parameters
| Name | In | Type | Required | Description |
|---|---|---|---|---|
| dataset_id | path | DatasetId | true | Dataset id |
Example responses
200 Response
{
"id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
"project_id": "405d8375-3514-403b-8c43-83ae74cfe0e9",
"name": "string",
"description": "string",
"created": "2019-08-24T14:15:22Z",
"deleted_at": "2019-08-24T14:15:22Z",
"user_id": "a169451c-8525-4352-b8ca-070dd449a1a5"
}400 Response
"string"Responses
| Status | Meaning | Description | Schema |
|---|---|---|---|
| 200 | OK | Returns the dataset object | Dataset |
| 400 | Bad Request | The request was unacceptable, often due to missing a required parameter | Inline |
| 401 | Unauthorized | No valid API key provided | Inline |
| 403 | Forbidden | The API key doesn’t have permissions to perform the request | Inline |
| 429 | Too Many Requests | Too many requests hit the API too quickly. We recommend an exponential backoff of your requests | Inline |
| 500 | Internal Server Error | Something went wrong on Braintrust's end. (These are rare.) | Inline |
Response Schema
Status Code 400
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 401
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 403
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 429
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 500
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
🔒
This endpoint requires authentication.
Partially update dataset
Code samples
const inputBody = JSON.stringify({
name: "string",
description: "string",
});
const headers = {
"Content-Type": "application/json",
Accept: "application/json",
Authorization: "Bearer {access-token}",
};
fetch("https://api.braintrustdata.com/v1/dataset/{dataset_id}", {
method: "PATCH",
body: inputBody,
headers: headers,
})
.then(function (res) {
return res.json();
})
.then(function (body) {
console.log(body);
});PATCH /v1/dataset/{dataset_id}
Partially update a dataset object. Specify the fields to update in the payload. Any object-type fields will be deep-merged with existing content. Currently we do not support removing fields or setting them to null.
Body parameter
{
"name": "string",
"description": "string"
}Parameters
| Name | In | Type | Required | Description |
|---|---|---|---|---|
| dataset_id | path | DatasetId | true | Dataset id |
| body | body | PatchDataset | false | Fields to update |
Example responses
200 Response
{
"id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
"project_id": "405d8375-3514-403b-8c43-83ae74cfe0e9",
"name": "string",
"description": "string",
"created": "2019-08-24T14:15:22Z",
"deleted_at": "2019-08-24T14:15:22Z",
"user_id": "a169451c-8525-4352-b8ca-070dd449a1a5"
}400 Response
"string"Responses
| Status | Meaning | Description | Schema |
|---|---|---|---|
| 200 | OK | Returns the dataset object | Dataset |
| 400 | Bad Request | The request was unacceptable, often due to missing a required parameter | Inline |
| 401 | Unauthorized | No valid API key provided | Inline |
| 403 | Forbidden | The API key doesn’t have permissions to perform the request | Inline |
| 429 | Too Many Requests | Too many requests hit the API too quickly. We recommend an exponential backoff of your requests | Inline |
| 500 | Internal Server Error | Something went wrong on Braintrust's end. (These are rare.) | Inline |
Response Schema
Status Code 400
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 401
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 403
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 429
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 500
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
🔒
This endpoint requires authentication.
Delete dataset
Code samples
const headers = {
Accept: "application/json",
Authorization: "Bearer {access-token}",
};
fetch("https://api.braintrustdata.com/v1/dataset/{dataset_id}", {
method: "DELETE",
headers: headers,
})
.then(function (res) {
return res.json();
})
.then(function (body) {
console.log(body);
});DELETE /v1/dataset/{dataset_id}
Delete a dataset object by its id
Parameters
| Name | In | Type | Required | Description |
|---|---|---|---|---|
| dataset_id | path | DatasetId | true | Dataset id |
Example responses
200 Response
{
"id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
"project_id": "405d8375-3514-403b-8c43-83ae74cfe0e9",
"name": "string",
"description": "string",
"created": "2019-08-24T14:15:22Z",
"deleted_at": "2019-08-24T14:15:22Z",
"user_id": "a169451c-8525-4352-b8ca-070dd449a1a5"
}400 Response
"string"Responses
| Status | Meaning | Description | Schema |
|---|---|---|---|
| 200 | OK | Returns the deleted dataset object | Dataset |
| 400 | Bad Request | The request was unacceptable, often due to missing a required parameter | Inline |
| 401 | Unauthorized | No valid API key provided | Inline |
| 403 | Forbidden | The API key doesn’t have permissions to perform the request | Inline |
| 429 | Too Many Requests | Too many requests hit the API too quickly. We recommend an exponential backoff of your requests | Inline |
| 500 | Internal Server Error | Something went wrong on Braintrust's end. (These are rare.) | Inline |
Response Schema
Status Code 400
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 401
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 403
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 429
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 500
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
🔒
This endpoint requires authentication.
Insert dataset events
Code samples
const inputBody = JSON.stringify({
events: [
{
input: null,
expected: null,
metadata: {
property1: null,
property2: null,
},
tags: ["string"],
id: "string",
_object_delete: true,
_is_merge: false,
_parent_id: "string",
},
],
});
const headers = {
"Content-Type": "application/json",
Accept: "application/json",
Authorization: "Bearer {access-token}",
};
fetch("https://api.braintrustdata.com/v1/dataset/{dataset_id}/insert", {
method: "POST",
body: inputBody,
headers: headers,
})
.then(function (res) {
return res.json();
})
.then(function (body) {
console.log(body);
});POST /v1/dataset/{dataset_id}/insert
Insert a set of events into the dataset
Body parameter
{
"events": [
{
"input": null,
"expected": null,
"metadata": {
"property1": null,
"property2": null
},
"tags": ["string"],
"id": "string",
"_object_delete": true,
"_is_merge": false,
"_parent_id": "string"
}
]
}Parameters
| Name | In | Type | Required | Description |
|---|---|---|---|---|
| dataset_id | path | DatasetId | true | Dataset id |
| body | body | InsertDatasetEventRequest | true | An array of dataset events to insert |
Example responses
200 Response
{
"row_ids": ["string"]
}400 Response
"string"Responses
| Status | Meaning | Description | Schema |
|---|---|---|---|
| 200 | OK | Returns the inserted row ids | InsertEventsResponse |
| 400 | Bad Request | The request was unacceptable, often due to missing a required parameter | Inline |
| 401 | Unauthorized | No valid API key provided | Inline |
| 403 | Forbidden | The API key doesn’t have permissions to perform the request | Inline |
| 429 | Too Many Requests | Too many requests hit the API too quickly. We recommend an exponential backoff of your requests | Inline |
| 500 | Internal Server Error | Something went wrong on Braintrust's end. (These are rare.) | Inline |
Response Schema
Status Code 400
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 401
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 403
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 429
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 500
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
🔒
This endpoint requires authentication.
Fetch dataset (POST form)
Code samples
const inputBody = JSON.stringify({
limit: 0,
cursor: "string",
max_xact_id: "string",
max_root_span_id: "string",
filters: [
{
type: "path_lookup",
path: ["string"],
value: null,
},
],
version: "string",
});
const headers = {
"Content-Type": "application/json",
Accept: "application/json",
Authorization: "Bearer {access-token}",
};
fetch("https://api.braintrustdata.com/v1/dataset/{dataset_id}/fetch", {
method: "POST",
body: inputBody,
headers: headers,
})
.then(function (res) {
return res.json();
})
.then(function (body) {
console.log(body);
});POST /v1/dataset/{dataset_id}/fetch
Fetch the events in a dataset. Equivalent to the GET form of the same path, but with the parameters in the request body rather than in the URL query
Body parameter
{
"limit": 0,
"cursor": "string",
"max_xact_id": "string",
"max_root_span_id": "string",
"filters": [
{
"type": "path_lookup",
"path": ["string"],
"value": null
}
],
"version": "string"
}Parameters
| Name | In | Type | Required | Description |
|---|---|---|---|---|
| dataset_id | path | DatasetId | true | Dataset id |
| body | body | FetchEventsRequest | false | Filters for the fetch query |
Example responses
200 Response
{
"events": [
{
"id": "string",
"_xact_id": "string",
"created": "2019-08-24T14:15:22Z",
"project_id": "405d8375-3514-403b-8c43-83ae74cfe0e9",
"dataset_id": "8c4c51f1-f6f3-43bc-b65d-7415e8ef22c0",
"input": null,
"expected": null,
"metadata": {
"property1": null,
"property2": null
},
"tags": ["string"],
"span_id": "string",
"root_span_id": "string"
}
],
"cursor": "string"
}400 Response
"string"Responses
| Status | Meaning | Description | Schema |
|---|---|---|---|
| 200 | OK | Returns the fetched rows | FetchDatasetEventsResponse |
| 400 | Bad Request | The request was unacceptable, often due to missing a required parameter | Inline |
| 401 | Unauthorized | No valid API key provided | Inline |
| 403 | Forbidden | The API key doesn’t have permissions to perform the request | Inline |
| 429 | Too Many Requests | Too many requests hit the API too quickly. We recommend an exponential backoff of your requests | Inline |
| 500 | Internal Server Error | Something went wrong on Braintrust's end. (These are rare.) | Inline |
Response Schema
Status Code 400
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 401
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 403
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 429
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 500
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
🔒
This endpoint requires authentication.
Fetch dataset (GET form)
Code samples
const headers = {
Accept: "application/json",
Authorization: "Bearer {access-token}",
};
fetch("https://api.braintrustdata.com/v1/dataset/{dataset_id}/fetch", {
method: "GET",
headers: headers,
})
.then(function (res) {
return res.json();
})
.then(function (body) {
console.log(body);
});GET /v1/dataset/{dataset_id}/fetch
Fetch the events in a dataset. Equivalent to the POST form of the same path, but with the parameters in the URL query rather than in the request body
Parameters
| Name | In | Type | Required | Description |
|---|---|---|---|---|
| dataset_id | path | DatasetId | true | Dataset id |
| limit | query | FetchLimitParam | false | limit the number of traces fetched |
| max_xact_id | query | MaxXactId | false | DEPRECATION NOTICE: The manually-constructed pagination cursor is deprecated in favor of the explicit 'cursor' returned by object fetch requests. Please prefer the 'cursor' argument going forwards. |
| max_root_span_id | query | MaxRootSpanId | false | DEPRECATION NOTICE: The manually-constructed pagination cursor is deprecated in favor of the explicit 'cursor' returned by object fetch requests. Please prefer the 'cursor' argument going forwards. |
| version | query | any | false | Retrieve prompt at a specific version. |
Detailed descriptions
limit: limit the number of traces fetched
Fetch queries may be paginated if the total result size is expected to be large (e.g. project_logs which accumulate over a long time). Note that fetch queries only support pagination in descending time order (from latest to earliest _xact_id. Furthermore, later pages may return rows which showed up in earlier pages, except with an earlier _xact_id. This happens because pagination occurs over the whole version history of the event log. You will most likely want to exclude any such duplicate, outdated rows (by id) from your combined result set.
The limit parameter controls the number of full traces to return. So you may end up with more individual rows than the specified limit if you are fetching events containing traces.
max_xact_id: DEPRECATION NOTICE: The manually-constructed pagination cursor is deprecated in favor of the explicit 'cursor' returned by object fetch requests. Please prefer the 'cursor' argument going forwards.
Together, max_xact_id and max_root_span_id form a pagination cursor
Since a paginated fetch query returns results in order from latest to earliest, the cursor for the next page can be found as the row with the minimum (earliest) value of the tuple (_xact_id, root_span_id). See the documentation of limit for an overview of paginating fetch queries.
max_root_span_id: DEPRECATION NOTICE: The manually-constructed pagination cursor is deprecated in favor of the explicit 'cursor' returned by object fetch requests. Please prefer the 'cursor' argument going forwards.
Together, max_xact_id and max_root_span_id form a pagination cursor
Since a paginated fetch query returns results in order from latest to earliest, the cursor for the next page can be found as the row with the minimum (earliest) value of the tuple (_xact_id, root_span_id). See the documentation of limit for an overview of paginating fetch queries.
version: Retrieve prompt at a specific version.
The version id can either be a transaction id (e.g. '1000192656880881099') or a version identifier (e.g. '81cd05ee665fdfb3').
Example responses
200 Response
{
"events": [
{
"id": "string",
"_xact_id": "string",
"created": "2019-08-24T14:15:22Z",
"project_id": "405d8375-3514-403b-8c43-83ae74cfe0e9",
"dataset_id": "8c4c51f1-f6f3-43bc-b65d-7415e8ef22c0",
"input": null,
"expected": null,
"metadata": {
"property1": null,
"property2": null
},
"tags": ["string"],
"span_id": "string",
"root_span_id": "string"
}
],
"cursor": "string"
}400 Response
"string"Responses
| Status | Meaning | Description | Schema |
|---|---|---|---|
| 200 | OK | Returns the fetched rows | FetchDatasetEventsResponse |
| 400 | Bad Request | The request was unacceptable, often due to missing a required parameter | Inline |
| 401 | Unauthorized | No valid API key provided | Inline |
| 403 | Forbidden | The API key doesn’t have permissions to perform the request | Inline |
| 429 | Too Many Requests | Too many requests hit the API too quickly. We recommend an exponential backoff of your requests | Inline |
| 500 | Internal Server Error | Something went wrong on Braintrust's end. (These are rare.) | Inline |
Response Schema
Status Code 400
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 401
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 403
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 429
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 500
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
🔒
This endpoint requires authentication.
Feedback for dataset events
Code samples
const inputBody = JSON.stringify({
feedback: [
{
id: "string",
comment: "string",
metadata: {
property1: null,
property2: null,
},
source: "app",
},
],
});
const headers = {
"Content-Type": "application/json",
Accept: "text/plain",
Authorization: "Bearer {access-token}",
};
fetch("https://api.braintrustdata.com/v1/dataset/{dataset_id}/feedback", {
method: "POST",
body: inputBody,
headers: headers,
})
.then(function (res) {
return res.json();
})
.then(function (body) {
console.log(body);
});POST /v1/dataset/{dataset_id}/feedback
Log feedback for a set of dataset events
Body parameter
{
"feedback": [
{
"id": "string",
"comment": "string",
"metadata": {
"property1": null,
"property2": null
},
"source": "app"
}
]
}Parameters
| Name | In | Type | Required | Description |
|---|---|---|---|---|
| dataset_id | path | DatasetId | true | Dataset id |
| body | body | FeedbackDatasetEventRequest | true | An array of feedback objects |
Example responses
400 Response
"string"nullResponses
| Status | Meaning | Description | Schema |
|---|---|---|---|
| 200 | OK | No return value | None |
| 400 | Bad Request | The request was unacceptable, often due to missing a required parameter | Inline |
| 401 | Unauthorized | No valid API key provided | Inline |
| 403 | Forbidden | The API key doesn’t have permissions to perform the request | Inline |
| 429 | Too Many Requests | Too many requests hit the API too quickly. We recommend an exponential backoff of your requests | Inline |
| 500 | Internal Server Error | Something went wrong on Braintrust's end. (These are rare.) | Inline |
Response Schema
Status Code 400
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 401
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 403
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 429
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 500
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
🔒
This endpoint requires authentication.
Summarize dataset
Code samples
const headers = {
Accept: "application/json",
Authorization: "Bearer {access-token}",
};
fetch("https://api.braintrustdata.com/v1/dataset/{dataset_id}/summarize", {
method: "GET",
headers: headers,
})
.then(function (res) {
return res.json();
})
.then(function (body) {
console.log(body);
});GET /v1/dataset/{dataset_id}/summarize
Summarize dataset
Parameters
| Name | In | Type | Required | Description |
|---|---|---|---|---|
| dataset_id | path | DatasetId | true | Dataset id |
| summarize_data | query | SummarizeData | false | Whether to summarize the data. If false (or omitted), only the metadata will be returned. |
Example responses
200 Response
{
"project_name": "string",
"dataset_name": "string",
"project_url": "http://example.com",
"dataset_url": "http://example.com",
"data_summary": {
"total_records": 0
}
}400 Response
"string"Responses
| Status | Meaning | Description | Schema |
|---|---|---|---|
| 200 | OK | Dataset summary | SummarizeDatasetResponse |
| 400 | Bad Request | The request was unacceptable, often due to missing a required parameter | Inline |
| 401 | Unauthorized | No valid API key provided | Inline |
| 403 | Forbidden | The API key doesn’t have permissions to perform the request | Inline |
| 429 | Too Many Requests | Too many requests hit the API too quickly. We recommend an exponential backoff of your requests | Inline |
| 500 | Internal Server Error | Something went wrong on Braintrust's end. (These are rare.) | Inline |
Response Schema
Status Code 400
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 401
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 403
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 429
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 500
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
🔒
This endpoint requires authentication.
Prompts
Create prompt
Code samples
const inputBody = JSON.stringify({
project_id: "405d8375-3514-403b-8c43-83ae74cfe0e9",
name: "string",
slug: "string",
description: "string",
prompt_data: {
prompt: {
type: "completion",
content: "string",
},
options: {
model: "string",
params: {
use_cache: true,
temperature: 0,
top_p: 0,
max_tokens: 0,
frequency_penalty: 0,
presence_penalty: 0,
response_format: {
type: "json_object",
},
tool_choice: "auto",
function_call: "auto",
n: 0,
stop: ["string"],
},
position: "string",
},
origin: {
prompt_id: "string",
project_id: "string",
prompt_version: "string",
},
},
tags: ["string"],
});
const headers = {
"Content-Type": "application/json",
Accept: "application/json",
Authorization: "Bearer {access-token}",
};
fetch("https://api.braintrustdata.com/v1/prompt", {
method: "POST",
body: inputBody,
headers: headers,
})
.then(function (res) {
return res.json();
})
.then(function (body) {
console.log(body);
});POST /v1/prompt
Create a new prompt. If there is an existing prompt in the project with the same slug as the one specified in the request, will return the existing prompt unmodified
Body parameter
{
"project_id": "405d8375-3514-403b-8c43-83ae74cfe0e9",
"name": "string",
"slug": "string",
"description": "string",
"prompt_data": {
"prompt": {
"type": "completion",
"content": "string"
},
"options": {
"model": "string",
"params": {
"use_cache": true,
"temperature": 0,
"top_p": 0,
"max_tokens": 0,
"frequency_penalty": 0,
"presence_penalty": 0,
"response_format": {
"type": "json_object"
},
"tool_choice": "auto",
"function_call": "auto",
"n": 0,
"stop": ["string"]
},
"position": "string"
},
"origin": {
"prompt_id": "string",
"project_id": "string",
"prompt_version": "string"
}
},
"tags": ["string"]
}Parameters
| Name | In | Type | Required | Description |
|---|---|---|---|---|
| body | body | CreatePrompt | true | Any desired information about the new prompt object |
Example responses
200 Response
{
"id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
"_xact_id": "string",
"project_id": "405d8375-3514-403b-8c43-83ae74cfe0e9",
"log_id": "p",
"org_id": "a40f5d1f-d889-42e9-94ea-b9b33585fc6b",
"name": "string",
"slug": "string",
"description": "string",
"created": "2019-08-24T14:15:22Z",
"prompt_data": {
"prompt": {
"type": "completion",
"content": "string"
},
"options": {
"model": "string",
"params": {
"use_cache": true,
"temperature": 0,
"top_p": 0,
"max_tokens": 0,
"frequency_penalty": 0,
"presence_penalty": 0,
"response_format": {
"type": "json_object"
},
"tool_choice": "auto",
"function_call": "auto",
"n": 0,
"stop": ["string"]
},
"position": "string"
},
"origin": {
"prompt_id": "string",
"project_id": "string",
"prompt_version": "string"
}
},
"tags": ["string"],
"metadata": {
"property1": null,
"property2": null
}
}400 Response
"string"Responses
| Status | Meaning | Description | Schema |
|---|---|---|---|
| 200 | OK | Returns the new prompt object | Prompt |
| 400 | Bad Request | The request was unacceptable, often due to missing a required parameter | Inline |
| 401 | Unauthorized | No valid API key provided | Inline |
| 403 | Forbidden | The API key doesn’t have permissions to perform the request | Inline |
| 429 | Too Many Requests | Too many requests hit the API too quickly. We recommend an exponential backoff of your requests | Inline |
| 500 | Internal Server Error | Something went wrong on Braintrust's end. (These are rare.) | Inline |
Response Schema
Status Code 400
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 401
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 403
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 429
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 500
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
🔒
This endpoint requires authentication.
DEPRECATED. Create or replace prompt
Code samples
const inputBody = JSON.stringify({
project_id: "405d8375-3514-403b-8c43-83ae74cfe0e9",
name: "string",
slug: "string",
description: "string",
prompt_data: {
prompt: {
type: "completion",
content: "string",
},
options: {
model: "string",
params: {
use_cache: true,
temperature: 0,
top_p: 0,
max_tokens: 0,
frequency_penalty: 0,
presence_penalty: 0,
response_format: {
type: "json_object",
},
tool_choice: "auto",
function_call: "auto",
n: 0,
stop: ["string"],
},
position: "string",
},
origin: {
prompt_id: "string",
project_id: "string",
prompt_version: "string",
},
},
tags: ["string"],
});
const headers = {
"Content-Type": "application/json",
Accept: "application/json",
Authorization: "Bearer {access-token}",
};
fetch("https://api.braintrustdata.com/v1/prompt", {
method: "PUT",
body: inputBody,
headers: headers,
})
.then(function (res) {
return res.json();
})
.then(function (body) {
console.log(body);
});PUT /v1/prompt
NOTE: This operation is deprecated and will be removed in a future revision of the API. Create or replace a new prompt. If there is an existing prompt in the project with the same slug as the one specified in the request, will return the existing prompt unmodified, will replace the existing prompt with the provided fields
Body parameter
{
"project_id": "405d8375-3514-403b-8c43-83ae74cfe0e9",
"name": "string",
"slug": "string",
"description": "string",
"prompt_data": {
"prompt": {
"type": "completion",
"content": "string"
},
"options": {
"model": "string",
"params": {
"use_cache": true,
"temperature": 0,
"top_p": 0,
"max_tokens": 0,
"frequency_penalty": 0,
"presence_penalty": 0,
"response_format": {
"type": "json_object"
},
"tool_choice": "auto",
"function_call": "auto",
"n": 0,
"stop": ["string"]
},
"position": "string"
},
"origin": {
"prompt_id": "string",
"project_id": "string",
"prompt_version": "string"
}
},
"tags": ["string"]
}Parameters
| Name | In | Type | Required | Description |
|---|---|---|---|---|
| body | body | CreatePrompt | true | Any desired information about the new prompt object |
Example responses
200 Response
{
"id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
"_xact_id": "string",
"project_id": "405d8375-3514-403b-8c43-83ae74cfe0e9",
"log_id": "p",
"org_id": "a40f5d1f-d889-42e9-94ea-b9b33585fc6b",
"name": "string",
"slug": "string",
"description": "string",
"created": "2019-08-24T14:15:22Z",
"prompt_data": {
"prompt": {
"type": "completion",
"content": "string"
},
"options": {
"model": "string",
"params": {
"use_cache": true,
"temperature": 0,
"top_p": 0,
"max_tokens": 0,
"frequency_penalty": 0,
"presence_penalty": 0,
"response_format": {
"type": "json_object"
},
"tool_choice": "auto",
"function_call": "auto",
"n": 0,
"stop": ["string"]
},
"position": "string"
},
"origin": {
"prompt_id": "string",
"project_id": "string",
"prompt_version": "string"
}
},
"tags": ["string"],
"metadata": {
"property1": null,
"property2": null
}
}400 Response
"string"Responses
| Status | Meaning | Description | Schema |
|---|---|---|---|
| 200 | OK | Returns the new prompt object | Prompt |
| 400 | Bad Request | The request was unacceptable, often due to missing a required parameter | Inline |
| 401 | Unauthorized | No valid API key provided | Inline |
| 403 | Forbidden | The API key doesn’t have permissions to perform the request | Inline |
| 429 | Too Many Requests | Too many requests hit the API too quickly. We recommend an exponential backoff of your requests | Inline |
| 500 | Internal Server Error | Something went wrong on Braintrust's end. (These are rare.) | Inline |
Response Schema
Status Code 400
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 401
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 403
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 429
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 500
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
🔒
This endpoint requires authentication.
List prompts
Code samples
const headers = {
Accept: "application/json",
Authorization: "Bearer {access-token}",
};
fetch("https://api.braintrustdata.com/v1/prompt", {
method: "GET",
headers: headers,
})
.then(function (res) {
return res.json();
})
.then(function (body) {
console.log(body);
});GET /v1/prompt
List out all prompts. The prompts are sorted by creation date, with the most recently-created prompts coming first
Parameters
| Name | In | Type | Required | Description |
|---|---|---|---|---|
| limit | query | AppLimitParam | false | Limit the number of objects to return |
| starting_after | query | StartingAfter | false | Pagination cursor id. |
| ending_before | query | EndingBefore | false | Pagination cursor id. |
| prompt_name | query | PromptName | false | Name of the prompt to search for |
| project_name | query | ProjectName | false | Name of the project to search for |
| slug | query | Slug | false | Retrieve prompt with a specific slug |
| version | query | any | false | Retrieve prompt at a specific version. |
| org_name | query | OrgName | false | Filter search results to within a particular organization |
| ids | query | Ids | false | Filter search results to a particular set of object IDs. To specify a list of IDs, include the query param multiple times |
Detailed descriptions
starting_after: Pagination cursor id.
For example, if the final item in the last page you fetched had an id of foo, pass starting_after=foo to fetch the next page. Note: you may only pass one of starting_after and ending_before
ending_before: Pagination cursor id.
For example, if the initial item in the last page you fetched had an id of foo, pass ending_before=foo to fetch the previous page. Note: you may only pass one of starting_after and ending_before
version: Retrieve prompt at a specific version.
The version id can either be a transaction id (e.g. '1000192656880881099') or a version identifier (e.g. '81cd05ee665fdfb3').
Example responses
200 Response
{
"objects": [
{
"id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
"_xact_id": "string",
"project_id": "405d8375-3514-403b-8c43-83ae74cfe0e9",
"log_id": "p",
"org_id": "a40f5d1f-d889-42e9-94ea-b9b33585fc6b",
"name": "string",
"slug": "string",
"description": "string",
"created": "2019-08-24T14:15:22Z",
"prompt_data": {
"prompt": {
"type": "completion",
"content": "string"
},
"options": {
"model": "string",
"params": {
"use_cache": true,
"temperature": 0,
"top_p": 0,
"max_tokens": 0,
"frequency_penalty": 0,
"presence_penalty": 0,
"response_format": {
"type": "json_object"
},
"tool_choice": "auto",
"function_call": "auto",
"n": 0,
"stop": ["string"]
},
"position": "string"
},
"origin": {
"prompt_id": "string",
"project_id": "string",
"prompt_version": "string"
}
},
"tags": ["string"],
"metadata": {
"property1": null,
"property2": null
}
}
]
}400 Response
"string"Responses
| Status | Meaning | Description | Schema |
|---|---|---|---|
| 200 | OK | Returns a list of prompt objects | Inline |
| 400 | Bad Request | The request was unacceptable, often due to missing a required parameter | Inline |
| 401 | Unauthorized | No valid API key provided | Inline |
| 403 | Forbidden | The API key doesn’t have permissions to perform the request | Inline |
| 429 | Too Many Requests | Too many requests hit the API too quickly. We recommend an exponential backoff of your requests | Inline |
| 500 | Internal Server Error | Something went wrong on Braintrust's end. (These are rare.) | Inline |
Response Schema
Status Code 200
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| » objects | [Prompt] | true | none | A list of prompt objects |
| »» id | string(uuid) | true | none | Unique identifier for the prompt |
| »» _xact_id | string | true | none | The transaction id of an event is unique to the network operation that processed the event insertion. Transaction ids are monotonically increasing over time and can be used to retrieve a versioned snapshot of the prompt (see the version parameter) |
| »» project_id | string(uuid) | true | none | Unique identifier for the project that the prompt belongs under |
| »» log_id | string | true | none | A literal 'p' which identifies the object as a project prompt |
| »» org_id | string(uuid) | true | none | Unique identifier for the organization |
| »» name | string | true | none | Name of the prompt |
| »» slug | string | true | none | Unique identifier for the prompt |
| »» description | string¦null | false | none | Textual description of the prompt |
| »» created | string(date-time)¦null | false | none | Date of prompt creation |
| »» prompt_data | PromptData¦null | false | none | The prompt, model, and its parameters |
| »»» prompt | any | false | none | none |
anyOf
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| »»»» anonymous | object | false | none | none |
| »»»»» type | string | true | none | none |
| »»»»» content | string | true | none | none |
or
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| »»»» anonymous | object | false | none | none |
| »»»»» type | string | true | none | none |
| »»»»» messages | [anyOf] | true | none | none |
anyOf
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| »»»»»» anonymous | object | false | none | none |
| »»»»»»» content | string | false | none | none |
| »»»»»»» role | string | true | none | none |
| »»»»»»» name | string | false | none | none |
or
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| »»»»»» anonymous | object | false | none | none |
| »»»»»»» content | any | false | none | none |
anyOf
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| »»»»»»»» anonymous | string | false | none | none |
or
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| »»»»»»»» anonymous | [anyOf] | false | none | none |
anyOf
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| »»»»»»»»» anonymous | object | false | none | none |
| »»»»»»»»»» text | string | false | none | none |
| »»»»»»»»»» type | string | true | none | none |
or
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| »»»»»»»»» anonymous | object | false | none | none |
| »»»»»»»»»» image_url | object | true | none | none |
| »»»»»»»»»»» url | string | true | none | none |
| »»»»»»»»»»» detail | any | false | none | none |
anyOf
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| »»»»»»»»»»»» anonymous | string | false | none | none |
or
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| »»»»»»»»»»»» anonymous | string | false | none | none |
or
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| »»»»»»»»»»»» anonymous | string | false | none | none |
continued
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| »»»»»»»»»» type | string | true | none | none |
| »»»»»»» role | string | true | none | none |
| »»»»»»» name | string | false | none | none |
or
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| »»»»»» anonymous | object | false | none | none |
| »»»»»»» role | string | true | none | none |
| »»»»»»» content | string¦null | false | none | none |
| »»»»»»» function_call | object | false | none | none |
| »»»»»»»» arguments | string | true | none | none |
| »»»»»»»» name | string | true | none | none |
| »»»»»»» name | string | false | none | none |
| »»»»»»» tool_calls | [object] | false | none | none |
| »»»»»»»» id | string | true | none | none |
| »»»»»»»» function | object | true | none | none |
| »»»»»»»»» arguments | string | true | none | none |
| »»»»»»»»» name | string | true | none | none |
| »»»»»»»» type | string | true | none | none |
or
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| »»»»»» anonymous | object | false | none | none |
| »»»»»»» content | string | false | none | none |
| »»»»»»» role | string | true | none | none |
| »»»»»»» tool_call_id | string | true | none | none |
or
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| »»»»»» anonymous | object | false | none | none |
| »»»»»»» content | string | false | none | none |
| »»»»»»» name | string | true | none | none |
| »»»»»»» role | string | true | none | none |
continued
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| »»»»» tools | string | false | none | none |
or
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| »»»» anonymous | object¦null | false | none | none |
continued
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| »»» options | object¦null | false | none | none |
| »»»» model | string | false | none | none |
| »»»» params | any | false | none | none |
anyOf
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| »»»»» anonymous | object | false | none | none |
| »»»»»» use_cache | boolean | false | none | none |
| »»»»»» temperature | number | false | none | none |
| »»»»»» top_p | number | false | none | none |
| »»»»»» max_tokens | number | false | none | none |
| »»»»»» frequency_penalty | number | false | none | none |
| »»»»»» presence_penalty | number | false | none | none |
| »»»»»» response_format | object¦null | false | none | none |
| »»»»»»» type | string | true | none | none |
| »»»»»» tool_choice | any | false | none | none |
anyOf
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| »»»»»»» anonymous | string | false | none | none |
or
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| »»»»»»» anonymous | string | false | none | none |
or
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| »»»»»»» anonymous | object | false | none | none |
| »»»»»»»» type | string | true | none | none |
| »»»»»»»» function | object | true | none | none |
| »»»»»»»»» name | string | true | none | none |
continued
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| »»»»»» function_call | any | false | none | none |
anyOf
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| »»»»»»» anonymous | string | false | none | none |
or
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| »»»»»»» anonymous | string | false | none | none |
or
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| »»»»»»» anonymous | object | false | none | none |
| »»»»»»»» name | string | true | none | none |
continued
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| »»»»»» n | number | false | none | none |
| »»»»»» stop | [string] | false | none | none |
or
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| »»»»» anonymous | object | false | none | none |
| »»»»»» use_cache | boolean | false | none | none |
| »»»»»» max_tokens | number | true | none | none |
| »»»»»» temperature | number | true | none | none |
| »»»»»» top_p | number | false | none | none |
| »»»»»» top_k | number | false | none | none |
| »»»»»» stop_sequences | [string] | false | none | none |
| »»»»»» max_tokens_to_sample | number | false | none | This is a legacy parameter that should not be used. |
or
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| »»»»» anonymous | object | false | none | none |
| »»»»»» use_cache | boolean | false | none | none |
| »»»»»» temperature | number | true | none | none |
| »»»»»» maxOutputTokens | number | false | none | none |
| »»»»»» topP | number | false | none | none |
| »»»»»» topK | number | false | none | none |
or
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| »»»»» anonymous | object | false | none | none |
| »»»»»» use_cache | boolean | false | none | none |
continued
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| »»»» position | string | false | none | none |
| »»» origin | object¦null | false | none | none |
| »»»» prompt_id | string | false | none | none |
| »»»» project_id | string | false | none | none |
| »»»» prompt_version | string | false | none | none |
| »» tags | [string]¦null | false | none | A list of tags for the prompt |
| »» metadata | object¦null | false | none | User-controlled metadata about the prompt |
| »»» additionalProperties | any | false | none | none |
Enumerated Values
| Property | Value |
|---|---|
| log_id | p |
| type | completion |
| type | chat |
| role | system |
| type | text |
| anonymous | auto |
| anonymous | low |
| anonymous | high |
| type | image_url |
| role | user |
| role | assistant |
| type | function |
| role | tool |
| role | function |
| type | json_object |
| anonymous | auto |
| anonymous | none |
| type | function |
| anonymous | auto |
| anonymous | none |
Status Code 400
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 401
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 403
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 429
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 500
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
🔒
This endpoint requires authentication.
Get prompt
Code samples
const headers = {
Accept: "application/json",
Authorization: "Bearer {access-token}",
};
fetch("https://api.braintrustdata.com/v1/prompt/{prompt_id}", {
method: "GET",
headers: headers,
})
.then(function (res) {
return res.json();
})
.then(function (body) {
console.log(body);
});GET /v1/prompt/{prompt_id}
Get a prompt object by its id
Parameters
| Name | In | Type | Required | Description |
|---|---|---|---|---|
| prompt_id | path | PromptId | true | Prompt id |
Example responses
200 Response
{
"id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
"_xact_id": "string",
"project_id": "405d8375-3514-403b-8c43-83ae74cfe0e9",
"log_id": "p",
"org_id": "a40f5d1f-d889-42e9-94ea-b9b33585fc6b",
"name": "string",
"slug": "string",
"description": "string",
"created": "2019-08-24T14:15:22Z",
"prompt_data": {
"prompt": {
"type": "completion",
"content": "string"
},
"options": {
"model": "string",
"params": {
"use_cache": true,
"temperature": 0,
"top_p": 0,
"max_tokens": 0,
"frequency_penalty": 0,
"presence_penalty": 0,
"response_format": {
"type": "json_object"
},
"tool_choice": "auto",
"function_call": "auto",
"n": 0,
"stop": ["string"]
},
"position": "string"
},
"origin": {
"prompt_id": "string",
"project_id": "string",
"prompt_version": "string"
}
},
"tags": ["string"],
"metadata": {
"property1": null,
"property2": null
}
}400 Response
"string"Responses
| Status | Meaning | Description | Schema |
|---|---|---|---|
| 200 | OK | Returns the prompt object | Prompt |
| 400 | Bad Request | The request was unacceptable, often due to missing a required parameter | Inline |
| 401 | Unauthorized | No valid API key provided | Inline |
| 403 | Forbidden | The API key doesn’t have permissions to perform the request | Inline |
| 429 | Too Many Requests | Too many requests hit the API too quickly. We recommend an exponential backoff of your requests | Inline |
| 500 | Internal Server Error | Something went wrong on Braintrust's end. (These are rare.) | Inline |
Response Schema
Status Code 400
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 401
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 403
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 429
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 500
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
🔒
This endpoint requires authentication.
Partially update prompt
Code samples
const inputBody = JSON.stringify({
name: "string",
description: "string",
prompt_data: {
prompt: {
type: "completion",
content: "string",
},
options: {
model: "string",
params: {
use_cache: true,
temperature: 0,
top_p: 0,
max_tokens: 0,
frequency_penalty: 0,
presence_penalty: 0,
response_format: {
type: "json_object",
},
tool_choice: "auto",
function_call: "auto",
n: 0,
stop: ["string"],
},
position: "string",
},
origin: {
prompt_id: "string",
project_id: "string",
prompt_version: "string",
},
},
tags: ["string"],
});
const headers = {
"Content-Type": "application/json",
Accept: "application/json",
Authorization: "Bearer {access-token}",
};
fetch("https://api.braintrustdata.com/v1/prompt/{prompt_id}", {
method: "PATCH",
body: inputBody,
headers: headers,
})
.then(function (res) {
return res.json();
})
.then(function (body) {
console.log(body);
});PATCH /v1/prompt/{prompt_id}
Partially update a prompt object. Specify the fields to update in the payload. Any object-type fields will be deep-merged with existing content. Currently we do not support removing fields or setting them to null.
Body parameter
{
"name": "string",
"description": "string",
"prompt_data": {
"prompt": {
"type": "completion",
"content": "string"
},
"options": {
"model": "string",
"params": {
"use_cache": true,
"temperature": 0,
"top_p": 0,
"max_tokens": 0,
"frequency_penalty": 0,
"presence_penalty": 0,
"response_format": {
"type": "json_object"
},
"tool_choice": "auto",
"function_call": "auto",
"n": 0,
"stop": ["string"]
},
"position": "string"
},
"origin": {
"prompt_id": "string",
"project_id": "string",
"prompt_version": "string"
}
},
"tags": ["string"]
}Parameters
| Name | In | Type | Required | Description |
|---|---|---|---|---|
| prompt_id | path | PromptId | true | Prompt id |
| body | body | PatchPrompt | false | Fields to update |
Example responses
200 Response
{
"id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
"_xact_id": "string",
"project_id": "405d8375-3514-403b-8c43-83ae74cfe0e9",
"log_id": "p",
"org_id": "a40f5d1f-d889-42e9-94ea-b9b33585fc6b",
"name": "string",
"slug": "string",
"description": "string",
"created": "2019-08-24T14:15:22Z",
"prompt_data": {
"prompt": {
"type": "completion",
"content": "string"
},
"options": {
"model": "string",
"params": {
"use_cache": true,
"temperature": 0,
"top_p": 0,
"max_tokens": 0,
"frequency_penalty": 0,
"presence_penalty": 0,
"response_format": {
"type": "json_object"
},
"tool_choice": "auto",
"function_call": "auto",
"n": 0,
"stop": ["string"]
},
"position": "string"
},
"origin": {
"prompt_id": "string",
"project_id": "string",
"prompt_version": "string"
}
},
"tags": ["string"],
"metadata": {
"property1": null,
"property2": null
}
}400 Response
"string"Responses
| Status | Meaning | Description | Schema |
|---|---|---|---|
| 200 | OK | Returns the prompt object | Prompt |
| 400 | Bad Request | The request was unacceptable, often due to missing a required parameter | Inline |
| 401 | Unauthorized | No valid API key provided | Inline |
| 403 | Forbidden | The API key doesn’t have permissions to perform the request | Inline |
| 429 | Too Many Requests | Too many requests hit the API too quickly. We recommend an exponential backoff of your requests | Inline |
| 500 | Internal Server Error | Something went wrong on Braintrust's end. (These are rare.) | Inline |
Response Schema
Status Code 400
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 401
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 403
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 429
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 500
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
🔒
This endpoint requires authentication.
Delete prompt
Code samples
const headers = {
Accept: "application/json",
Authorization: "Bearer {access-token}",
};
fetch("https://api.braintrustdata.com/v1/prompt/{prompt_id}", {
method: "DELETE",
headers: headers,
})
.then(function (res) {
return res.json();
})
.then(function (body) {
console.log(body);
});DELETE /v1/prompt/{prompt_id}
Delete a prompt object by its id
Parameters
| Name | In | Type | Required | Description |
|---|---|---|---|---|
| prompt_id | path | PromptId | true | Prompt id |
Example responses
200 Response
{
"id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
"_xact_id": "string",
"project_id": "405d8375-3514-403b-8c43-83ae74cfe0e9",
"log_id": "p",
"org_id": "a40f5d1f-d889-42e9-94ea-b9b33585fc6b",
"name": "string",
"slug": "string",
"description": "string",
"created": "2019-08-24T14:15:22Z",
"prompt_data": {
"prompt": {
"type": "completion",
"content": "string"
},
"options": {
"model": "string",
"params": {
"use_cache": true,
"temperature": 0,
"top_p": 0,
"max_tokens": 0,
"frequency_penalty": 0,
"presence_penalty": 0,
"response_format": {
"type": "json_object"
},
"tool_choice": "auto",
"function_call": "auto",
"n": 0,
"stop": ["string"]
},
"position": "string"
},
"origin": {
"prompt_id": "string",
"project_id": "string",
"prompt_version": "string"
}
},
"tags": ["string"],
"metadata": {
"property1": null,
"property2": null
}
}400 Response
"string"Responses
| Status | Meaning | Description | Schema |
|---|---|---|---|
| 200 | OK | Returns the deleted prompt object | Prompt |
| 400 | Bad Request | The request was unacceptable, often due to missing a required parameter | Inline |
| 401 | Unauthorized | No valid API key provided | Inline |
| 403 | Forbidden | The API key doesn’t have permissions to perform the request | Inline |
| 429 | Too Many Requests | Too many requests hit the API too quickly. We recommend an exponential backoff of your requests | Inline |
| 500 | Internal Server Error | Something went wrong on Braintrust's end. (These are rare.) | Inline |
Response Schema
Status Code 400
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 401
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 403
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 429
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 500
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
🔒
This endpoint requires authentication.
Feedback for prompt events
Code samples
const inputBody = JSON.stringify({
feedback: [
{
id: "string",
comment: "string",
metadata: {
property1: null,
property2: null,
},
source: "app",
},
],
});
const headers = {
"Content-Type": "application/json",
Accept: "text/plain",
Authorization: "Bearer {access-token}",
};
fetch("https://api.braintrustdata.com/v1/prompt/{prompt_id}/feedback", {
method: "POST",
body: inputBody,
headers: headers,
})
.then(function (res) {
return res.json();
})
.then(function (body) {
console.log(body);
});POST /v1/prompt/{prompt_id}/feedback
Log feedback for a set of prompt events
Body parameter
{
"feedback": [
{
"id": "string",
"comment": "string",
"metadata": {
"property1": null,
"property2": null
},
"source": "app"
}
]
}Parameters
| Name | In | Type | Required | Description |
|---|---|---|---|---|
| prompt_id | path | PromptId | true | Prompt id |
| body | body | FeedbackPromptEventRequest | true | An array of feedback objects |
Example responses
400 Response
"string"nullResponses
| Status | Meaning | Description | Schema |
|---|---|---|---|
| 200 | OK | No return value | None |
| 400 | Bad Request | The request was unacceptable, often due to missing a required parameter | Inline |
| 401 | Unauthorized | No valid API key provided | Inline |
| 403 | Forbidden | The API key doesn’t have permissions to perform the request | Inline |
| 429 | Too Many Requests | Too many requests hit the API too quickly. We recommend an exponential backoff of your requests | Inline |
| 500 | Internal Server Error | Something went wrong on Braintrust's end. (These are rare.) | Inline |
Response Schema
Status Code 400
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 401
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 403
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 429
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 500
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
🔒
This endpoint requires authentication.
Roles
Create role
Code samples
const inputBody = JSON.stringify({
name: "string",
description: "string",
member_permissions: ["create"],
member_roles: ["497f6eca-6276-4993-bfeb-53cbbbba6f08"],
org_name: "string",
});
const headers = {
"Content-Type": "application/json",
Accept: "application/json",
Authorization: "Bearer {access-token}",
};
fetch("https://api.braintrustdata.com/v1/role", {
method: "POST",
body: inputBody,
headers: headers,
})
.then(function (res) {
return res.json();
})
.then(function (body) {
console.log(body);
});POST /v1/role
Create a new role. If there is an existing role with the same name as the one specified in the request, will return the existing role unmodified
Body parameter
{
"name": "string",
"description": "string",
"member_permissions": ["create"],
"member_roles": ["497f6eca-6276-4993-bfeb-53cbbbba6f08"],
"org_name": "string"
}Parameters
| Name | In | Type | Required | Description |
|---|---|---|---|---|
| body | body | CreateRole | true | Any desired information about the new role object |
Example responses
200 Response
{
"id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
"org_id": "a40f5d1f-d889-42e9-94ea-b9b33585fc6b",
"user_id": "a169451c-8525-4352-b8ca-070dd449a1a5",
"created": "2019-08-24T14:15:22Z",
"name": "string",
"description": "string",
"deleted_at": "2019-08-24T14:15:22Z",
"member_permissions": ["create"],
"member_roles": ["497f6eca-6276-4993-bfeb-53cbbbba6f08"]
}400 Response
"string"Responses
| Status | Meaning | Description | Schema |
|---|---|---|---|
| 200 | OK | Returns the new role object | Role |
| 400 | Bad Request | The request was unacceptable, often due to missing a required parameter | Inline |
| 401 | Unauthorized | No valid API key provided | Inline |
| 403 | Forbidden | The API key doesn’t have permissions to perform the request | Inline |
| 429 | Too Many Requests | Too many requests hit the API too quickly. We recommend an exponential backoff of your requests | Inline |
| 500 | Internal Server Error | Something went wrong on Braintrust's end. (These are rare.) | Inline |
Response Schema
Status Code 400
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 401
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 403
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 429
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 500
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
🔒
This endpoint requires authentication.
DEPRECATED. Create or replace role
Code samples
const inputBody = JSON.stringify({
name: "string",
description: "string",
member_permissions: ["create"],
member_roles: ["497f6eca-6276-4993-bfeb-53cbbbba6f08"],
org_name: "string",
});
const headers = {
"Content-Type": "application/json",
Accept: "application/json",
Authorization: "Bearer {access-token}",
};
fetch("https://api.braintrustdata.com/v1/role", {
method: "PUT",
body: inputBody,
headers: headers,
})
.then(function (res) {
return res.json();
})
.then(function (body) {
console.log(body);
});PUT /v1/role
NOTE: This operation is deprecated and will be removed in a future revision of the API. Create or replace a new role. If there is an existing role with the same name as the one specified in the request, will return the existing role unmodified, will replace the existing role with the provided fields
Body parameter
{
"name": "string",
"description": "string",
"member_permissions": ["create"],
"member_roles": ["497f6eca-6276-4993-bfeb-53cbbbba6f08"],
"org_name": "string"
}Parameters
| Name | In | Type | Required | Description |
|---|---|---|---|---|
| body | body | CreateRole | true | Any desired information about the new role object |
Example responses
200 Response
{
"id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
"org_id": "a40f5d1f-d889-42e9-94ea-b9b33585fc6b",
"user_id": "a169451c-8525-4352-b8ca-070dd449a1a5",
"created": "2019-08-24T14:15:22Z",
"name": "string",
"description": "string",
"deleted_at": "2019-08-24T14:15:22Z",
"member_permissions": ["create"],
"member_roles": ["497f6eca-6276-4993-bfeb-53cbbbba6f08"]
}400 Response
"string"Responses
| Status | Meaning | Description | Schema |
|---|---|---|---|
| 200 | OK | Returns the new role object | Role |
| 400 | Bad Request | The request was unacceptable, often due to missing a required parameter | Inline |
| 401 | Unauthorized | No valid API key provided | Inline |
| 403 | Forbidden | The API key doesn’t have permissions to perform the request | Inline |
| 429 | Too Many Requests | Too many requests hit the API too quickly. We recommend an exponential backoff of your requests | Inline |
| 500 | Internal Server Error | Something went wrong on Braintrust's end. (These are rare.) | Inline |
Response Schema
Status Code 400
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 401
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 403
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 429
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 500
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
🔒
This endpoint requires authentication.
List roles
Code samples
const headers = {
Accept: "application/json",
Authorization: "Bearer {access-token}",
};
fetch("https://api.braintrustdata.com/v1/role", {
method: "GET",
headers: headers,
})
.then(function (res) {
return res.json();
})
.then(function (body) {
console.log(body);
});GET /v1/role
List out all roles. The roles are sorted by creation date, with the most recently-created roles coming first
Parameters
| Name | In | Type | Required | Description |
|---|---|---|---|---|
| limit | query | AppLimitParam | false | Limit the number of objects to return |
| starting_after | query | StartingAfter | false | Pagination cursor id. |
| ending_before | query | EndingBefore | false | Pagination cursor id. |
| role_name | query | RoleName | false | Name of the role to search for |
| org_name | query | OrgName | false | Filter search results to within a particular organization |
| ids | query | Ids | false | Filter search results to a particular set of object IDs. To specify a list of IDs, include the query param multiple times |
Detailed descriptions
starting_after: Pagination cursor id.
For example, if the final item in the last page you fetched had an id of foo, pass starting_after=foo to fetch the next page. Note: you may only pass one of starting_after and ending_before
ending_before: Pagination cursor id.
For example, if the initial item in the last page you fetched had an id of foo, pass ending_before=foo to fetch the previous page. Note: you may only pass one of starting_after and ending_before
Example responses
200 Response
{
"objects": [
{
"id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
"org_id": "a40f5d1f-d889-42e9-94ea-b9b33585fc6b",
"user_id": "a169451c-8525-4352-b8ca-070dd449a1a5",
"created": "2019-08-24T14:15:22Z",
"name": "string",
"description": "string",
"deleted_at": "2019-08-24T14:15:22Z",
"member_permissions": ["create"],
"member_roles": ["497f6eca-6276-4993-bfeb-53cbbbba6f08"]
}
]
}400 Response
"string"Responses
| Status | Meaning | Description | Schema |
|---|---|---|---|
| 200 | OK | Returns a list of role objects | Inline |
| 400 | Bad Request | The request was unacceptable, often due to missing a required parameter | Inline |
| 401 | Unauthorized | No valid API key provided | Inline |
| 403 | Forbidden | The API key doesn’t have permissions to perform the request | Inline |
| 429 | Too Many Requests | Too many requests hit the API too quickly. We recommend an exponential backoff of your requests | Inline |
| 500 | Internal Server Error | Something went wrong on Braintrust's end. (These are rare.) | Inline |
Response Schema
Status Code 200
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| » objects | [Role] | true | none | A list of role objects |
| »» id | string(uuid) | true | none | Unique identifier for the role |
| »» org_id | string(uuid)¦null | false | none | Unique id for the organization that the role belongs under A null org_id indicates a system role, which may be assigned to anybody and inherited by any other role, but cannot be edited. It is forbidden to change the org after creating a role |
| »» user_id | string(uuid)¦null | false | none | Identifies the user who created the role |
| »» created | string(date-time)¦null | false | none | Date of role creation |
| »» name | string | true | none | Name of the role |
| »» description | string¦null | false | none | Textual description of the role |
| »» deleted_at | string(date-time)¦null | false | none | Date of role deletion, or null if the role is still active |
| »» member_permissions | [string]¦null | false | none | Permissions which belong to this role |
| »» member_roles | [string]¦null | false | none | Ids of the roles this role inherits from An inheriting role has all the permissions contained in its member roles, as well as all of their inherited permissions |
Status Code 400
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 401
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 403
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 429
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 500
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
🔒
This endpoint requires authentication.
Get role
Code samples
const headers = {
Accept: "application/json",
Authorization: "Bearer {access-token}",
};
fetch("https://api.braintrustdata.com/v1/role/{role_id}", {
method: "GET",
headers: headers,
})
.then(function (res) {
return res.json();
})
.then(function (body) {
console.log(body);
});GET /v1/role/{role_id}
Get a role object by its id
Parameters
| Name | In | Type | Required | Description |
|---|---|---|---|---|
| role_id | path | RoleId | true | Role id |
Example responses
200 Response
{
"id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
"org_id": "a40f5d1f-d889-42e9-94ea-b9b33585fc6b",
"user_id": "a169451c-8525-4352-b8ca-070dd449a1a5",
"created": "2019-08-24T14:15:22Z",
"name": "string",
"description": "string",
"deleted_at": "2019-08-24T14:15:22Z",
"member_permissions": ["create"],
"member_roles": ["497f6eca-6276-4993-bfeb-53cbbbba6f08"]
}400 Response
"string"Responses
| Status | Meaning | Description | Schema |
|---|---|---|---|
| 200 | OK | Returns the role object | Role |
| 400 | Bad Request | The request was unacceptable, often due to missing a required parameter | Inline |
| 401 | Unauthorized | No valid API key provided | Inline |
| 403 | Forbidden | The API key doesn’t have permissions to perform the request | Inline |
| 429 | Too Many Requests | Too many requests hit the API too quickly. We recommend an exponential backoff of your requests | Inline |
| 500 | Internal Server Error | Something went wrong on Braintrust's end. (These are rare.) | Inline |
Response Schema
Status Code 400
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 401
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 403
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 429
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 500
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
🔒
This endpoint requires authentication.
Partially update role
Code samples
const inputBody = JSON.stringify({
description: "string",
member_permissions: ["create"],
member_roles: ["497f6eca-6276-4993-bfeb-53cbbbba6f08"],
name: "string",
});
const headers = {
"Content-Type": "application/json",
Accept: "application/json",
Authorization: "Bearer {access-token}",
};
fetch("https://api.braintrustdata.com/v1/role/{role_id}", {
method: "PATCH",
body: inputBody,
headers: headers,
})
.then(function (res) {
return res.json();
})
.then(function (body) {
console.log(body);
});PATCH /v1/role/{role_id}
Partially update a role object. Specify the fields to update in the payload. Any object-type fields will be deep-merged with existing content. Currently we do not support removing fields or setting them to null.
Body parameter
{
"description": "string",
"member_permissions": ["create"],
"member_roles": ["497f6eca-6276-4993-bfeb-53cbbbba6f08"],
"name": "string"
}Parameters
| Name | In | Type | Required | Description |
|---|---|---|---|---|
| role_id | path | RoleId | true | Role id |
| body | body | PatchRole | false | Fields to update |
Example responses
200 Response
{
"id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
"org_id": "a40f5d1f-d889-42e9-94ea-b9b33585fc6b",
"user_id": "a169451c-8525-4352-b8ca-070dd449a1a5",
"created": "2019-08-24T14:15:22Z",
"name": "string",
"description": "string",
"deleted_at": "2019-08-24T14:15:22Z",
"member_permissions": ["create"],
"member_roles": ["497f6eca-6276-4993-bfeb-53cbbbba6f08"]
}400 Response
"string"Responses
| Status | Meaning | Description | Schema |
|---|---|---|---|
| 200 | OK | Returns the role object | Role |
| 400 | Bad Request | The request was unacceptable, often due to missing a required parameter | Inline |
| 401 | Unauthorized | No valid API key provided | Inline |
| 403 | Forbidden | The API key doesn’t have permissions to perform the request | Inline |
| 429 | Too Many Requests | Too many requests hit the API too quickly. We recommend an exponential backoff of your requests | Inline |
| 500 | Internal Server Error | Something went wrong on Braintrust's end. (These are rare.) | Inline |
Response Schema
Status Code 400
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 401
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 403
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 429
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 500
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
🔒
This endpoint requires authentication.
Delete role
Code samples
const headers = {
Accept: "application/json",
Authorization: "Bearer {access-token}",
};
fetch("https://api.braintrustdata.com/v1/role/{role_id}", {
method: "DELETE",
headers: headers,
})
.then(function (res) {
return res.json();
})
.then(function (body) {
console.log(body);
});DELETE /v1/role/{role_id}
Delete a role object by its id
Parameters
| Name | In | Type | Required | Description |
|---|---|---|---|---|
| role_id | path | RoleId | true | Role id |
Example responses
200 Response
{
"id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
"org_id": "a40f5d1f-d889-42e9-94ea-b9b33585fc6b",
"user_id": "a169451c-8525-4352-b8ca-070dd449a1a5",
"created": "2019-08-24T14:15:22Z",
"name": "string",
"description": "string",
"deleted_at": "2019-08-24T14:15:22Z",
"member_permissions": ["create"],
"member_roles": ["497f6eca-6276-4993-bfeb-53cbbbba6f08"]
}400 Response
"string"Responses
| Status | Meaning | Description | Schema |
|---|---|---|---|
| 200 | OK | Returns the deleted role object | Role |
| 400 | Bad Request | The request was unacceptable, often due to missing a required parameter | Inline |
| 401 | Unauthorized | No valid API key provided | Inline |
| 403 | Forbidden | The API key doesn’t have permissions to perform the request | Inline |
| 429 | Too Many Requests | Too many requests hit the API too quickly. We recommend an exponential backoff of your requests | Inline |
| 500 | Internal Server Error | Something went wrong on Braintrust's end. (These are rare.) | Inline |
Response Schema
Status Code 400
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 401
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 403
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 429
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 500
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
🔒
This endpoint requires authentication.
Groups
Create group
Code samples
const inputBody = JSON.stringify({
name: "string",
description: "string",
member_users: ["497f6eca-6276-4993-bfeb-53cbbbba6f08"],
member_groups: ["497f6eca-6276-4993-bfeb-53cbbbba6f08"],
org_name: "string",
});
const headers = {
"Content-Type": "application/json",
Accept: "application/json",
Authorization: "Bearer {access-token}",
};
fetch("https://api.braintrustdata.com/v1/group", {
method: "POST",
body: inputBody,
headers: headers,
})
.then(function (res) {
return res.json();
})
.then(function (body) {
console.log(body);
});POST /v1/group
Create a new group. If there is an existing group with the same name as the one specified in the request, will return the existing group unmodified
Body parameter
{
"name": "string",
"description": "string",
"member_users": ["497f6eca-6276-4993-bfeb-53cbbbba6f08"],
"member_groups": ["497f6eca-6276-4993-bfeb-53cbbbba6f08"],
"org_name": "string"
}Parameters
| Name | In | Type | Required | Description |
|---|---|---|---|---|
| body | body | CreateGroup | true | Any desired information about the new group object |
Example responses
200 Response
{
"id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
"org_id": "a40f5d1f-d889-42e9-94ea-b9b33585fc6b",
"user_id": "a169451c-8525-4352-b8ca-070dd449a1a5",
"created": "2019-08-24T14:15:22Z",
"name": "string",
"description": "string",
"deleted_at": "2019-08-24T14:15:22Z",
"member_users": ["497f6eca-6276-4993-bfeb-53cbbbba6f08"],
"member_groups": ["497f6eca-6276-4993-bfeb-53cbbbba6f08"]
}400 Response
"string"Responses
| Status | Meaning | Description | Schema |
|---|---|---|---|
| 200 | OK | Returns the new group object | Group |
| 400 | Bad Request | The request was unacceptable, often due to missing a required parameter | Inline |
| 401 | Unauthorized | No valid API key provided | Inline |
| 403 | Forbidden | The API key doesn’t have permissions to perform the request | Inline |
| 429 | Too Many Requests | Too many requests hit the API too quickly. We recommend an exponential backoff of your requests | Inline |
| 500 | Internal Server Error | Something went wrong on Braintrust's end. (These are rare.) | Inline |
Response Schema
Status Code 400
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 401
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 403
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 429
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 500
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
🔒
This endpoint requires authentication.
DEPRECATED. Create or replace group
Code samples
const inputBody = JSON.stringify({
name: "string",
description: "string",
member_users: ["497f6eca-6276-4993-bfeb-53cbbbba6f08"],
member_groups: ["497f6eca-6276-4993-bfeb-53cbbbba6f08"],
org_name: "string",
});
const headers = {
"Content-Type": "application/json",
Accept: "application/json",
Authorization: "Bearer {access-token}",
};
fetch("https://api.braintrustdata.com/v1/group", {
method: "PUT",
body: inputBody,
headers: headers,
})
.then(function (res) {
return res.json();
})
.then(function (body) {
console.log(body);
});PUT /v1/group
NOTE: This operation is deprecated and will be removed in a future revision of the API. Create or replace a new group. If there is an existing group with the same name as the one specified in the request, will return the existing group unmodified, will replace the existing group with the provided fields
Body parameter
{
"name": "string",
"description": "string",
"member_users": ["497f6eca-6276-4993-bfeb-53cbbbba6f08"],
"member_groups": ["497f6eca-6276-4993-bfeb-53cbbbba6f08"],
"org_name": "string"
}Parameters
| Name | In | Type | Required | Description |
|---|---|---|---|---|
| body | body | CreateGroup | true | Any desired information about the new group object |
Example responses
200 Response
{
"id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
"org_id": "a40f5d1f-d889-42e9-94ea-b9b33585fc6b",
"user_id": "a169451c-8525-4352-b8ca-070dd449a1a5",
"created": "2019-08-24T14:15:22Z",
"name": "string",
"description": "string",
"deleted_at": "2019-08-24T14:15:22Z",
"member_users": ["497f6eca-6276-4993-bfeb-53cbbbba6f08"],
"member_groups": ["497f6eca-6276-4993-bfeb-53cbbbba6f08"]
}400 Response
"string"Responses
| Status | Meaning | Description | Schema |
|---|---|---|---|
| 200 | OK | Returns the new group object | Group |
| 400 | Bad Request | The request was unacceptable, often due to missing a required parameter | Inline |
| 401 | Unauthorized | No valid API key provided | Inline |
| 403 | Forbidden | The API key doesn’t have permissions to perform the request | Inline |
| 429 | Too Many Requests | Too many requests hit the API too quickly. We recommend an exponential backoff of your requests | Inline |
| 500 | Internal Server Error | Something went wrong on Braintrust's end. (These are rare.) | Inline |
Response Schema
Status Code 400
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 401
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 403
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 429
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 500
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
🔒
This endpoint requires authentication.
List groups
Code samples
const headers = {
Accept: "application/json",
Authorization: "Bearer {access-token}",
};
fetch("https://api.braintrustdata.com/v1/group", {
method: "GET",
headers: headers,
})
.then(function (res) {
return res.json();
})
.then(function (body) {
console.log(body);
});GET /v1/group
List out all groups. The groups are sorted by creation date, with the most recently-created groups coming first
Parameters
| Name | In | Type | Required | Description |
|---|---|---|---|---|
| limit | query | AppLimitParam | false | Limit the number of objects to return |
| starting_after | query | StartingAfter | false | Pagination cursor id. |
| ending_before | query | EndingBefore | false | Pagination cursor id. |
| group_name | query | GroupName | false | Name of the group to search for |
| org_name | query | OrgName | false | Filter search results to within a particular organization |
| ids | query | Ids | false | Filter search results to a particular set of object IDs. To specify a list of IDs, include the query param multiple times |
Detailed descriptions
starting_after: Pagination cursor id.
For example, if the final item in the last page you fetched had an id of foo, pass starting_after=foo to fetch the next page. Note: you may only pass one of starting_after and ending_before
ending_before: Pagination cursor id.
For example, if the initial item in the last page you fetched had an id of foo, pass ending_before=foo to fetch the previous page. Note: you may only pass one of starting_after and ending_before
Example responses
200 Response
{
"objects": [
{
"id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
"org_id": "a40f5d1f-d889-42e9-94ea-b9b33585fc6b",
"user_id": "a169451c-8525-4352-b8ca-070dd449a1a5",
"created": "2019-08-24T14:15:22Z",
"name": "string",
"description": "string",
"deleted_at": "2019-08-24T14:15:22Z",
"member_users": ["497f6eca-6276-4993-bfeb-53cbbbba6f08"],
"member_groups": ["497f6eca-6276-4993-bfeb-53cbbbba6f08"]
}
]
}400 Response
"string"Responses
| Status | Meaning | Description | Schema |
|---|---|---|---|
| 200 | OK | Returns a list of group objects | Inline |
| 400 | Bad Request | The request was unacceptable, often due to missing a required parameter | Inline |
| 401 | Unauthorized | No valid API key provided | Inline |
| 403 | Forbidden | The API key doesn’t have permissions to perform the request | Inline |
| 429 | Too Many Requests | Too many requests hit the API too quickly. We recommend an exponential backoff of your requests | Inline |
| 500 | Internal Server Error | Something went wrong on Braintrust's end. (These are rare.) | Inline |
Response Schema
Status Code 200
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| » objects | [Group] | true | none | A list of group objects |
| »» id | string(uuid) | true | none | Unique identifier for the group |
| »» org_id | string(uuid) | true | none | Unique id for the organization that the group belongs under It is forbidden to change the org after creating a group |
| »» user_id | string(uuid)¦null | false | none | Identifies the user who created the group |
| »» created | string(date-time)¦null | false | none | Date of group creation |
| »» name | string | true | none | Name of the group |
| »» description | string¦null | false | none | Textual description of the group |
| »» deleted_at | string(date-time)¦null | false | none | Date of group deletion, or null if the group is still active |
| »» member_users | [string]¦null | false | none | Ids of users which belong to this group |
| »» member_groups | [string]¦null | false | none | Ids of the groups this group inherits from An inheriting group has all the users contained in its member groups, as well as all of their inherited users |
Status Code 400
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 401
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 403
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 429
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 500
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
🔒
This endpoint requires authentication.
Get group
Code samples
const headers = {
Accept: "application/json",
Authorization: "Bearer {access-token}",
};
fetch("https://api.braintrustdata.com/v1/group/{group_id}", {
method: "GET",
headers: headers,
})
.then(function (res) {
return res.json();
})
.then(function (body) {
console.log(body);
});GET /v1/group/{group_id}
Get a group object by its id
Parameters
| Name | In | Type | Required | Description |
|---|---|---|---|---|
| group_id | path | GroupId | true | Group id |
Example responses
200 Response
{
"id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
"org_id": "a40f5d1f-d889-42e9-94ea-b9b33585fc6b",
"user_id": "a169451c-8525-4352-b8ca-070dd449a1a5",
"created": "2019-08-24T14:15:22Z",
"name": "string",
"description": "string",
"deleted_at": "2019-08-24T14:15:22Z",
"member_users": ["497f6eca-6276-4993-bfeb-53cbbbba6f08"],
"member_groups": ["497f6eca-6276-4993-bfeb-53cbbbba6f08"]
}400 Response
"string"Responses
| Status | Meaning | Description | Schema |
|---|---|---|---|
| 200 | OK | Returns the group object | Group |
| 400 | Bad Request | The request was unacceptable, often due to missing a required parameter | Inline |
| 401 | Unauthorized | No valid API key provided | Inline |
| 403 | Forbidden | The API key doesn’t have permissions to perform the request | Inline |
| 429 | Too Many Requests | Too many requests hit the API too quickly. We recommend an exponential backoff of your requests | Inline |
| 500 | Internal Server Error | Something went wrong on Braintrust's end. (These are rare.) | Inline |
Response Schema
Status Code 400
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 401
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 403
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 429
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 500
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
🔒
This endpoint requires authentication.
Partially update group
Code samples
const inputBody = JSON.stringify({
description: "string",
member_users: ["497f6eca-6276-4993-bfeb-53cbbbba6f08"],
member_groups: ["497f6eca-6276-4993-bfeb-53cbbbba6f08"],
name: "string",
});
const headers = {
"Content-Type": "application/json",
Accept: "application/json",
Authorization: "Bearer {access-token}",
};
fetch("https://api.braintrustdata.com/v1/group/{group_id}", {
method: "PATCH",
body: inputBody,
headers: headers,
})
.then(function (res) {
return res.json();
})
.then(function (body) {
console.log(body);
});PATCH /v1/group/{group_id}
Partially update a group object. Specify the fields to update in the payload. Any object-type fields will be deep-merged with existing content. Currently we do not support removing fields or setting them to null.
Body parameter
{
"description": "string",
"member_users": ["497f6eca-6276-4993-bfeb-53cbbbba6f08"],
"member_groups": ["497f6eca-6276-4993-bfeb-53cbbbba6f08"],
"name": "string"
}Parameters
| Name | In | Type | Required | Description |
|---|---|---|---|---|
| group_id | path | GroupId | true | Group id |
| body | body | PatchGroup | false | Fields to update |
Example responses
200 Response
{
"id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
"org_id": "a40f5d1f-d889-42e9-94ea-b9b33585fc6b",
"user_id": "a169451c-8525-4352-b8ca-070dd449a1a5",
"created": "2019-08-24T14:15:22Z",
"name": "string",
"description": "string",
"deleted_at": "2019-08-24T14:15:22Z",
"member_users": ["497f6eca-6276-4993-bfeb-53cbbbba6f08"],
"member_groups": ["497f6eca-6276-4993-bfeb-53cbbbba6f08"]
}400 Response
"string"Responses
| Status | Meaning | Description | Schema |
|---|---|---|---|
| 200 | OK | Returns the group object | Group |
| 400 | Bad Request | The request was unacceptable, often due to missing a required parameter | Inline |
| 401 | Unauthorized | No valid API key provided | Inline |
| 403 | Forbidden | The API key doesn’t have permissions to perform the request | Inline |
| 429 | Too Many Requests | Too many requests hit the API too quickly. We recommend an exponential backoff of your requests | Inline |
| 500 | Internal Server Error | Something went wrong on Braintrust's end. (These are rare.) | Inline |
Response Schema
Status Code 400
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 401
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 403
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 429
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 500
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
🔒
This endpoint requires authentication.
Delete group
Code samples
const headers = {
Accept: "application/json",
Authorization: "Bearer {access-token}",
};
fetch("https://api.braintrustdata.com/v1/group/{group_id}", {
method: "DELETE",
headers: headers,
})
.then(function (res) {
return res.json();
})
.then(function (body) {
console.log(body);
});DELETE /v1/group/{group_id}
Delete a group object by its id
Parameters
| Name | In | Type | Required | Description |
|---|---|---|---|---|
| group_id | path | GroupId | true | Group id |
Example responses
200 Response
{
"id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
"org_id": "a40f5d1f-d889-42e9-94ea-b9b33585fc6b",
"user_id": "a169451c-8525-4352-b8ca-070dd449a1a5",
"created": "2019-08-24T14:15:22Z",
"name": "string",
"description": "string",
"deleted_at": "2019-08-24T14:15:22Z",
"member_users": ["497f6eca-6276-4993-bfeb-53cbbbba6f08"],
"member_groups": ["497f6eca-6276-4993-bfeb-53cbbbba6f08"]
}400 Response
"string"Responses
| Status | Meaning | Description | Schema |
|---|---|---|---|
| 200 | OK | Returns the deleted group object | Group |
| 400 | Bad Request | The request was unacceptable, often due to missing a required parameter | Inline |
| 401 | Unauthorized | No valid API key provided | Inline |
| 403 | Forbidden | The API key doesn’t have permissions to perform the request | Inline |
| 429 | Too Many Requests | Too many requests hit the API too quickly. We recommend an exponential backoff of your requests | Inline |
| 500 | Internal Server Error | Something went wrong on Braintrust's end. (These are rare.) | Inline |
Response Schema
Status Code 400
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 401
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 403
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 429
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 500
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
🔒
This endpoint requires authentication.
Acls
Create acl
Code samples
const inputBody = JSON.stringify({
object_type: "organization",
object_id: "463a83d0-a816-4902-abba-2486e0c0a0bb",
restrict_object_type: "organization",
user_id: "a169451c-8525-4352-b8ca-070dd449a1a5",
group_id: "306db4e0-7449-4501-b76f-075576fe2d8f",
permission: "create",
role_id: "ac4e70c8-d5be-48af-93eb-760f58fc91a9",
});
const headers = {
"Content-Type": "application/json",
Accept: "application/json",
Authorization: "Bearer {access-token}",
};
fetch("https://api.braintrustdata.com/v1/acl", {
method: "POST",
body: inputBody,
headers: headers,
})
.then(function (res) {
return res.json();
})
.then(function (body) {
console.log(body);
});POST /v1/acl
Create a new acl. If there is an existing acl with the same contents as the one specified in the request, will return the existing acl unmodified
Body parameter
{
"object_type": "organization",
"object_id": "463a83d0-a816-4902-abba-2486e0c0a0bb",
"restrict_object_type": "organization",
"user_id": "a169451c-8525-4352-b8ca-070dd449a1a5",
"group_id": "306db4e0-7449-4501-b76f-075576fe2d8f",
"permission": "create",
"role_id": "ac4e70c8-d5be-48af-93eb-760f58fc91a9"
}Parameters
| Name | In | Type | Required | Description |
|---|---|---|---|---|
| body | body | CreateAcl | true | Any desired information about the new acl object |
Example responses
200 Response
{
"id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
"object_type": "organization",
"object_id": "463a83d0-a816-4902-abba-2486e0c0a0bb",
"restrict_object_type": "organization",
"_object_org_id": "4d272dc1-1d6f-4a99-8c76-7bcbfc11ce4e",
"created": "2019-08-24T14:15:22Z",
"user_id": "a169451c-8525-4352-b8ca-070dd449a1a5",
"group_id": "306db4e0-7449-4501-b76f-075576fe2d8f",
"permission": "create",
"role_id": "ac4e70c8-d5be-48af-93eb-760f58fc91a9"
}400 Response
"string"Responses
| Status | Meaning | Description | Schema |
|---|---|---|---|
| 200 | OK | Returns the new acl object | Acl |
| 400 | Bad Request | The request was unacceptable, often due to missing a required parameter | Inline |
| 401 | Unauthorized | No valid API key provided | Inline |
| 403 | Forbidden | The API key doesn’t have permissions to perform the request | Inline |
| 429 | Too Many Requests | Too many requests hit the API too quickly. We recommend an exponential backoff of your requests | Inline |
| 500 | Internal Server Error | Something went wrong on Braintrust's end. (These are rare.) | Inline |
Response Schema
Status Code 400
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 401
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 403
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 429
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 500
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
🔒
This endpoint requires authentication.
DEPRECATED. Create or replace acl
Code samples
const inputBody = JSON.stringify({
object_type: "organization",
object_id: "463a83d0-a816-4902-abba-2486e0c0a0bb",
restrict_object_type: "organization",
user_id: "a169451c-8525-4352-b8ca-070dd449a1a5",
group_id: "306db4e0-7449-4501-b76f-075576fe2d8f",
permission: "create",
role_id: "ac4e70c8-d5be-48af-93eb-760f58fc91a9",
});
const headers = {
"Content-Type": "application/json",
Accept: "application/json",
Authorization: "Bearer {access-token}",
};
fetch("https://api.braintrustdata.com/v1/acl", {
method: "PUT",
body: inputBody,
headers: headers,
})
.then(function (res) {
return res.json();
})
.then(function (body) {
console.log(body);
});PUT /v1/acl
NOTE: This operation is deprecated and will be removed in a future revision of the API. Create or replace a new acl. If there is an existing acl with the same contents as the one specified in the request, will return the existing acl unmodified, will replace the existing acl with the provided fields
Body parameter
{
"object_type": "organization",
"object_id": "463a83d0-a816-4902-abba-2486e0c0a0bb",
"restrict_object_type": "organization",
"user_id": "a169451c-8525-4352-b8ca-070dd449a1a5",
"group_id": "306db4e0-7449-4501-b76f-075576fe2d8f",
"permission": "create",
"role_id": "ac4e70c8-d5be-48af-93eb-760f58fc91a9"
}Parameters
| Name | In | Type | Required | Description |
|---|---|---|---|---|
| body | body | CreateAcl | true | Any desired information about the new acl object |
Example responses
200 Response
{
"id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
"object_type": "organization",
"object_id": "463a83d0-a816-4902-abba-2486e0c0a0bb",
"restrict_object_type": "organization",
"_object_org_id": "4d272dc1-1d6f-4a99-8c76-7bcbfc11ce4e",
"created": "2019-08-24T14:15:22Z",
"user_id": "a169451c-8525-4352-b8ca-070dd449a1a5",
"group_id": "306db4e0-7449-4501-b76f-075576fe2d8f",
"permission": "create",
"role_id": "ac4e70c8-d5be-48af-93eb-760f58fc91a9"
}400 Response
"string"Responses
| Status | Meaning | Description | Schema |
|---|---|---|---|
| 200 | OK | Returns the new acl object | Acl |
| 400 | Bad Request | The request was unacceptable, often due to missing a required parameter | Inline |
| 401 | Unauthorized | No valid API key provided | Inline |
| 403 | Forbidden | The API key doesn’t have permissions to perform the request | Inline |
| 429 | Too Many Requests | Too many requests hit the API too quickly. We recommend an exponential backoff of your requests | Inline |
| 500 | Internal Server Error | Something went wrong on Braintrust's end. (These are rare.) | Inline |
Response Schema
Status Code 400
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 401
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 403
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 429
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 500
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
🔒
This endpoint requires authentication.
List acls
Code samples
const headers = {
Accept: "application/json",
Authorization: "Bearer {access-token}",
};
fetch(
"https://api.braintrustdata.com/v1/acl?object_type=organization&object_id=497f6eca-6276-4993-bfeb-53cbbbba6f08",
{
method: "GET",
headers: headers,
},
)
.then(function (res) {
return res.json();
})
.then(function (body) {
console.log(body);
});GET /v1/acl
List out all acls. The acls are sorted by creation date, with the most recently-created acls coming first
Parameters
| Name | In | Type | Required | Description |
|---|---|---|---|---|
| limit | query | AppLimitParam | false | Limit the number of objects to return |
| starting_after | query | StartingAfter | false | Pagination cursor id. |
| ending_before | query | EndingBefore | false | Pagination cursor id. |
| object_type | query | AclObjectType | true | The object type that the ACL applies to |
| object_id | query | AclObjectId | true | The id of the object the ACL applies to |
Detailed descriptions
starting_after: Pagination cursor id.
For example, if the final item in the last page you fetched had an id of foo, pass starting_after=foo to fetch the next page. Note: you may only pass one of starting_after and ending_before
ending_before: Pagination cursor id.
For example, if the initial item in the last page you fetched had an id of foo, pass ending_before=foo to fetch the previous page. Note: you may only pass one of starting_after and ending_before
Enumerated Values
| Parameter | Value |
|---|---|
| object_type | organization |
| object_type | project |
| object_type | experiment |
| object_type | dataset |
| object_type | prompt |
| object_type | prompt_session |
| object_type | project_score |
| object_type | project_tag |
| object_type | group |
| object_type | role |
| object_type | null |
Example responses
200 Response
{
"objects": [
{
"id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
"object_type": "organization",
"object_id": "463a83d0-a816-4902-abba-2486e0c0a0bb",
"restrict_object_type": "organization",
"_object_org_id": "4d272dc1-1d6f-4a99-8c76-7bcbfc11ce4e",
"created": "2019-08-24T14:15:22Z",
"user_id": "a169451c-8525-4352-b8ca-070dd449a1a5",
"group_id": "306db4e0-7449-4501-b76f-075576fe2d8f",
"permission": "create",
"role_id": "ac4e70c8-d5be-48af-93eb-760f58fc91a9"
}
]
}400 Response
"string"Responses
| Status | Meaning | Description | Schema |
|---|---|---|---|
| 200 | OK | Returns a list of acl objects | Inline |
| 400 | Bad Request | The request was unacceptable, often due to missing a required parameter | Inline |
| 401 | Unauthorized | No valid API key provided | Inline |
| 403 | Forbidden | The API key doesn’t have permissions to perform the request | Inline |
| 429 | Too Many Requests | Too many requests hit the API too quickly. We recommend an exponential backoff of your requests | Inline |
| 500 | Internal Server Error | Something went wrong on Braintrust's end. (These are rare.) | Inline |
Response Schema
Status Code 200
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| » objects | [Acl] | true | none | A list of acl objects |
| »» id | string(uuid) | true | none | Unique identifier for the acl |
| »» object_type | string | true | none | The object type that the ACL applies to |
| »» object_id | string(uuid) | true | none | The id of the object the ACL applies to |
| »» restrict_object_type | string¦null | false | none | Optionally restricts the permission grant to just the specified object type |
| »» _object_org_id | string(uuid) | true | none | The organization the ACL's referred object belongs to |
| »» created | string(date-time)¦null | false | none | Date of acl creation |
| »» user_id | string(uuid)¦null | false | none | Id of the user the ACL applies to. Exactly one of user_id and group_id will be provided |
| »» group_id | string(uuid)¦null | false | none | Id of the group the ACL applies to. Exactly one of user_id and group_id will be provided |
| »» permission | string¦null | false | none | Permission the ACL grants. Exactly one of permission and role_id will be provided |
| »» role_id | string(uuid)¦null | false | none | Id of the role the ACL grants. Exactly one of permission and role_id will be provided |
Enumerated Values
| Property | Value |
|---|---|
| object_type | organization |
| object_type | project |
| object_type | experiment |
| object_type | dataset |
| object_type | prompt |
| object_type | prompt_session |
| object_type | project_score |
| object_type | project_tag |
| object_type | group |
| object_type | role |
| object_type | null |
| restrict_object_type | organization |
| restrict_object_type | project |
| restrict_object_type | experiment |
| restrict_object_type | dataset |
| restrict_object_type | prompt |
| restrict_object_type | prompt_session |
| restrict_object_type | project_score |
| restrict_object_type | project_tag |
| restrict_object_type | group |
| restrict_object_type | role |
| restrict_object_type | null |
| permission | create |
| permission | read |
| permission | update |
| permission | delete |
| permission | create_acls |
| permission | read_acls |
| permission | update_acls |
| permission | delete_acls |
| permission | null |
Status Code 400
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 401
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 403
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 429
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 500
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
🔒
This endpoint requires authentication.
Get acl
Code samples
const headers = {
Accept: "application/json",
Authorization: "Bearer {access-token}",
};
fetch("https://api.braintrustdata.com/v1/acl/{acl_id}", {
method: "GET",
headers: headers,
})
.then(function (res) {
return res.json();
})
.then(function (body) {
console.log(body);
});GET /v1/acl/{acl_id}
Get an acl object by its id
Parameters
| Name | In | Type | Required | Description |
|---|---|---|---|---|
| acl_id | path | AclId | true | Acl id |
Example responses
200 Response
{
"id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
"object_type": "organization",
"object_id": "463a83d0-a816-4902-abba-2486e0c0a0bb",
"restrict_object_type": "organization",
"_object_org_id": "4d272dc1-1d6f-4a99-8c76-7bcbfc11ce4e",
"created": "2019-08-24T14:15:22Z",
"user_id": "a169451c-8525-4352-b8ca-070dd449a1a5",
"group_id": "306db4e0-7449-4501-b76f-075576fe2d8f",
"permission": "create",
"role_id": "ac4e70c8-d5be-48af-93eb-760f58fc91a9"
}400 Response
"string"Responses
| Status | Meaning | Description | Schema |
|---|---|---|---|
| 200 | OK | Returns the acl object | Acl |
| 400 | Bad Request | The request was unacceptable, often due to missing a required parameter | Inline |
| 401 | Unauthorized | No valid API key provided | Inline |
| 403 | Forbidden | The API key doesn’t have permissions to perform the request | Inline |
| 429 | Too Many Requests | Too many requests hit the API too quickly. We recommend an exponential backoff of your requests | Inline |
| 500 | Internal Server Error | Something went wrong on Braintrust's end. (These are rare.) | Inline |
Response Schema
Status Code 400
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 401
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 403
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 429
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 500
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
🔒
This endpoint requires authentication.
Delete acl
Code samples
const headers = {
Accept: "application/json",
Authorization: "Bearer {access-token}",
};
fetch("https://api.braintrustdata.com/v1/acl/{acl_id}", {
method: "DELETE",
headers: headers,
})
.then(function (res) {
return res.json();
})
.then(function (body) {
console.log(body);
});DELETE /v1/acl/{acl_id}
Delete an acl object by its id
Parameters
| Name | In | Type | Required | Description |
|---|---|---|---|---|
| acl_id | path | AclId | true | Acl id |
Example responses
200 Response
{
"id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
"object_type": "organization",
"object_id": "463a83d0-a816-4902-abba-2486e0c0a0bb",
"restrict_object_type": "organization",
"_object_org_id": "4d272dc1-1d6f-4a99-8c76-7bcbfc11ce4e",
"created": "2019-08-24T14:15:22Z",
"user_id": "a169451c-8525-4352-b8ca-070dd449a1a5",
"group_id": "306db4e0-7449-4501-b76f-075576fe2d8f",
"permission": "create",
"role_id": "ac4e70c8-d5be-48af-93eb-760f58fc91a9"
}400 Response
"string"Responses
| Status | Meaning | Description | Schema |
|---|---|---|---|
| 200 | OK | Returns the deleted acl object | Acl |
| 400 | Bad Request | The request was unacceptable, often due to missing a required parameter | Inline |
| 401 | Unauthorized | No valid API key provided | Inline |
| 403 | Forbidden | The API key doesn’t have permissions to perform the request | Inline |
| 429 | Too Many Requests | Too many requests hit the API too quickly. We recommend an exponential backoff of your requests | Inline |
| 500 | Internal Server Error | Something went wrong on Braintrust's end. (These are rare.) | Inline |
Response Schema
Status Code 400
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 401
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 403
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 429
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 500
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
🔒
This endpoint requires authentication.
Users
List users
Code samples
const headers = {
Accept: "application/json",
Authorization: "Bearer {access-token}",
};
fetch("https://api.braintrustdata.com/v1/user", {
method: "GET",
headers: headers,
})
.then(function (res) {
return res.json();
})
.then(function (body) {
console.log(body);
});GET /v1/user
List out all users. The users are sorted by creation date, with the most recently-created users coming first
Parameters
| Name | In | Type | Required | Description |
|---|---|---|---|---|
| limit | query | AppLimitParam | false | Limit the number of objects to return |
| starting_after | query | StartingAfter | false | Pagination cursor id. |
| ending_before | query | EndingBefore | false | Pagination cursor id. |
| given_name | query | UserGivenName | false | Given name of the user to search for |
| family_name | query | UserFamilyName | false | Family name of the user to search for |
| query | UserEmail | false | Email of the user to search for | |
| org_name | query | OrgName | false | Filter search results to within a particular organization |
| ids | query | Ids | false | Filter search results to a particular set of object IDs. To specify a list of IDs, include the query param multiple times |
Detailed descriptions
starting_after: Pagination cursor id.
For example, if the final item in the last page you fetched had an id of foo, pass starting_after=foo to fetch the next page. Note: you may only pass one of starting_after and ending_before
ending_before: Pagination cursor id.
For example, if the initial item in the last page you fetched had an id of foo, pass ending_before=foo to fetch the previous page. Note: you may only pass one of starting_after and ending_before
Example responses
200 Response
{
"objects": [
{
"id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
"given_name": "string",
"family_name": "string",
"email": "string",
"avatar_url": "string",
"created": "2019-08-24T14:15:22Z"
}
]
}400 Response
"string"Responses
| Status | Meaning | Description | Schema |
|---|---|---|---|
| 200 | OK | Returns a list of user objects | Inline |
| 400 | Bad Request | The request was unacceptable, often due to missing a required parameter | Inline |
| 401 | Unauthorized | No valid API key provided | Inline |
| 403 | Forbidden | The API key doesn’t have permissions to perform the request | Inline |
| 429 | Too Many Requests | Too many requests hit the API too quickly. We recommend an exponential backoff of your requests | Inline |
| 500 | Internal Server Error | Something went wrong on Braintrust's end. (These are rare.) | Inline |
Response Schema
Status Code 200
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| » objects | [User] | true | none | A list of user objects |
| »» id | string(uuid) | true | none | Unique identifier for the user |
| »» given_name | string¦null | false | none | Given name of the user |
| »» family_name | string¦null | false | none | Family name of the user |
| string¦null | false | none | The user's email | |
| »» avatar_url | string¦null | false | none | URL of the user's Avatar image |
| »» created | string(date-time)¦null | false | none | Date of user creation |
Status Code 400
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 401
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 403
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 429
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 500
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
🔒
This endpoint requires authentication.
Get user
Code samples
const headers = {
Accept: "application/json",
Authorization: "Bearer {access-token}",
};
fetch("https://api.braintrustdata.com/v1/user/{user_id}", {
method: "GET",
headers: headers,
})
.then(function (res) {
return res.json();
})
.then(function (body) {
console.log(body);
});GET /v1/user/{user_id}
Get a user object by its id
Parameters
| Name | In | Type | Required | Description |
|---|---|---|---|---|
| user_id | path | UserId | true | User id |
Example responses
200 Response
{
"id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
"given_name": "string",
"family_name": "string",
"email": "string",
"avatar_url": "string",
"created": "2019-08-24T14:15:22Z"
}400 Response
"string"Responses
| Status | Meaning | Description | Schema |
|---|---|---|---|
| 200 | OK | Returns the user object | User |
| 400 | Bad Request | The request was unacceptable, often due to missing a required parameter | Inline |
| 401 | Unauthorized | No valid API key provided | Inline |
| 403 | Forbidden | The API key doesn’t have permissions to perform the request | Inline |
| 429 | Too Many Requests | Too many requests hit the API too quickly. We recommend an exponential backoff of your requests | Inline |
| 500 | Internal Server Error | Something went wrong on Braintrust's end. (These are rare.) | Inline |
Response Schema
Status Code 400
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 401
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 403
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 429
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 500
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
🔒
This endpoint requires authentication.
Other
Hello world endpoint
Code samples
const headers = {
Accept: "text/plain",
};
fetch("https://api.braintrustdata.com/v1", {
method: "GET",
headers: headers,
})
.then(function (res) {
return res.json();
})
.then(function (body) {
console.log(body);
});GET /v1
Default endpoint. Simply replies with 'Hello, World!'. Authorization is not required
Example responses
200 Response
"string"400 Response
nullResponses
| Status | Meaning | Description | Schema |
|---|---|---|---|
| 200 | OK | Hello world string | string |
| 400 | Bad Request | The request was unacceptable, often due to missing a required parameter | Inline |
| 500 | Internal Server Error | Something went wrong on Braintrust's end. (These are rare.) | Inline |
Response Schema
Status Code 400
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 500
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
🌐
This operation does not require authentication
CrossObject
Cross-object insert
Code samples
const inputBody = JSON.stringify({
experiment: {
property1: {
events: [
{
input: null,
output: null,
expected: null,
scores: {
property1: 1,
property2: 1,
},
metadata: {
property1: null,
property2: null,
},
tags: ["string"],
metrics: {
start: 0,
end: 0,
prompt_tokens: 0,
completion_tokens: 0,
tokens: 0,
property1: null,
property2: null,
},
context: {
caller_functionname: "string",
caller_filename: "string",
caller_lineno: 0,
property1: null,
property2: null,
},
span_attributes: {
name: "string",
type: "llm",
property1: null,
property2: null,
},
id: "string",
dataset_record_id: "string",
_object_delete: true,
_is_merge: false,
_parent_id: "string",
},
],
feedback: [
{
id: "string",
scores: {
property1: 1,
property2: 1,
},
expected: null,
comment: "string",
metadata: {
property1: null,
property2: null,
},
source: "app",
},
],
},
property2: {
events: [
{
input: null,
output: null,
expected: null,
scores: {
property1: 1,
property2: 1,
},
metadata: {
property1: null,
property2: null,
},
tags: ["string"],
metrics: {
start: 0,
end: 0,
prompt_tokens: 0,
completion_tokens: 0,
tokens: 0,
property1: null,
property2: null,
},
context: {
caller_functionname: "string",
caller_filename: "string",
caller_lineno: 0,
property1: null,
property2: null,
},
span_attributes: {
name: "string",
type: "llm",
property1: null,
property2: null,
},
id: "string",
dataset_record_id: "string",
_object_delete: true,
_is_merge: false,
_parent_id: "string",
},
],
feedback: [
{
id: "string",
scores: {
property1: 1,
property2: 1,
},
expected: null,
comment: "string",
metadata: {
property1: null,
property2: null,
},
source: "app",
},
],
},
},
dataset: {
property1: {
events: [
{
input: null,
expected: null,
metadata: {
property1: null,
property2: null,
},
tags: ["string"],
id: "string",
_object_delete: true,
_is_merge: false,
_parent_id: "string",
},
],
feedback: [
{
id: "string",
comment: "string",
metadata: {
property1: null,
property2: null,
},
source: "app",
},
],
},
property2: {
events: [
{
input: null,
expected: null,
metadata: {
property1: null,
property2: null,
},
tags: ["string"],
id: "string",
_object_delete: true,
_is_merge: false,
_parent_id: "string",
},
],
feedback: [
{
id: "string",
comment: "string",
metadata: {
property1: null,
property2: null,
},
source: "app",
},
],
},
},
project_logs: {
property1: {
events: [
{
input: null,
output: null,
expected: null,
scores: {
property1: 1,
property2: 1,
},
metadata: {
property1: null,
property2: null,
},
tags: ["string"],
metrics: {
start: 0,
end: 0,
prompt_tokens: 0,
completion_tokens: 0,
tokens: 0,
property1: null,
property2: null,
},
context: {
caller_functionname: "string",
caller_filename: "string",
caller_lineno: 0,
property1: null,
property2: null,
},
span_attributes: {
name: "string",
type: "llm",
property1: null,
property2: null,
},
id: "string",
_object_delete: true,
_is_merge: false,
_parent_id: "string",
},
],
feedback: [
{
id: "string",
scores: {
property1: 1,
property2: 1,
},
expected: null,
comment: "string",
metadata: {
property1: null,
property2: null,
},
source: "app",
},
],
},
property2: {
events: [
{
input: null,
output: null,
expected: null,
scores: {
property1: 1,
property2: 1,
},
metadata: {
property1: null,
property2: null,
},
tags: ["string"],
metrics: {
start: 0,
end: 0,
prompt_tokens: 0,
completion_tokens: 0,
tokens: 0,
property1: null,
property2: null,
},
context: {
caller_functionname: "string",
caller_filename: "string",
caller_lineno: 0,
property1: null,
property2: null,
},
span_attributes: {
name: "string",
type: "llm",
property1: null,
property2: null,
},
id: "string",
_object_delete: true,
_is_merge: false,
_parent_id: "string",
},
],
feedback: [
{
id: "string",
scores: {
property1: 1,
property2: 1,
},
expected: null,
comment: "string",
metadata: {
property1: null,
property2: null,
},
source: "app",
},
],
},
},
});
const headers = {
"Content-Type": "application/json",
Accept: "application/json",
Authorization: "Bearer {access-token}",
};
fetch("https://api.braintrustdata.com/v1/insert", {
method: "POST",
body: inputBody,
headers: headers,
})
.then(function (res) {
return res.json();
})
.then(function (body) {
console.log(body);
});POST /v1/insert
Insert events and feedback across object types
Body parameter
{
"experiment": {
"property1": {
"events": [
{
"input": null,
"output": null,
"expected": null,
"scores": {
"property1": 1,
"property2": 1
},
"metadata": {
"property1": null,
"property2": null
},
"tags": ["string"],
"metrics": {
"start": 0,
"end": 0,
"prompt_tokens": 0,
"completion_tokens": 0,
"tokens": 0,
"property1": null,
"property2": null
},
"context": {
"caller_functionname": "string",
"caller_filename": "string",
"caller_lineno": 0,
"property1": null,
"property2": null
},
"span_attributes": {
"name": "string",
"type": "llm",
"property1": null,
"property2": null
},
"id": "string",
"dataset_record_id": "string",
"_object_delete": true,
"_is_merge": false,
"_parent_id": "string"
}
],
"feedback": [
{
"id": "string",
"scores": {
"property1": 1,
"property2": 1
},
"expected": null,
"comment": "string",
"metadata": {
"property1": null,
"property2": null
},
"source": "app"
}
]
},
"property2": {
"events": [
{
"input": null,
"output": null,
"expected": null,
"scores": {
"property1": 1,
"property2": 1
},
"metadata": {
"property1": null,
"property2": null
},
"tags": ["string"],
"metrics": {
"start": 0,
"end": 0,
"prompt_tokens": 0,
"completion_tokens": 0,
"tokens": 0,
"property1": null,
"property2": null
},
"context": {
"caller_functionname": "string",
"caller_filename": "string",
"caller_lineno": 0,
"property1": null,
"property2": null
},
"span_attributes": {
"name": "string",
"type": "llm",
"property1": null,
"property2": null
},
"id": "string",
"dataset_record_id": "string",
"_object_delete": true,
"_is_merge": false,
"_parent_id": "string"
}
],
"feedback": [
{
"id": "string",
"scores": {
"property1": 1,
"property2": 1
},
"expected": null,
"comment": "string",
"metadata": {
"property1": null,
"property2": null
},
"source": "app"
}
]
}
},
"dataset": {
"property1": {
"events": [
{
"input": null,
"expected": null,
"metadata": {
"property1": null,
"property2": null
},
"tags": ["string"],
"id": "string",
"_object_delete": true,
"_is_merge": false,
"_parent_id": "string"
}
],
"feedback": [
{
"id": "string",
"comment": "string",
"metadata": {
"property1": null,
"property2": null
},
"source": "app"
}
]
},
"property2": {
"events": [
{
"input": null,
"expected": null,
"metadata": {
"property1": null,
"property2": null
},
"tags": ["string"],
"id": "string",
"_object_delete": true,
"_is_merge": false,
"_parent_id": "string"
}
],
"feedback": [
{
"id": "string",
"comment": "string",
"metadata": {
"property1": null,
"property2": null
},
"source": "app"
}
]
}
},
"project_logs": {
"property1": {
"events": [
{
"input": null,
"output": null,
"expected": null,
"scores": {
"property1": 1,
"property2": 1
},
"metadata": {
"property1": null,
"property2": null
},
"tags": ["string"],
"metrics": {
"start": 0,
"end": 0,
"prompt_tokens": 0,
"completion_tokens": 0,
"tokens": 0,
"property1": null,
"property2": null
},
"context": {
"caller_functionname": "string",
"caller_filename": "string",
"caller_lineno": 0,
"property1": null,
"property2": null
},
"span_attributes": {
"name": "string",
"type": "llm",
"property1": null,
"property2": null
},
"id": "string",
"_object_delete": true,
"_is_merge": false,
"_parent_id": "string"
}
],
"feedback": [
{
"id": "string",
"scores": {
"property1": 1,
"property2": 1
},
"expected": null,
"comment": "string",
"metadata": {
"property1": null,
"property2": null
},
"source": "app"
}
]
},
"property2": {
"events": [
{
"input": null,
"output": null,
"expected": null,
"scores": {
"property1": 1,
"property2": 1
},
"metadata": {
"property1": null,
"property2": null
},
"tags": ["string"],
"metrics": {
"start": 0,
"end": 0,
"prompt_tokens": 0,
"completion_tokens": 0,
"tokens": 0,
"property1": null,
"property2": null
},
"context": {
"caller_functionname": "string",
"caller_filename": "string",
"caller_lineno": 0,
"property1": null,
"property2": null
},
"span_attributes": {
"name": "string",
"type": "llm",
"property1": null,
"property2": null
},
"id": "string",
"_object_delete": true,
"_is_merge": false,
"_parent_id": "string"
}
],
"feedback": [
{
"id": "string",
"scores": {
"property1": 1,
"property2": 1
},
"expected": null,
"comment": "string",
"metadata": {
"property1": null,
"property2": null
},
"source": "app"
}
]
}
}
}Parameters
| Name | In | Type | Required | Description |
|---|---|---|---|---|
| body | body | CrossObjectInsertRequest | false | A mapping from event object type -> object id -> events to insert |
Example responses
200 Response
{
"experiment": {
"property1": {
"row_ids": ["string"]
},
"property2": {
"row_ids": ["string"]
}
},
"dataset": {
"property1": {
"row_ids": ["string"]
},
"property2": {
"row_ids": ["string"]
}
},
"project_logs": {
"property1": {
"row_ids": ["string"]
},
"property2": {
"row_ids": ["string"]
}
}
}400 Response
"string"Responses
| Status | Meaning | Description | Schema |
|---|---|---|---|
| 200 | OK | Returns the inserted row ids for the events on each individual object | CrossObjectInsertResponse |
| 400 | Bad Request | The request was unacceptable, often due to missing a required parameter | Inline |
| 401 | Unauthorized | No valid API key provided | Inline |
| 403 | Forbidden | The API key doesn’t have permissions to perform the request | Inline |
| 429 | Too Many Requests | Too many requests hit the API too quickly. We recommend an exponential backoff of your requests | Inline |
| 500 | Internal Server Error | Something went wrong on Braintrust's end. (These are rare.) | Inline |
Response Schema
Status Code 400
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 401
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 403
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 429
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
Status Code 500
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | none |
🔒
This endpoint requires authentication.
Schemas
ExperimentId
"497f6eca-6276-4993-bfeb-53cbbbba6f08"Experiment id
Properties
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string(uuid) | false | none | Experiment id |
DatasetId
"497f6eca-6276-4993-bfeb-53cbbbba6f08"Dataset id
Properties
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string(uuid) | false | none | Dataset id |
ProjectId
"497f6eca-6276-4993-bfeb-53cbbbba6f08"Project id
Properties
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string(uuid) | false | none | Project id |
PromptId
"497f6eca-6276-4993-bfeb-53cbbbba6f08"Prompt id
Properties
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string(uuid) | false | none | Prompt id |
RoleId
"497f6eca-6276-4993-bfeb-53cbbbba6f08"Role id
Properties
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string(uuid) | false | none | Role id |
GroupId
"497f6eca-6276-4993-bfeb-53cbbbba6f08"Group id
Properties
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string(uuid) | false | none | Group id |
AclId
"497f6eca-6276-4993-bfeb-53cbbbba6f08"Acl id
Properties
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string(uuid) | false | none | Acl id |
UserId
"497f6eca-6276-4993-bfeb-53cbbbba6f08"User id
Properties
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string(uuid) | false | none | User id |
ExperimentName
"string"Name of the experiment to search for
Properties
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | Name of the experiment to search for |
DatasetName
"string"Name of the dataset to search for
Properties
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | Name of the dataset to search for |
ProjectName
"string"Name of the project to search for
Properties
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | Name of the project to search for |
PromptName
"string"Name of the prompt to search for
Properties
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | Name of the prompt to search for |
RoleName
"string"Name of the role to search for
Properties
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | Name of the role to search for |
GroupName
"string"Name of the group to search for
Properties
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | Name of the group to search for |
OrgName
"string"Filter search results to within a particular organization
Properties
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | Filter search results to within a particular organization |
Ids
"497f6eca-6276-4993-bfeb-53cbbbba6f08"Filter search results to a particular set of object IDs. To specify a list of IDs, include the query param multiple times
Properties
anyOf
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string(uuid) | false | none | none |
or
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | [string] | false | none | none |
AppLimitParam
0Limit the number of objects to return
Properties
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | integer | false | none | Limit the number of objects to return |
FetchLimitParam
0limit the number of traces fetched
Fetch queries may be paginated if the total result size is expected to be large (e.g. project_logs which accumulate over a long time). Note that fetch queries only support pagination in descending time order (from latest to earliest _xact_id. Furthermore, later pages may return rows which showed up in earlier pages, except with an earlier _xact_id. This happens because pagination occurs over the whole version history of the event log. You will most likely want to exclude any such duplicate, outdated rows (by id) from your combined result set.
The limit parameter controls the number of full traces to return. So you may end up with more individual rows than the specified limit if you are fetching events containing traces.
Properties
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | integer | false | none | limit the number of traces fetched Fetch queries may be paginated if the total result size is expected to be large (e.g. project_logs which accumulate over a long time). Note that fetch queries only support pagination in descending time order (from latest to earliest _xact_id. Furthermore, later pages may return rows which showed up in earlier pages, except with an earlier _xact_id. This happens because pagination occurs over the whole version history of the event log. You will most likely want to exclude any such duplicate, outdated rows (by id) from your combined result set.The limit parameter controls the number of full traces to return. So you may end up with more individual rows than the specified limit if you are fetching events containing traces. |
StartingAfter
"497f6eca-6276-4993-bfeb-53cbbbba6f08"Pagination cursor id.
For example, if the final item in the last page you fetched had an id of foo, pass starting_after=foo to fetch the next page. Note: you may only pass one of starting_after and ending_before
Properties
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string(uuid) | false | none | Pagination cursor id. For example, if the final item in the last page you fetched had an id of foo, pass starting_after=foo to fetch the next page. Note: you may only pass one of starting_after and ending_before |
EndingBefore
"497f6eca-6276-4993-bfeb-53cbbbba6f08"Pagination cursor id.
For example, if the initial item in the last page you fetched had an id of foo, pass ending_before=foo to fetch the previous page. Note: you may only pass one of starting_after and ending_before
Properties
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string(uuid) | false | none | Pagination cursor id. For example, if the initial item in the last page you fetched had an id of foo, pass ending_before=foo to fetch the previous page. Note: you may only pass one of starting_after and ending_before |
MaxXactId
"string"DEPRECATION NOTICE: The manually-constructed pagination cursor is deprecated in favor of the explicit 'cursor' returned by object fetch requests. Please prefer the 'cursor' argument going forwards.
Together, max_xact_id and max_root_span_id form a pagination cursor
Since a paginated fetch query returns results in order from latest to earliest, the cursor for the next page can be found as the row with the minimum (earliest) value of the tuple (_xact_id, root_span_id). See the documentation of limit for an overview of paginating fetch queries.
Properties
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | DEPRECATION NOTICE: The manually-constructed pagination cursor is deprecated in favor of the explicit 'cursor' returned by object fetch requests. Please prefer the 'cursor' argument going forwards. Together, max_xact_id and max_root_span_id form a pagination cursorSince a paginated fetch query returns results in order from latest to earliest, the cursor for the next page can be found as the row with the minimum (earliest) value of the tuple (_xact_id, root_span_id). See the documentation of limit for an overview of paginating fetch queries. |
MaxRootSpanId
"string"DEPRECATION NOTICE: The manually-constructed pagination cursor is deprecated in favor of the explicit 'cursor' returned by object fetch requests. Please prefer the 'cursor' argument going forwards.
Together, max_xact_id and max_root_span_id form a pagination cursor
Since a paginated fetch query returns results in order from latest to earliest, the cursor for the next page can be found as the row with the minimum (earliest) value of the tuple (_xact_id, root_span_id). See the documentation of limit for an overview of paginating fetch queries.
Properties
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | DEPRECATION NOTICE: The manually-constructed pagination cursor is deprecated in favor of the explicit 'cursor' returned by object fetch requests. Please prefer the 'cursor' argument going forwards. Together, max_xact_id and max_root_span_id form a pagination cursorSince a paginated fetch query returns results in order from latest to earliest, the cursor for the next page can be found as the row with the minimum (earliest) value of the tuple (_xact_id, root_span_id). See the documentation of limit for an overview of paginating fetch queries. |
Version
"string"Retrieve a snapshot of events from a past time
The version id is essentially a filter on the latest event transaction id. You can use the max_xact_id returned by a past fetch as the version to reproduce that exact fetch.
Properties
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | Retrieve a snapshot of events from a past time The version id is essentially a filter on the latest event transaction id. You can use the max_xact_id returned by a past fetch as the version to reproduce that exact fetch. |
SummarizeScores
trueWhether to summarize the scores and metrics. If false (or omitted), only the metadata will be returned.
Properties
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | boolean | false | none | Whether to summarize the scores and metrics. If false (or omitted), only the metadata will be returned. |
ComparisonExperimentId
"497f6eca-6276-4993-bfeb-53cbbbba6f08"The experiment to compare against, if summarizing scores and metrics. If omitted, will fall back to the base_exp_id stored in the experiment metadata, and then to the most recent experiment run in the same project. Must pass summarize_scores=true for this id to be used
Properties
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string(uuid) | false | none | The experiment to compare against, if summarizing scores and metrics. If omitted, will fall back to the base_exp_id stored in the experiment metadata, and then to the most recent experiment run in the same project. Must pass summarize_scores=true for this id to be used |
SummarizeData
trueWhether to summarize the data. If false (or omitted), only the metadata will be returned.
Properties
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | boolean | false | none | Whether to summarize the data. If false (or omitted), only the metadata will be returned. |
Slug
"string"Retrieve prompt with a specific slug
Properties
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | Retrieve prompt with a specific slug |
UserGivenName
"string"Given name of the user to search for
Properties
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | Given name of the user to search for |
UserFamilyName
"string"Family name of the user to search for
Properties
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | Family name of the user to search for |
UserEmail
"string"Email of the user to search for
Properties
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | Email of the user to search for |
AclObjectType
"organization"The object type that the ACL applies to
Properties
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string | false | none | The object type that the ACL applies to |
Enumerated Values
| Property | Value |
|---|---|
| anonymous | organization |
| anonymous | project |
| anonymous | experiment |
| anonymous | dataset |
| anonymous | prompt |
| anonymous | prompt_session |
| anonymous | project_score |
| anonymous | project_tag |
| anonymous | group |
| anonymous | role |
| anonymous | null |
AclObjectId
"497f6eca-6276-4993-bfeb-53cbbbba6f08"The id of the object the ACL applies to
Properties
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | string(uuid) | false | none | The id of the object the ACL applies to |
Project
{
"id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
"org_id": "a40f5d1f-d889-42e9-94ea-b9b33585fc6b",
"name": "string",
"created": "2019-08-24T14:15:22Z",
"deleted_at": "2019-08-24T14:15:22Z",
"user_id": "a169451c-8525-4352-b8ca-070dd449a1a5"
}Properties
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| id | string(uuid) | true | none | Unique identifier for the project |
| org_id | string(uuid) | true | none | Unique id for the organization that the project belongs under |
| name | string | true | none | Name of the project |
| created | string(date-time)¦null | false | none | Date of project creation |
| deleted_at | string(date-time)¦null | false | none | Date of project deletion, or null if the project is still active |
| user_id | string(uuid)¦null | false | none | Identifies the user who created the project |
CreateProject
{
"name": "string",
"org_name": "string"
}Properties
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| name | string | true | none | Name of the project |
| org_name | string¦null | false | none | For nearly all users, this parameter should be unnecessary. But in the rare case that your API key belongs to multiple organizations, you may specify the name of the organization the project belongs in. |
PatchProject
{
"name": "string"
}Properties
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| name | string¦null | false | none | Name of the project |
InsertEventsResponse
{
"row_ids": ["string"]
}Properties
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| row_ids | [string] | true | none | The ids of all rows that were inserted, aligning one-to-one with the rows provided as input |
InsertProjectLogsEventReplace
{
"input": null,
"output": null,
"expected": null,
"scores": {
"property1": 1,
"property2": 1
},
"metadata": {
"property1": null,
"property2": null
},
"tags": ["string"],
"metrics": {
"start": 0,
"end": 0,
"prompt_tokens": 0,
"completion_tokens": 0,
"tokens": 0,
"property1": null,
"property2": null
},
"context": {
"caller_functionname": "string",
"caller_filename": "string",
"caller_lineno": 0,
"property1": null,
"property2": null
},
"span_attributes": {
"name": "string",
"type": "llm",
"property1": null,
"property2": null
},
"id": "string",
"_object_delete": true,
"_is_merge": false,
"_parent_id": "string"
}Properties
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| input | any | false | none | The arguments that uniquely define a user input (an arbitrary, JSON serializable object). |
| output | any | false | none | The output of your application, including post-processing (an arbitrary, JSON serializable object), that allows you to determine whether the result is correct or not. For example, in an app that generates SQL queries, the output should be the result of the SQL query generated by the model, not the query itself, because there may be multiple valid queries that answer a single question. |
| expected | any | false | none | The ground truth value (an arbitrary, JSON serializable object) that you'd compare to output to determine if your output value is correct or not. Braintrust currently does not compare output to expected for you, since there are so many different ways to do that correctly. Instead, these values are just used to help you navigate while digging into analyses. However, we may later use these values to re-score outputs or fine-tune your models. |
| scores | object¦null | false | none | A dictionary of numeric values (between 0 and 1) to log. The scores should give you a variety of signals that help you determine how accurate the outputs are compared to what you expect and diagnose failures. For example, a summarization app might have one score that tells you how accurate the summary is, and another that measures the word similarity between the generated and grouth truth summary. The word similarity score could help you determine whether the summarization was covering similar concepts or not. You can use these scores to help you sort, filter, and compare logs. |
| » additionalProperties | number¦null | false | none | none |
| metadata | object¦null | false | none | A dictionary with additional data about the test example, model outputs, or just about anything else that's relevant, that you can use to help find and analyze examples later. For example, you could log the prompt, example's id, or anything else that would be useful to slice/dice later. The values in metadata can be any JSON-serializable type, but its keys must be strings |
| » additionalProperties | any | false | none | none |
| tags | [string]¦null | false | none | A list of tags to log |
| metrics | object¦null | false | none | Metrics are numerical measurements tracking the execution of the code that produced the project logs event. Use "start" and "end" to track the time span over which the project logs event was produced |
| » additionalProperties | any | false | none | none |
| » start | number¦null | false | none | A unix timestamp recording when the section of code which produced the project logs event started |
| » end | number¦null | false | none | A unix timestamp recording when the section of code which produced the project logs event finished |
| » prompt_tokens | integer¦null | false | none | The number of tokens in the prompt used to generate the project logs event (only set if this is an LLM span) |
| » completion_tokens | integer¦null | false | none | The number of tokens in the completion generated by the model (only set if this is an LLM span) |
| » tokens | integer¦null | false | none | The total number of tokens in the input and output of the project logs event. |
| context | object¦null | false | none | Context is additional information about the code that produced the project logs event. It is essentially the textual counterpart to metrics. Use the caller_* attributes to track the location in code which produced the project logs event |
| » additionalProperties | any | false | none | none |
| » caller_functionname | string¦null | false | none | The function in code which created the project logs event |
| » caller_filename | string¦null | false | none | Name of the file in code where the project logs event was created |
| » caller_lineno | integer¦null | false | none | Line of code where the project logs event was created |
| span_attributes | object¦null | false | none | Human-identifying attributes of the span, such as name, type, etc. |
| » additionalProperties | any | false | none | none |
| » name | string¦null | false | none | Name of the span, for display purposes only |
| » type | string¦null | false | none | Type of the span, for display purposes only |
| id | string¦null | false | none | A unique identifier for the project logs event. If you don't provide one, BrainTrust will generate one for you |
| _object_delete | boolean¦null | false | none | Pass _object_delete=true to mark the project logs event deleted. Deleted events will not show up in subsequent fetches for this project logs |
| _is_merge | boolean¦null | false | none | The _is_merge field controls how the row is merged with any existing row with the same id in the DB. By default (or when set to false), the existing row is completely replaced by the new row. When set to true, the new row is deep-merged into the existing rowFor example, say there is an existing row in the DB {"id": "foo", "input": {"a": 5, "b": 10}}. If we merge a new row as {"_is_merge": true, "id": "foo", "input": {"b": 11, "c": 20}}, the new row will be {"id": "foo", "input": {"a": 5, "b": 11, "c": 20}}. If we replace the new row as {"id": "foo", "input": {"b": 11, "c": 20}}, the new row will be {"id": "foo", "input": {"b": 11, "c": 20}} |
| _parent_id | string¦null | false | none | Use the _parent_id field to create this row as a subspan of an existing row. It cannot be specified alongside _is_merge=true. Tracking hierarchical relationships are important for tracing (see the guide for full details).For example, say we have logged a row {"id": "abc", "input": "foo", "output": "bar", "expected": "boo", "scores": {"correctness": 0.33}}. We can create a sub-span of the parent row by logging {"_parent_id": "abc", "id": "llm_call", "input": {"prompt": "What comes after foo?"}, "output": "bar", "metrics": {"tokens": 1}}. In the webapp, only the root span row "abc" will show up in the summary view. You can view the full trace hierarchy (in this case, the "llm_call" row) by clicking on the "abc" row. |
Enumerated Values
| Property | Value |
|---|---|
| type | llm |
| type | score |
| type | function |
| type | eval |
| type | task |
| type | tool |
| type | null |
| _is_merge | false |
| _is_merge | null |
InsertProjectLogsEventMerge
{
"input": null,
"output": null,
"expected": null,
"scores": {
"property1": 1,
"property2": 1
},
"metadata": {
"property1": null,
"property2": null
},
"tags": ["string"],
"metrics": {
"start": 0,
"end": 0,
"prompt_tokens": 0,
"completion_tokens": 0,
"tokens": 0,
"property1": null,
"property2": null
},
"context": {
"caller_functionname": "string",
"caller_filename": "string",
"caller_lineno": 0,
"property1": null,
"property2": null
},
"span_attributes": {
"name": "string",
"type": "llm",
"property1": null,
"property2": null
},
"id": "string",
"_object_delete": true,
"_is_merge": true,
"_merge_paths": [["string"]]
}Properties
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| input | any | false | none | The arguments that uniquely define a user input (an arbitrary, JSON serializable object). |
| output | any | false | none | The output of your application, including post-processing (an arbitrary, JSON serializable object), that allows you to determine whether the result is correct or not. For example, in an app that generates SQL queries, the output should be the result of the SQL query generated by the model, not the query itself, because there may be multiple valid queries that answer a single question. |
| expected | any | false | none | The ground truth value (an arbitrary, JSON serializable object) that you'd compare to output to determine if your output value is correct or not. Braintrust currently does not compare output to expected for you, since there are so many different ways to do that correctly. Instead, these values are just used to help you navigate while digging into analyses. However, we may later use these values to re-score outputs or fine-tune your models. |
| scores | object¦null | false | none | A dictionary of numeric values (between 0 and 1) to log. The scores should give you a variety of signals that help you determine how accurate the outputs are compared to what you expect and diagnose failures. For example, a summarization app might have one score that tells you how accurate the summary is, and another that measures the word similarity between the generated and grouth truth summary. The word similarity score could help you determine whether the summarization was covering similar concepts or not. You can use these scores to help you sort, filter, and compare logs. |
| » additionalProperties | number¦null | false | none | none |
| metadata | object¦null | false | none | A dictionary with additional data about the test example, model outputs, or just about anything else that's relevant, that you can use to help find and analyze examples later. For example, you could log the prompt, example's id, or anything else that would be useful to slice/dice later. The values in metadata can be any JSON-serializable type, but its keys must be strings |
| » additionalProperties | any | false | none | none |
| tags | [string]¦null | false | none | A list of tags to log |
| metrics | object¦null | false | none | Metrics are numerical measurements tracking the execution of the code that produced the project logs event. Use "start" and "end" to track the time span over which the project logs event was produced |
| » additionalProperties | any | false | none | none |
| » start | number¦null | false | none | A unix timestamp recording when the section of code which produced the project logs event started |
| » end | number¦null | false | none | A unix timestamp recording when the section of code which produced the project logs event finished |
| » prompt_tokens | integer¦null | false | none | The number of tokens in the prompt used to generate the project logs event (only set if this is an LLM span) |
| » completion_tokens | integer¦null | false | none | The number of tokens in the completion generated by the model (only set if this is an LLM span) |
| » tokens | integer¦null | false | none | The total number of tokens in the input and output of the project logs event. |
| context | object¦null | false | none | Context is additional information about the code that produced the project logs event. It is essentially the textual counterpart to metrics. Use the caller_* attributes to track the location in code which produced the project logs event |
| » additionalProperties | any | false | none | none |
| » caller_functionname | string¦null | false | none | The function in code which created the project logs event |
| » caller_filename | string¦null | false | none | Name of the file in code where the project logs event was created |
| » caller_lineno | integer¦null | false | none | Line of code where the project logs event was created |
| span_attributes | object¦null | false | none | Human-identifying attributes of the span, such as name, type, etc. |
| » additionalProperties | any | false | none | none |
| » name | string¦null | false | none | Name of the span, for display purposes only |
| » type | string¦null | false | none | Type of the span, for display purposes only |
| id | string¦null | false | none | A unique identifier for the project logs event. If you don't provide one, BrainTrust will generate one for you |
| _object_delete | boolean¦null | false | none | Pass _object_delete=true to mark the project logs event deleted. Deleted events will not show up in subsequent fetches for this project logs |
| _is_merge | boolean | true | none | The _is_merge field controls how the row is merged with any existing row with the same id in the DB. By default (or when set to false), the existing row is completely replaced by the new row. When set to true, the new row is deep-merged into the existing rowFor example, say there is an existing row in the DB {"id": "foo", "input": {"a": 5, "b": 10}}. If we merge a new row as {"_is_merge": true, "id": "foo", "input": {"b": 11, "c": 20}}, the new row will be {"id": "foo", "input": {"a": 5, "b": 11, "c": 20}}. If we replace the new row as {"id": "foo", "input": {"b": 11, "c": 20}}, the new row will be {"id": "foo", "input": {"b": 11, "c": 20}} |
| _merge_paths | [array]¦null | false | none | The _merge_paths field allows controlling the depth of the merge. It can only be specified alongside _is_merge=true. _merge_paths is a list of paths, where each path is a list of field names. The deep merge will not descend below any of the specified merge paths.For example, say there is an existing row in the DB {"id": "foo", "input": {"a": {"b": 10}, "c": {"d": 20}}, "output": {"a": 20}}. If we merge a new row as {"_is_merge": true, "_merge_paths": [["input", "a"], ["output"]], "input": {"a": {"q": 30}, "c": {"e": 30}, "bar": "baz"}, "output": {"d": 40}}, the new row will be {"id": "foo": "input": {"a": {"q": 30}, "c": {"d": 20, "e": 30}, "bar": "baz"}, "output": {"d": 40}}. In this case, due to the merge paths, we have replaced input.a and output, but have still deep-merged input and input.c. |
Enumerated Values
| Property | Value |
|---|---|
| type | llm |
| type | score |
| type | function |
| type | eval |
| type | task |
| type | tool |
| type | null |
| _is_merge | true |
InsertProjectLogsEvent
{
"input": null,
"output": null,
"expected": null,
"scores": {
"property1": 1,
"property2": 1
},
"metadata": {
"property1": null,
"property2": null
},
"tags": ["string"],
"metrics": {
"start": 0,
"end": 0,
"prompt_tokens": 0,
"completion_tokens": 0,
"tokens": 0,
"property1": null,
"property2": null
},
"context": {
"caller_functionname": "string",
"caller_filename": "string",
"caller_lineno": 0,
"property1": null,
"property2": null
},
"span_attributes": {
"name": "string",
"type": "llm",
"property1": null,
"property2": null
},
"id": "string",
"_object_delete": true,
"_is_merge": false,
"_parent_id": "string"
}A project logs event
Properties
anyOf
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | InsertProjectLogsEventReplace | false | none | none |
or
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | InsertProjectLogsEventMerge | false | none | none |
InsertProjectLogsEventRequest
{
"events": [
{
"input": null,
"output": null,
"expected": null,
"scores": {
"property1": 1,
"property2": 1
},
"metadata": {
"property1": null,
"property2": null
},
"tags": ["string"],
"metrics": {
"start": 0,
"end": 0,
"prompt_tokens": 0,
"completion_tokens": 0,
"tokens": 0,
"property1": null,
"property2": null
},
"context": {
"caller_functionname": "string",
"caller_filename": "string",
"caller_lineno": 0,
"property1": null,
"property2": null
},
"span_attributes": {
"name": "string",
"type": "llm",
"property1": null,
"property2": null
},
"id": "string",
"_object_delete": true,
"_is_merge": false,
"_parent_id": "string"
}
]
}Properties
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| events | [InsertProjectLogsEvent] | true | none | A list of project logs events to insert |
ProjectLogsEvent
{
"id": "string",
"_xact_id": "string",
"created": "2019-08-24T14:15:22Z",
"org_id": "a40f5d1f-d889-42e9-94ea-b9b33585fc6b",
"project_id": "405d8375-3514-403b-8c43-83ae74cfe0e9",
"log_id": "g",
"input": null,
"output": null,
"expected": null,
"scores": {
"property1": 1,
"property2": 1
},
"metadata": {
"property1": null,
"property2": null
},
"tags": ["string"],
"metrics": {
"start": 0,
"end": 0,
"prompt_tokens": 0,
"completion_tokens": 0,
"tokens": 0,
"property1": null,
"property2": null
},
"context": {
"caller_functionname": "string",
"caller_filename": "string",
"caller_lineno": 0,
"property1": null,
"property2": null
},
"span_id": "string",
"span_parents": ["string"],
"root_span_id": "string",
"span_attributes": {
"name": "string",
"type": "llm",
"property1": null,
"property2": null
}
}Properties
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| id | string | true | none | A unique identifier for the project logs event. If you don't provide one, BrainTrust will generate one for you |
| _xact_id | string | true | none | The transaction id of an event is unique to the network operation that processed the event insertion. Transaction ids are monotonically increasing over time and can be used to retrieve a versioned snapshot of the project logs (see the version parameter) |
| created | string(date-time) | true | none | The timestamp the project logs event was created |
| org_id | string(uuid) | true | none | Unique id for the organization that the project belongs under |
| project_id | string(uuid) | true | none | Unique identifier for the project |
| log_id | string | true | none | A literal 'g' which identifies the log as a project log |
| input | any | false | none | The arguments that uniquely define a user input (an arbitrary, JSON serializable object). |
| output | any | false | none | The output of your application, including post-processing (an arbitrary, JSON serializable object), that allows you to determine whether the result is correct or not. For example, in an app that generates SQL queries, the output should be the result of the SQL query generated by the model, not the query itself, because there may be multiple valid queries that answer a single question. |
| expected | any | false | none | The ground truth value (an arbitrary, JSON serializable object) that you'd compare to output to determine if your output value is correct or not. Braintrust currently does not compare output to expected for you, since there are so many different ways to do that correctly. Instead, these values are just used to help you navigate while digging into analyses. However, we may later use these values to re-score outputs or fine-tune your models. |
| scores | object¦null | false | none | A dictionary of numeric values (between 0 and 1) to log. The scores should give you a variety of signals that help you determine how accurate the outputs are compared to what you expect and diagnose failures. For example, a summarization app might have one score that tells you how accurate the summary is, and another that measures the word similarity between the generated and grouth truth summary. The word similarity score could help you determine whether the summarization was covering similar concepts or not. You can use these scores to help you sort, filter, and compare logs. |
| » additionalProperties | number¦null | false | none | none |
| metadata | object¦null | false | none | A dictionary with additional data about the test example, model outputs, or just about anything else that's relevant, that you can use to help find and analyze examples later. For example, you could log the prompt, example's id, or anything else that would be useful to slice/dice later. The values in metadata can be any JSON-serializable type, but its keys must be strings |
| » additionalProperties | any | false | none | none |
| tags | [string]¦null | false | none | A list of tags to log |
| metrics | object¦null | false | none | Metrics are numerical measurements tracking the execution of the code that produced the project logs event. Use "start" and "end" to track the time span over which the project logs event was produced |
| » additionalProperties | any | false | none | none |
| » start | number¦null | false | none | A unix timestamp recording when the section of code which produced the project logs event started |
| » end | number¦null | false | none | A unix timestamp recording when the section of code which produced the project logs event finished |
| » prompt_tokens | integer¦null | false | none | The number of tokens in the prompt used to generate the project logs event (only set if this is an LLM span) |
| » completion_tokens | integer¦null | false | none | The number of tokens in the completion generated by the model (only set if this is an LLM span) |
| » tokens | integer¦null | false | none | The total number of tokens in the input and output of the project logs event. |
| context | object¦null | false | none | Context is additional information about the code that produced the project logs event. It is essentially the textual counterpart to metrics. Use the caller_* attributes to track the location in code which produced the project logs event |
| » additionalProperties | any | false | none | none |
| » caller_functionname | string¦null | false | none | The function in code which created the project logs event |
| » caller_filename | string¦null | false | none | Name of the file in code where the project logs event was created |
| » caller_lineno | integer¦null | false | none | Line of code where the project logs event was created |
| span_id | string | true | none | A unique identifier used to link different project logs events together as part of a full trace. See the tracing guide for full details on tracing |
| span_parents | [string]¦null | false | none | An array of the parent span_ids of this project logs event. This should be empty for the root span of a trace, and should most often contain just one parent element for subspans |
| root_span_id | string | true | none | The span_id of the root of the trace this project logs event belongs to |
| span_attributes | object¦null | false | none | Human-identifying attributes of the span, such as name, type, etc. |
| » additionalProperties | any | false | none | none |
| » name | string¦null | false | none | Name of the span, for display purposes only |
| » type | string¦null | false | none | Type of the span, for display purposes only |
Enumerated Values
| Property | Value |
|---|---|
| log_id | g |
| type | llm |
| type | score |
| type | function |
| type | eval |
| type | task |
| type | tool |
| type | null |
FetchProjectLogsEventsResponse
{
"events": [
{
"id": "string",
"_xact_id": "string",
"created": "2019-08-24T14:15:22Z",
"org_id": "a40f5d1f-d889-42e9-94ea-b9b33585fc6b",
"project_id": "405d8375-3514-403b-8c43-83ae74cfe0e9",
"log_id": "g",
"input": null,
"output": null,
"expected": null,
"scores": {
"property1": 1,
"property2": 1
},
"metadata": {
"property1": null,
"property2": null
},
"tags": ["string"],
"metrics": {
"start": 0,
"end": 0,
"prompt_tokens": 0,
"completion_tokens": 0,
"tokens": 0,
"property1": null,
"property2": null
},
"context": {
"caller_functionname": "string",
"caller_filename": "string",
"caller_lineno": 0,
"property1": null,
"property2": null
},
"span_id": "string",
"span_parents": ["string"],
"root_span_id": "string",
"span_attributes": {
"name": "string",
"type": "llm",
"property1": null,
"property2": null
}
}
],
"cursor": "string"
}Properties
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| events | [ProjectLogsEvent] | true | none | A list of fetched events |
| cursor | string¦null | false | none | Pagination cursor Pass this string directly as the cursor param to your next fetch request to get the next page of results. Not provided if the returned result set is empty. |
PathLookupFilter
{
"type": "path_lookup",
"path": ["string"],
"value": null
}A path-lookup filter describes an equality comparison against a specific sub-field in the event row. For instance, if you wish to filter on the value of c in {"input": {"a": {"b": {"c": "hello"}}}}, pass path=["input", "a", "b", "c"] and value="hello"
Properties
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| type | string | true | none | Denotes the type of filter as a path-lookup filter |
| path | [string] | true | none | List of fields describing the path to the value to be checked against. For instance, if you wish to filter on the value of c in {"input": {"a": {"b": {"c": "hello"}}}}, pass path=["input", "a", "b", "c"] |
| value | any | false | none | The value to compare equality-wise against the event value at the specified path. The value must be a "primitive", that is, any JSON-serializable object except for objects and arrays. For instance, if you wish to filter on the value of "input.a.b.c" in the object {"input": {"a": {"b": {"c": "hello"}}}}, pass value="hello" |
Enumerated Values
| Property | Value |
|---|---|
| type | path_lookup |
FetchEventsFilters
[
{
"type": "path_lookup",
"path": ["string"],
"value": null
}
]A list of filters on the events to fetch. Currently, only path-lookup type filters are supported, but we may add more in the future
Properties
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | [PathLookupFilter]¦null | false | none | A list of filters on the events to fetch. Currently, only path-lookup type filters are supported, but we may add more in the future |
FetchEventsRequest
{
"limit": 0,
"cursor": "string",
"max_xact_id": "string",
"max_root_span_id": "string",
"filters": [
{
"type": "path_lookup",
"path": ["string"],
"value": null
}
],
"version": "string"
}Properties
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| limit | integer¦null | false | none | limit the number of traces fetched Fetch queries may be paginated if the total result size is expected to be large (e.g. project_logs which accumulate over a long time). Note that fetch queries only support pagination in descending time order (from latest to earliest _xact_id. Furthermore, later pages may return rows which showed up in earlier pages, except with an earlier _xact_id. This happens because pagination occurs over the whole version history of the event log. You will most likely want to exclude any such duplicate, outdated rows (by id) from your combined result set.The limit parameter controls the number of full traces to return. So you may end up with more individual rows than the specified limit if you are fetching events containing traces. |
| cursor | string¦null | false | none | An opaque string to be used as a cursor for the next page of results, in order from latest to earliest. The string can be obtained directly from the cursor property of the previous fetch query |
| max_xact_id | string¦null | false | none | DEPRECATION NOTICE: The manually-constructed pagination cursor is deprecated in favor of the explicit 'cursor' returned by object fetch requests. Please prefer the 'cursor' argument going forwards. Together, max_xact_id and max_root_span_id form a pagination cursorSince a paginated fetch query returns results in order from latest to earliest, the cursor for the next page can be found as the row with the minimum (earliest) value of the tuple (_xact_id, root_span_id). See the documentation of limit for an overview of paginating fetch queries. |
| max_root_span_id | string¦null | false | none | DEPRECATION NOTICE: The manually-constructed pagination cursor is deprecated in favor of the explicit 'cursor' returned by object fetch requests. Please prefer the 'cursor' argument going forwards. Together, max_xact_id and max_root_span_id form a pagination cursorSince a paginated fetch query returns results in order from latest to earliest, the cursor for the next page can be found as the row with the minimum (earliest) value of the tuple (_xact_id, root_span_id). See the documentation of limit for an overview of paginating fetch queries. |
| filters | FetchEventsFilters | false | none | A list of filters on the events to fetch. Currently, only path-lookup type filters are supported, but we may add more in the future |
| version | string¦null | false | none | Retrieve a snapshot of events from a past time The version id is essentially a filter on the latest event transaction id. You can use the max_xact_id returned by a past fetch as the version to reproduce that exact fetch. |
FeedbackProjectLogsItem
{
"id": "string",
"scores": {
"property1": 1,
"property2": 1
},
"expected": null,
"comment": "string",
"metadata": {
"property1": null,
"property2": null
},
"source": "app"
}Properties
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| id | string | true | none | The id of the project logs event to log feedback for. This is the row id returned by POST /v1/project_logs/{project_id}/insert |
| scores | object¦null | false | none | A dictionary of numeric values (between 0 and 1) to log. These scores will be merged into the existing scores for the project logs event |
| » additionalProperties | number¦null | false | none | none |
| expected | any | false | none | The ground truth value (an arbitrary, JSON serializable object) that you'd compare to output to determine if your output value is correct or not |
| comment | string¦null | false | none | An optional comment string to log about the project logs event |
| metadata | object¦null | false | none | A dictionary with additional data about the feedback. If you have a user_id, you can log it here and access it in the Braintrust UI. |
| » additionalProperties | any | false | none | none |
| source | string¦null | false | none | The source of the feedback. Must be one of "external" (default), "app", or "api" |
Enumerated Values
| Property | Value |
|---|---|
| source | app |
| source | api |
| source | external |
| source | null |
FeedbackProjectLogsEventRequest
{
"feedback": [
{
"id": "string",
"scores": {
"property1": 1,
"property2": 1
},
"expected": null,
"comment": "string",
"metadata": {
"property1": null,
"property2": null
},
"source": "app"
}
]
}Properties
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| feedback | [FeedbackProjectLogsItem] | true | none | A list of project logs feedback items |
RepoInfo
{
"commit": "string",
"branch": "string",
"tag": "string",
"dirty": true,
"author_name": "string",
"author_email": "string",
"commit_message": "string",
"commit_time": "string",
"git_diff": "string"
}Metadata about the state of the repo when the experiment was created
Properties
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| commit | string¦null | false | none | SHA of most recent commit |
| branch | string¦null | false | none | Name of the branch the most recent commit belongs to |
| tag | string¦null | false | none | Name of the tag on the most recent commit |
| dirty | boolean¦null | false | none | Whether or not the repo had uncommitted changes when snapshotted |
| author_name | string¦null | false | none | Name of the author of the most recent commit |
| author_email | string¦null | false | none | Email of the author of the most recent commit |
| commit_message | string¦null | false | none | Most recent commit message |
| commit_time | string¦null | false | none | Time of the most recent commit |
| git_diff | string¦null | false | none | If the repo was dirty when run, this includes the diff between the current state of the repo and the most recent commit. |
Experiment
{
"id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
"project_id": "405d8375-3514-403b-8c43-83ae74cfe0e9",
"name": "string",
"description": "string",
"created": "2019-08-24T14:15:22Z",
"repo_info": {
"commit": "string",
"branch": "string",
"tag": "string",
"dirty": true,
"author_name": "string",
"author_email": "string",
"commit_message": "string",
"commit_time": "string",
"git_diff": "string"
},
"commit": "string",
"base_exp_id": "4838cee2-a665-4545-aa9f-483678c01a6b",
"deleted_at": "2019-08-24T14:15:22Z",
"dataset_id": "8c4c51f1-f6f3-43bc-b65d-7415e8ef22c0",
"dataset_version": "string",
"public": true,
"user_id": "a169451c-8525-4352-b8ca-070dd449a1a5",
"metadata": {
"property1": null,
"property2": null
}
}Properties
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| id | string(uuid) | true | none | Unique identifier for the experiment |
| project_id | string(uuid) | true | none | Unique identifier for the project that the experiment belongs under |
| name | string | true | none | Name of the experiment. Within a project, experiment names are unique |
| description | string¦null | false | none | Textual description of the experiment |
| created | string(date-time)¦null | false | none | Date of experiment creation |
| repo_info | RepoInfo | false | none | Metadata about the state of the repo when the experiment was created |
| commit | string¦null | false | none | Commit, taken directly from repo_info.commit |
| base_exp_id | string(uuid)¦null | false | none | Id of default base experiment to compare against when viewing this experiment |
| deleted_at | string(date-time)¦null | false | none | Date of experiment deletion, or null if the experiment is still active |
| dataset_id | string(uuid)¦null | false | none | Identifier of the linked dataset, or null if the experiment is not linked to a dataset |
| dataset_version | string¦null | false | none | Version number of the linked dataset the experiment was run against. This can be used to reproduce the experiment after the dataset has been modified. |
| public | boolean | true | none | Whether or not the experiment is public. Public experiments can be viewed by anybody inside or outside the organization |
| user_id | string(uuid)¦null | false | none | Identifies the user who created the experiment |
| metadata | object¦null | false | none | User-controlled metadata about the experiment |
| » additionalProperties | any | false | none | none |
CreateExperiment
{
"project_id": "405d8375-3514-403b-8c43-83ae74cfe0e9",
"name": "string",
"description": "string",
"repo_info": {
"commit": "string",
"branch": "string",
"tag": "string",
"dirty": true,
"author_name": "string",
"author_email": "string",
"commit_message": "string",
"commit_time": "string",
"git_diff": "string"
},
"base_exp_id": "4838cee2-a665-4545-aa9f-483678c01a6b",
"dataset_id": "8c4c51f1-f6f3-43bc-b65d-7415e8ef22c0",
"dataset_version": "string",
"public": true,
"metadata": {
"property1": null,
"property2": null
},
"ensure_new": true
}Properties
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| project_id | string(uuid) | true | none | Unique identifier for the project that the experiment belongs under |
| name | string¦null | false | none | Name of the experiment. Within a project, experiment names are unique |
| description | string¦null | false | none | Textual description of the experiment |
| repo_info | RepoInfo | false | none | Metadata about the state of the repo when the experiment was created |
| base_exp_id | string(uuid)¦null | false | none | Id of default base experiment to compare against when viewing this experiment |
| dataset_id | string(uuid)¦null | false | none | Identifier of the linked dataset, or null if the experiment is not linked to a dataset |
| dataset_version | string¦null | false | none | Version number of the linked dataset the experiment was run against. This can be used to reproduce the experiment after the dataset has been modified. |
| public | boolean¦null | false | none | Whether or not the experiment is public. Public experiments can be viewed by anybody inside or outside the organization |
| metadata | object¦null | false | none | User-controlled metadata about the experiment |
| » additionalProperties | any | false | none | none |
| ensure_new | boolean¦null | false | none | Normally, creating an experiment with the same name as an existing experiment will return the existing one un-modified. But if ensure_new is true, registration will generate a new experiment with a unique name in case of a conflict. |
PatchExperiment
{
"name": "string",
"description": "string",
"repo_info": {
"commit": "string",
"branch": "string",
"tag": "string",
"dirty": true,
"author_name": "string",
"author_email": "string",
"commit_message": "string",
"commit_time": "string",
"git_diff": "string"
},
"base_exp_id": "4838cee2-a665-4545-aa9f-483678c01a6b",
"dataset_id": "8c4c51f1-f6f3-43bc-b65d-7415e8ef22c0",
"dataset_version": "string",
"public": true,
"metadata": {
"property1": null,
"property2": null
}
}Properties
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| name | string¦null | false | none | Name of the experiment. Within a project, experiment names are unique |
| description | string¦null | false | none | Textual description of the experiment |
| repo_info | RepoInfo | false | none | Metadata about the state of the repo when the experiment was created |
| base_exp_id | string(uuid)¦null | false | none | Id of default base experiment to compare against when viewing this experiment |
| dataset_id | string(uuid)¦null | false | none | Identifier of the linked dataset, or null if the experiment is not linked to a dataset |
| dataset_version | string¦null | false | none | Version number of the linked dataset the experiment was run against. This can be used to reproduce the experiment after the dataset has been modified. |
| public | boolean¦null | false | none | Whether or not the experiment is public. Public experiments can be viewed by anybody inside or outside the organization |
| metadata | object¦null | false | none | User-controlled metadata about the experiment |
| » additionalProperties | any | false | none | none |
InsertExperimentEventReplace
{
"input": null,
"output": null,
"expected": null,
"scores": {
"property1": 1,
"property2": 1
},
"metadata": {
"property1": null,
"property2": null
},
"tags": ["string"],
"metrics": {
"start": 0,
"end": 0,
"prompt_tokens": 0,
"completion_tokens": 0,
"tokens": 0,
"property1": null,
"property2": null
},
"context": {
"caller_functionname": "string",
"caller_filename": "string",
"caller_lineno": 0,
"property1": null,
"property2": null
},
"span_attributes": {
"name": "string",
"type": "llm",
"property1": null,
"property2": null
},
"id": "string",
"dataset_record_id": "string",
"_object_delete": true,
"_is_merge": false,
"_parent_id": "string"
}Properties
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| input | any | false | none | The arguments that uniquely define a test case (an arbitrary, JSON serializable object). Later on, Braintrust will use the input to know whether two test cases are the same between experiments, so they should not contain experiment-specific state. A simple rule of thumb is that if you run the same experiment twice, the input should be identical |
| output | any | false | none | The output of your application, including post-processing (an arbitrary, JSON serializable object), that allows you to determine whether the result is correct or not. For example, in an app that generates SQL queries, the output should be the result of the SQL query generated by the model, not the query itself, because there may be multiple valid queries that answer a single question |
| expected | any | false | none | The ground truth value (an arbitrary, JSON serializable object) that you'd compare to output to determine if your output value is correct or not. Braintrust currently does not compare output to expected for you, since there are so many different ways to do that correctly. Instead, these values are just used to help you navigate your experiments while digging into analyses. However, we may later use these values to re-score outputs or fine-tune your models |
| scores | object¦null | false | none | A dictionary of numeric values (between 0 and 1) to log. The scores should give you a variety of signals that help you determine how accurate the outputs are compared to what you expect and diagnose failures. For example, a summarization app might have one score that tells you how accurate the summary is, and another that measures the word similarity between the generated and grouth truth summary. The word similarity score could help you determine whether the summarization was covering similar concepts or not. You can use these scores to help you sort, filter, and compare experiments |
| » additionalProperties | number¦null | false | none | none |
| metadata | object¦null | false | none | A dictionary with additional data about the test example, model outputs, or just about anything else that's relevant, that you can use to help find and analyze examples later. For example, you could log the prompt, example's id, or anything else that would be useful to slice/dice later. The values in metadata can be any JSON-serializable type, but its keys must be strings |
| » additionalProperties | any | false | none | none |
| tags | [string]¦null | false | none | A list of tags to log |
| metrics | object¦null | false | none | Metrics are numerical measurements tracking the execution of the code that produced the experiment event. Use "start" and "end" to track the time span over which the experiment event was produced |
| » additionalProperties | any | false | none | none |
| » start | number¦null | false | none | A unix timestamp recording when the section of code which produced the experiment event started |
| » end | number¦null | false | none | A unix timestamp recording when the section of code which produced the experiment event finished |
| » prompt_tokens | integer¦null | false | none | The number of tokens in the prompt used to generate the experiment event (only set if this is an LLM span) |
| » completion_tokens | integer¦null | false | none | The number of tokens in the completion generated by the model (only set if this is an LLM span) |
| » tokens | integer¦null | false | none | The total number of tokens in the input and output of the experiment event. |
| context | object¦null | false | none | Context is additional information about the code that produced the experiment event. It is essentially the textual counterpart to metrics. Use the caller_* attributes to track the location in code which produced the experiment event |
| » additionalProperties | any | false | none | none |
| » caller_functionname | string¦null | false | none | The function in code which created the experiment event |
| » caller_filename | string¦null | false | none | Name of the file in code where the experiment event was created |
| » caller_lineno | integer¦null | false | none | Line of code where the experiment event was created |
| span_attributes | object¦null | false | none | Human-identifying attributes of the span, such as name, type, etc. |
| » additionalProperties | any | false | none | none |
| » name | string¦null | false | none | Name of the span, for display purposes only |
| » type | string¦null | false | none | Type of the span, for display purposes only |
| id | string¦null | false | none | A unique identifier for the experiment event. If you don't provide one, BrainTrust will generate one for you |
| dataset_record_id | string¦null | false | none | If the experiment is associated to a dataset, this is the event-level dataset id this experiment event is tied to |
| _object_delete | boolean¦null | false | none | Pass _object_delete=true to mark the experiment event deleted. Deleted events will not show up in subsequent fetches for this experiment |
| _is_merge | boolean¦null | false | none | The _is_merge field controls how the row is merged with any existing row with the same id in the DB. By default (or when set to false), the existing row is completely replaced by the new row. When set to true, the new row is deep-merged into the existing rowFor example, say there is an existing row in the DB {"id": "foo", "input": {"a": 5, "b": 10}}. If we merge a new row as {"_is_merge": true, "id": "foo", "input": {"b": 11, "c": 20}}, the new row will be {"id": "foo", "input": {"a": 5, "b": 11, "c": 20}}. If we replace the new row as {"id": "foo", "input": {"b": 11, "c": 20}}, the new row will be {"id": "foo", "input": {"b": 11, "c": 20}} |
| _parent_id | string¦null | false | none | Use the _parent_id field to create this row as a subspan of an existing row. It cannot be specified alongside _is_merge=true. Tracking hierarchical relationships are important for tracing (see the guide for full details).For example, say we have logged a row {"id": "abc", "input": "foo", "output": "bar", "expected": "boo", "scores": {"correctness": 0.33}}. We can create a sub-span of the parent row by logging {"_parent_id": "abc", "id": "llm_call", "input": {"prompt": "What comes after foo?"}, "output": "bar", "metrics": {"tokens": 1}}. In the webapp, only the root span row "abc" will show up in the summary view. You can view the full trace hierarchy (in this case, the "llm_call" row) by clicking on the "abc" row. |
Enumerated Values
| Property | Value |
|---|---|
| type | llm |
| type | score |
| type | function |
| type | eval |
| type | task |
| type | tool |
| type | null |
| _is_merge | false |
| _is_merge | null |
InsertExperimentEventMerge
{
"input": null,
"output": null,
"expected": null,
"scores": {
"property1": 1,
"property2": 1
},
"metadata": {
"property1": null,
"property2": null
},
"tags": ["string"],
"metrics": {
"start": 0,
"end": 0,
"prompt_tokens": 0,
"completion_tokens": 0,
"tokens": 0,
"property1": null,
"property2": null
},
"context": {
"caller_functionname": "string",
"caller_filename": "string",
"caller_lineno": 0,
"property1": null,
"property2": null
},
"span_attributes": {
"name": "string",
"type": "llm",
"property1": null,
"property2": null
},
"id": "string",
"dataset_record_id": "string",
"_object_delete": true,
"_is_merge": true,
"_merge_paths": [["string"]]
}Properties
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| input | any | false | none | The arguments that uniquely define a test case (an arbitrary, JSON serializable object). Later on, Braintrust will use the input to know whether two test cases are the same between experiments, so they should not contain experiment-specific state. A simple rule of thumb is that if you run the same experiment twice, the input should be identical |
| output | any | false | none | The output of your application, including post-processing (an arbitrary, JSON serializable object), that allows you to determine whether the result is correct or not. For example, in an app that generates SQL queries, the output should be the result of the SQL query generated by the model, not the query itself, because there may be multiple valid queries that answer a single question |
| expected | any | false | none | The ground truth value (an arbitrary, JSON serializable object) that you'd compare to output to determine if your output value is correct or not. Braintrust currently does not compare output to expected for you, since there are so many different ways to do that correctly. Instead, these values are just used to help you navigate your experiments while digging into analyses. However, we may later use these values to re-score outputs or fine-tune your models |
| scores | object¦null | false | none | A dictionary of numeric values (between 0 and 1) to log. The scores should give you a variety of signals that help you determine how accurate the outputs are compared to what you expect and diagnose failures. For example, a summarization app might have one score that tells you how accurate the summary is, and another that measures the word similarity between the generated and grouth truth summary. The word similarity score could help you determine whether the summarization was covering similar concepts or not. You can use these scores to help you sort, filter, and compare experiments |
| » additionalProperties | number¦null | false | none | none |
| metadata | object¦null | false | none | A dictionary with additional data about the test example, model outputs, or just about anything else that's relevant, that you can use to help find and analyze examples later. For example, you could log the prompt, example's id, or anything else that would be useful to slice/dice later. The values in metadata can be any JSON-serializable type, but its keys must be strings |
| » additionalProperties | any | false | none | none |
| tags | [string]¦null | false | none | A list of tags to log |
| metrics | object¦null | false | none | Metrics are numerical measurements tracking the execution of the code that produced the experiment event. Use "start" and "end" to track the time span over which the experiment event was produced |
| » additionalProperties | any | false | none | none |
| » start | number¦null | false | none | A unix timestamp recording when the section of code which produced the experiment event started |
| » end | number¦null | false | none | A unix timestamp recording when the section of code which produced the experiment event finished |
| » prompt_tokens | integer¦null | false | none | The number of tokens in the prompt used to generate the experiment event (only set if this is an LLM span) |
| » completion_tokens | integer¦null | false | none | The number of tokens in the completion generated by the model (only set if this is an LLM span) |
| » tokens | integer¦null | false | none | The total number of tokens in the input and output of the experiment event. |
| context | object¦null | false | none | Context is additional information about the code that produced the experiment event. It is essentially the textual counterpart to metrics. Use the caller_* attributes to track the location in code which produced the experiment event |
| » additionalProperties | any | false | none | none |
| » caller_functionname | string¦null | false | none | The function in code which created the experiment event |
| » caller_filename | string¦null | false | none | Name of the file in code where the experiment event was created |
| » caller_lineno | integer¦null | false | none | Line of code where the experiment event was created |
| span_attributes | object¦null | false | none | Human-identifying attributes of the span, such as name, type, etc. |
| » additionalProperties | any | false | none | none |
| » name | string¦null | false | none | Name of the span, for display purposes only |
| » type | string¦null | false | none | Type of the span, for display purposes only |
| id | string¦null | false | none | A unique identifier for the experiment event. If you don't provide one, BrainTrust will generate one for you |
| dataset_record_id | string¦null | false | none | If the experiment is associated to a dataset, this is the event-level dataset id this experiment event is tied to |
| _object_delete | boolean¦null | false | none | Pass _object_delete=true to mark the experiment event deleted. Deleted events will not show up in subsequent fetches for this experiment |
| _is_merge | boolean | true | none | The _is_merge field controls how the row is merged with any existing row with the same id in the DB. By default (or when set to false), the existing row is completely replaced by the new row. When set to true, the new row is deep-merged into the existing rowFor example, say there is an existing row in the DB {"id": "foo", "input": {"a": 5, "b": 10}}. If we merge a new row as {"_is_merge": true, "id": "foo", "input": {"b": 11, "c": 20}}, the new row will be {"id": "foo", "input": {"a": 5, "b": 11, "c": 20}}. If we replace the new row as {"id": "foo", "input": {"b": 11, "c": 20}}, the new row will be {"id": "foo", "input": {"b": 11, "c": 20}} |
| _merge_paths | [array]¦null | false | none | The _merge_paths field allows controlling the depth of the merge. It can only be specified alongside _is_merge=true. _merge_paths is a list of paths, where each path is a list of field names. The deep merge will not descend below any of the specified merge paths.For example, say there is an existing row in the DB {"id": "foo", "input": {"a": {"b": 10}, "c": {"d": 20}}, "output": {"a": 20}}. If we merge a new row as {"_is_merge": true, "_merge_paths": [["input", "a"], ["output"]], "input": {"a": {"q": 30}, "c": {"e": 30}, "bar": "baz"}, "output": {"d": 40}}, the new row will be {"id": "foo": "input": {"a": {"q": 30}, "c": {"d": 20, "e": 30}, "bar": "baz"}, "output": {"d": 40}}. In this case, due to the merge paths, we have replaced input.a and output, but have still deep-merged input and input.c. |
Enumerated Values
| Property | Value |
|---|---|
| type | llm |
| type | score |
| type | function |
| type | eval |
| type | task |
| type | tool |
| type | null |
| _is_merge | true |
InsertExperimentEvent
{
"input": null,
"output": null,
"expected": null,
"scores": {
"property1": 1,
"property2": 1
},
"metadata": {
"property1": null,
"property2": null
},
"tags": ["string"],
"metrics": {
"start": 0,
"end": 0,
"prompt_tokens": 0,
"completion_tokens": 0,
"tokens": 0,
"property1": null,
"property2": null
},
"context": {
"caller_functionname": "string",
"caller_filename": "string",
"caller_lineno": 0,
"property1": null,
"property2": null
},
"span_attributes": {
"name": "string",
"type": "llm",
"property1": null,
"property2": null
},
"id": "string",
"dataset_record_id": "string",
"_object_delete": true,
"_is_merge": false,
"_parent_id": "string"
}An experiment event
Properties
anyOf
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | InsertExperimentEventReplace | false | none | none |
or
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | InsertExperimentEventMerge | false | none | none |
InsertExperimentEventRequest
{
"events": [
{
"input": null,
"output": null,
"expected": null,
"scores": {
"property1": 1,
"property2": 1
},
"metadata": {
"property1": null,
"property2": null
},
"tags": ["string"],
"metrics": {
"start": 0,
"end": 0,
"prompt_tokens": 0,
"completion_tokens": 0,
"tokens": 0,
"property1": null,
"property2": null
},
"context": {
"caller_functionname": "string",
"caller_filename": "string",
"caller_lineno": 0,
"property1": null,
"property2": null
},
"span_attributes": {
"name": "string",
"type": "llm",
"property1": null,
"property2": null
},
"id": "string",
"dataset_record_id": "string",
"_object_delete": true,
"_is_merge": false,
"_parent_id": "string"
}
]
}Properties
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| events | [InsertExperimentEvent] | true | none | A list of experiment events to insert |
ExperimentEvent
{
"id": "string",
"dataset_record_id": "string",
"_xact_id": "string",
"created": "2019-08-24T14:15:22Z",
"project_id": "405d8375-3514-403b-8c43-83ae74cfe0e9",
"experiment_id": "916afd89-cac5-4339-9c59-dd068abdfa69",
"input": null,
"output": null,
"expected": null,
"scores": {
"property1": 1,
"property2": 1
},
"metadata": {
"property1": null,
"property2": null
},
"tags": ["string"],
"metrics": {
"start": 0,
"end": 0,
"prompt_tokens": 0,
"completion_tokens": 0,
"tokens": 0,
"property1": null,
"property2": null
},
"context": {
"caller_functionname": "string",
"caller_filename": "string",
"caller_lineno": 0,
"property1": null,
"property2": null
},
"span_id": "string",
"span_parents": ["string"],
"root_span_id": "string",
"span_attributes": {
"name": "string",
"type": "llm",
"property1": null,
"property2": null
}
}Properties
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| id | string | true | none | A unique identifier for the experiment event. If you don't provide one, BrainTrust will generate one for you |
| dataset_record_id | string¦null | false | none | If the experiment is associated to a dataset, this is the event-level dataset id this experiment event is tied to |
| _xact_id | string | true | none | The transaction id of an event is unique to the network operation that processed the event insertion. Transaction ids are monotonically increasing over time and can be used to retrieve a versioned snapshot of the experiment (see the version parameter) |
| created | string(date-time) | true | none | The timestamp the experiment event was created |
| project_id | string(uuid) | true | none | Unique identifier for the project that the experiment belongs under |
| experiment_id | string(uuid) | true | none | Unique identifier for the experiment |
| input | any | false | none | The arguments that uniquely define a test case (an arbitrary, JSON serializable object). Later on, Braintrust will use the input to know whether two test cases are the same between experiments, so they should not contain experiment-specific state. A simple rule of thumb is that if you run the same experiment twice, the input should be identical |
| output | any | false | none | The output of your application, including post-processing (an arbitrary, JSON serializable object), that allows you to determine whether the result is correct or not. For example, in an app that generates SQL queries, the output should be the result of the SQL query generated by the model, not the query itself, because there may be multiple valid queries that answer a single question |
| expected | any | false | none | The ground truth value (an arbitrary, JSON serializable object) that you'd compare to output to determine if your output value is correct or not. Braintrust currently does not compare output to expected for you, since there are so many different ways to do that correctly. Instead, these values are just used to help you navigate your experiments while digging into analyses. However, we may later use these values to re-score outputs or fine-tune your models |
| scores | object¦null | false | none | A dictionary of numeric values (between 0 and 1) to log. The scores should give you a variety of signals that help you determine how accurate the outputs are compared to what you expect and diagnose failures. For example, a summarization app might have one score that tells you how accurate the summary is, and another that measures the word similarity between the generated and grouth truth summary. The word similarity score could help you determine whether the summarization was covering similar concepts or not. You can use these scores to help you sort, filter, and compare experiments |
| » additionalProperties | number¦null | false | none | none |
| metadata | object¦null | false | none | A dictionary with additional data about the test example, model outputs, or just about anything else that's relevant, that you can use to help find and analyze examples later. For example, you could log the prompt, example's id, or anything else that would be useful to slice/dice later. The values in metadata can be any JSON-serializable type, but its keys must be strings |
| » additionalProperties | any | false | none | none |
| tags | [string]¦null | false | none | A list of tags to log |
| metrics | object¦null | false | none | Metrics are numerical measurements tracking the execution of the code that produced the experiment event. Use "start" and "end" to track the time span over which the experiment event was produced |
| » additionalProperties | any | false | none | none |
| » start | number¦null | false | none | A unix timestamp recording when the section of code which produced the experiment event started |
| » end | number¦null | false | none | A unix timestamp recording when the section of code which produced the experiment event finished |
| » prompt_tokens | integer¦null | false | none | The number of tokens in the prompt used to generate the experiment event (only set if this is an LLM span) |
| » completion_tokens | integer¦null | false | none | The number of tokens in the completion generated by the model (only set if this is an LLM span) |
| » tokens | integer¦null | false | none | The total number of tokens in the input and output of the experiment event. |
| context | object¦null | false | none | Context is additional information about the code that produced the experiment event. It is essentially the textual counterpart to metrics. Use the caller_* attributes to track the location in code which produced the experiment event |
| » additionalProperties | any | false | none | none |
| » caller_functionname | string¦null | false | none | The function in code which created the experiment event |
| » caller_filename | string¦null | false | none | Name of the file in code where the experiment event was created |
| » caller_lineno | integer¦null | false | none | Line of code where the experiment event was created |
| span_id | string | true | none | A unique identifier used to link different experiment events together as part of a full trace. See the tracing guide for full details on tracing |
| span_parents | [string]¦null | false | none | An array of the parent span_ids of this experiment event. This should be empty for the root span of a trace, and should most often contain just one parent element for subspans |
| root_span_id | string | true | none | The span_id of the root of the trace this experiment event belongs to |
| span_attributes | object¦null | false | none | Human-identifying attributes of the span, such as name, type, etc. |
| » additionalProperties | any | false | none | none |
| » name | string¦null | false | none | Name of the span, for display purposes only |
| » type | string¦null | false | none | Type of the span, for display purposes only |
Enumerated Values
| Property | Value |
|---|---|
| type | llm |
| type | score |
| type | function |
| type | eval |
| type | task |
| type | tool |
| type | null |
FetchExperimentEventsResponse
{
"events": [
{
"id": "string",
"dataset_record_id": "string",
"_xact_id": "string",
"created": "2019-08-24T14:15:22Z",
"project_id": "405d8375-3514-403b-8c43-83ae74cfe0e9",
"experiment_id": "916afd89-cac5-4339-9c59-dd068abdfa69",
"input": null,
"output": null,
"expected": null,
"scores": {
"property1": 1,
"property2": 1
},
"metadata": {
"property1": null,
"property2": null
},
"tags": ["string"],
"metrics": {
"start": 0,
"end": 0,
"prompt_tokens": 0,
"completion_tokens": 0,
"tokens": 0,
"property1": null,
"property2": null
},
"context": {
"caller_functionname": "string",
"caller_filename": "string",
"caller_lineno": 0,
"property1": null,
"property2": null
},
"span_id": "string",
"span_parents": ["string"],
"root_span_id": "string",
"span_attributes": {
"name": "string",
"type": "llm",
"property1": null,
"property2": null
}
}
],
"cursor": "string"
}Properties
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| events | [ExperimentEvent] | true | none | A list of fetched events |
| cursor | string¦null | false | none | Pagination cursor Pass this string directly as the cursor param to your next fetch request to get the next page of results. Not provided if the returned result set is empty. |
FeedbackExperimentItem
{
"id": "string",
"scores": {
"property1": 1,
"property2": 1
},
"expected": null,
"comment": "string",
"metadata": {
"property1": null,
"property2": null
},
"source": "app"
}Properties
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| id | string | true | none | The id of the experiment event to log feedback for. This is the row id returned by POST /v1/experiment/{experiment_id}/insert |
| scores | object¦null | false | none | A dictionary of numeric values (between 0 and 1) to log. These scores will be merged into the existing scores for the experiment event |
| » additionalProperties | number¦null | false | none | none |
| expected | any | false | none | The ground truth value (an arbitrary, JSON serializable object) that you'd compare to output to determine if your output value is correct or not |
| comment | string¦null | false | none | An optional comment string to log about the experiment event |
| metadata | object¦null | false | none | A dictionary with additional data about the feedback. If you have a user_id, you can log it here and access it in the Braintrust UI. |
| » additionalProperties | any | false | none | none |
| source | string¦null | false | none | The source of the feedback. Must be one of "external" (default), "app", or "api" |
Enumerated Values
| Property | Value |
|---|---|
| source | app |
| source | api |
| source | external |
| source | null |
FeedbackExperimentEventRequest
{
"feedback": [
{
"id": "string",
"scores": {
"property1": 1,
"property2": 1
},
"expected": null,
"comment": "string",
"metadata": {
"property1": null,
"property2": null
},
"source": "app"
}
]
}Properties
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| feedback | [FeedbackExperimentItem] | true | none | A list of experiment feedback items |
ScoreSummary
{
"name": "string",
"score": 1,
"diff": -1,
"improvements": 0,
"regressions": 0
}Summary of a score's performance
Properties
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| name | string | true | none | Name of the score |
| score | number | true | none | Average score across all examples |
| diff | number | false | none | Difference in score between the current and comparison experiment |
| improvements | integer | true | none | Number of improvements in the score |
| regressions | integer | true | none | Number of regressions in the score |
MetricSummary
{
"name": "string",
"metric": 0,
"unit": "string",
"diff": 0,
"improvements": 0,
"regressions": 0
}Summary of a metric's performance
Properties
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| name | string | true | none | Name of the metric |
| metric | number | true | none | Average metric across all examples |
| unit | string | true | none | Unit label for the metric |
| diff | number | false | none | Difference in metric between the current and comparison experiment |
| improvements | integer | true | none | Number of improvements in the metric |
| regressions | integer | true | none | Number of regressions in the metric |
SummarizeExperimentResponse
{
"project_name": "string",
"experiment_name": "string",
"project_url": "http://example.com",
"experiment_url": "http://example.com",
"comparison_experiment_name": "string",
"scores": {
"property1": {
"name": "string",
"score": 1,
"diff": -1,
"improvements": 0,
"regressions": 0
},
"property2": {
"name": "string",
"score": 1,
"diff": -1,
"improvements": 0,
"regressions": 0
}
},
"metrics": {
"property1": {
"name": "string",
"metric": 0,
"unit": "string",
"diff": 0,
"improvements": 0,
"regressions": 0
},
"property2": {
"name": "string",
"metric": 0,
"unit": "string",
"diff": 0,
"improvements": 0,
"regressions": 0
}
}
}Summary of an experiment
Properties
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| project_name | string | true | none | Name of the project that the experiment belongs to |
| experiment_name | string | true | none | Name of the experiment |
| project_url | string(uri) | true | none | URL to the project's page in the Braintrust app |
| experiment_url | string(uri) | true | none | URL to the experiment's page in the Braintrust app |
| comparison_experiment_name | string¦null | false | none | The experiment which scores are baselined against |
| scores | object¦null | false | none | Summary of the experiment's scores |
| » additionalProperties | ScoreSummary | false | none | Summary of a score's performance |
| metrics | object¦null | false | none | Summary of the experiment's metrics |
| » additionalProperties | MetricSummary | false | none | Summary of a metric's performance |
Dataset
{
"id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
"project_id": "405d8375-3514-403b-8c43-83ae74cfe0e9",
"name": "string",
"description": "string",
"created": "2019-08-24T14:15:22Z",
"deleted_at": "2019-08-24T14:15:22Z",
"user_id": "a169451c-8525-4352-b8ca-070dd449a1a5"
}Properties
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| id | string(uuid) | true | none | Unique identifier for the dataset |
| project_id | string(uuid)¦null | false | none | Unique identifier for the project that the dataset belongs under |
| name | string | true | none | Name of the dataset. Within a project, dataset names are unique |
| description | string¦null | false | none | Textual description of the dataset |
| created | string(date-time)¦null | false | none | Date of dataset creation |
| deleted_at | string(date-time)¦null | false | none | Date of dataset deletion, or null if the dataset is still active |
| user_id | string(uuid)¦null | false | none | Identifies the user who created the dataset |
CreateDataset
{
"project_id": "405d8375-3514-403b-8c43-83ae74cfe0e9",
"name": "string",
"description": "string"
}Properties
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| project_id | string(uuid)¦null | false | none | Unique identifier for the project that the dataset belongs under |
| name | string | true | none | Name of the dataset. Within a project, dataset names are unique |
| description | string¦null | false | none | Textual description of the dataset |
PatchDataset
{
"name": "string",
"description": "string"
}Properties
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| name | string¦null | false | none | Name of the dataset. Within a project, dataset names are unique |
| description | string¦null | false | none | Textual description of the dataset |
InsertDatasetEventReplace
{
"input": null,
"expected": null,
"metadata": {
"property1": null,
"property2": null
},
"tags": ["string"],
"id": "string",
"_object_delete": true,
"_is_merge": false,
"_parent_id": "string"
}Properties
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| input | any | false | none | The argument that uniquely define an input case (an arbitrary, JSON serializable object) |
| expected | any | false | none | The output of your application, including post-processing (an arbitrary, JSON serializable object) |
| metadata | object¦null | false | none | A dictionary with additional data about the test example, model outputs, or just about anything else that's relevant, that you can use to help find and analyze examples later. For example, you could log the prompt, example's id, or anything else that would be useful to slice/dice later. The values in metadata can be any JSON-serializable type, but its keys must be strings |
| » additionalProperties | any | false | none | none |
| tags | [string]¦null | false | none | A list of tags to log |
| id | string¦null | false | none | A unique identifier for the dataset event. If you don't provide one, BrainTrust will generate one for you |
| _object_delete | boolean¦null | false | none | Pass _object_delete=true to mark the dataset event deleted. Deleted events will not show up in subsequent fetches for this dataset |
| _is_merge | boolean¦null | false | none | The _is_merge field controls how the row is merged with any existing row with the same id in the DB. By default (or when set to false), the existing row is completely replaced by the new row. When set to true, the new row is deep-merged into the existing rowFor example, say there is an existing row in the DB {"id": "foo", "input": {"a": 5, "b": 10}}. If we merge a new row as {"_is_merge": true, "id": "foo", "input": {"b": 11, "c": 20}}, the new row will be {"id": "foo", "input": {"a": 5, "b": 11, "c": 20}}. If we replace the new row as {"id": "foo", "input": {"b": 11, "c": 20}}, the new row will be {"id": "foo", "input": {"b": 11, "c": 20}} |
| _parent_id | string¦null | false | none | Use the _parent_id field to create this row as a subspan of an existing row. It cannot be specified alongside _is_merge=true. Tracking hierarchical relationships are important for tracing (see the guide for full details).For example, say we have logged a row {"id": "abc", "input": "foo", "output": "bar", "expected": "boo", "scores": {"correctness": 0.33}}. We can create a sub-span of the parent row by logging {"_parent_id": "abc", "id": "llm_call", "input": {"prompt": "What comes after foo?"}, "output": "bar", "metrics": {"tokens": 1}}. In the webapp, only the root span row "abc" will show up in the summary view. You can view the full trace hierarchy (in this case, the "llm_call" row) by clicking on the "abc" row. |
Enumerated Values
| Property | Value |
|---|---|
| _is_merge | false |
| _is_merge | null |
InsertDatasetEventMerge
{
"input": null,
"expected": null,
"metadata": {
"property1": null,
"property2": null
},
"tags": ["string"],
"id": "string",
"_object_delete": true,
"_is_merge": true,
"_merge_paths": [["string"]]
}Properties
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| input | any | false | none | The argument that uniquely define an input case (an arbitrary, JSON serializable object) |
| expected | any | false | none | The output of your application, including post-processing (an arbitrary, JSON serializable object) |
| metadata | object¦null | false | none | A dictionary with additional data about the test example, model outputs, or just about anything else that's relevant, that you can use to help find and analyze examples later. For example, you could log the prompt, example's id, or anything else that would be useful to slice/dice later. The values in metadata can be any JSON-serializable type, but its keys must be strings |
| » additionalProperties | any | false | none | none |
| tags | [string]¦null | false | none | A list of tags to log |
| id | string¦null | false | none | A unique identifier for the dataset event. If you don't provide one, BrainTrust will generate one for you |
| _object_delete | boolean¦null | false | none | Pass _object_delete=true to mark the dataset event deleted. Deleted events will not show up in subsequent fetches for this dataset |
| _is_merge | boolean | true | none | The _is_merge field controls how the row is merged with any existing row with the same id in the DB. By default (or when set to false), the existing row is completely replaced by the new row. When set to true, the new row is deep-merged into the existing rowFor example, say there is an existing row in the DB {"id": "foo", "input": {"a": 5, "b": 10}}. If we merge a new row as {"_is_merge": true, "id": "foo", "input": {"b": 11, "c": 20}}, the new row will be {"id": "foo", "input": {"a": 5, "b": 11, "c": 20}}. If we replace the new row as {"id": "foo", "input": {"b": 11, "c": 20}}, the new row will be {"id": "foo", "input": {"b": 11, "c": 20}} |
| _merge_paths | [array]¦null | false | none | The _merge_paths field allows controlling the depth of the merge. It can only be specified alongside _is_merge=true. _merge_paths is a list of paths, where each path is a list of field names. The deep merge will not descend below any of the specified merge paths.For example, say there is an existing row in the DB {"id": "foo", "input": {"a": {"b": 10}, "c": {"d": 20}}, "output": {"a": 20}}. If we merge a new row as {"_is_merge": true, "_merge_paths": [["input", "a"], ["output"]], "input": {"a": {"q": 30}, "c": {"e": 30}, "bar": "baz"}, "output": {"d": 40}}, the new row will be {"id": "foo": "input": {"a": {"q": 30}, "c": {"d": 20, "e": 30}, "bar": "baz"}, "output": {"d": 40}}. In this case, due to the merge paths, we have replaced input.a and output, but have still deep-merged input and input.c. |
Enumerated Values
| Property | Value |
|---|---|
| _is_merge | true |
InsertDatasetEvent
{
"input": null,
"expected": null,
"metadata": {
"property1": null,
"property2": null
},
"tags": ["string"],
"id": "string",
"_object_delete": true,
"_is_merge": false,
"_parent_id": "string"
}A dataset event
Properties
anyOf
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | InsertDatasetEventReplace | false | none | none |
or
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| anonymous | InsertDatasetEventMerge | false | none | none |
InsertDatasetEventRequest
{
"events": [
{
"input": null,
"expected": null,
"metadata": {
"property1": null,
"property2": null
},
"tags": ["string"],
"id": "string",
"_object_delete": true,
"_is_merge": false,
"_parent_id": "string"
}
]
}Properties
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| events | [InsertDatasetEvent] | true | none | A list of dataset events to insert |
DatasetEvent
{
"id": "string",
"_xact_id": "string",
"created": "2019-08-24T14:15:22Z",
"project_id": "405d8375-3514-403b-8c43-83ae74cfe0e9",
"dataset_id": "8c4c51f1-f6f3-43bc-b65d-7415e8ef22c0",
"input": null,
"expected": null,
"metadata": {
"property1": null,
"property2": null
},
"tags": ["string"],
"span_id": "string",
"root_span_id": "string"
}Properties
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| id | string | true | none | A unique identifier for the dataset event. If you don't provide one, BrainTrust will generate one for you |
| _xact_id | string | true | none | The transaction id of an event is unique to the network operation that processed the event insertion. Transaction ids are monotonically increasing over time and can be used to retrieve a versioned snapshot of the dataset (see the version parameter) |
| created | string(date-time) | true | none | The timestamp the dataset event was created |
| project_id | string(uuid)¦null | false | none | Unique identifier for the project that the dataset belongs under |
| dataset_id | string(uuid) | true | none | Unique identifier for the dataset |
| input | any | false | none | The argument that uniquely define an input case (an arbitrary, JSON serializable object) |
| expected | any | false | none | The output of your application, including post-processing (an arbitrary, JSON serializable object) |
| metadata | object¦null | false | none | A dictionary with additional data about the test example, model outputs, or just about anything else that's relevant, that you can use to help find and analyze examples later. For example, you could log the prompt, example's id, or anything else that would be useful to slice/dice later. The values in metadata can be any JSON-serializable type, but its keys must be strings |
| » additionalProperties | any | false | none | none |
| tags | [string]¦null | false | none | A list of tags to log |
| span_id | string | true | none | A unique identifier used to link different dataset events together as part of a full trace. See the tracing guide for full details on tracing |
| root_span_id | string | true | none | The span_id of the root of the trace this dataset event belongs to |
FetchDatasetEventsResponse
{
"events": [
{
"id": "string",
"_xact_id": "string",
"created": "2019-08-24T14:15:22Z",
"project_id": "405d8375-3514-403b-8c43-83ae74cfe0e9",
"dataset_id": "8c4c51f1-f6f3-43bc-b65d-7415e8ef22c0",
"input": null,
"expected": null,
"metadata": {
"property1": null,
"property2": null
},
"tags": ["string"],
"span_id": "string",
"root_span_id": "string"
}
],
"cursor": "string"
}Properties
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| events | [DatasetEvent] | true | none | A list of fetched events |
| cursor | string¦null | false | none | Pagination cursor Pass this string directly as the cursor param to your next fetch request to get the next page of results. Not provided if the returned result set is empty. |
FeedbackDatasetItem
{
"id": "string",
"comment": "string",
"metadata": {
"property1": null,
"property2": null
},
"source": "app"
}Properties
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| id | string | true | none | The id of the dataset event to log feedback for. This is the row id returned by POST /v1/dataset/{dataset_id}/insert |
| comment | string¦null | false | none | An optional comment string to log about the dataset event |
| metadata | object¦null | false | none | A dictionary with additional data about the feedback. If you have a user_id, you can log it here and access it in the Braintrust UI. |
| » additionalProperties | any | false | none | none |
| source | string¦null | false | none | The source of the feedback. Must be one of "external" (default), "app", or "api" |
Enumerated Values
| Property | Value |
|---|---|
| source | app |
| source | api |
| source | external |
| source | null |
FeedbackDatasetEventRequest
{
"feedback": [
{
"id": "string",
"comment": "string",
"metadata": {
"property1": null,
"property2": null
},
"source": "app"
}
]
}Properties
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| feedback | [FeedbackDatasetItem] | true | none | A list of dataset feedback items |
DataSummary
{
"total_records": 0
}Summary of a dataset's data
Properties
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| total_records | integer | true | none | Total number of records in the dataset |
SummarizeDatasetResponse
{
"project_name": "string",
"dataset_name": "string",
"project_url": "http://example.com",
"dataset_url": "http://example.com",
"data_summary": {
"total_records": 0
}
}Summary of a dataset
Properties
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| project_name | string | true | none | Name of the project that the dataset belongs to |
| dataset_name | string | true | none | Name of the dataset |
| project_url | string(uri) | true | none | URL to the project's page in the Braintrust app |
| dataset_url | string(uri) | true | none | URL to the dataset's page in the Braintrust app |
| data_summary | DataSummary | false | none | Summary of a dataset's data |
PromptData
{
"prompt": {
"type": "completion",
"content": "string"
},
"options": {
"model": "string",
"params": {
"use_cache": true,
"temperature": 0,
"top_p": 0,
"max_tokens": 0,
"frequency_penalty": 0,
"presence_penalty": 0,
"response_format": {
"type": "json_object"
},
"tool_choice": "auto",
"function_call": "auto",
"n": 0,
"stop": ["string"]
},
"position": "string"
},
"origin": {
"prompt_id": "string",
"project_id": "string",
"prompt_version": "string"
}
}The prompt, model, and its parameters
Properties
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| prompt | any | false | none | none |
anyOf
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| » anonymous | object | false | none | none |
| »» type | string | true | none | none |
| »» content | string | true | none | none |
or
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| » anonymous | object | false | none | none |
| »» type | string | true | none | none |
| »» messages | [anyOf] | true | none | none |
anyOf
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| »»» anonymous | object | false | none | none |
| »»»» content | string | false | none | none |
| »»»» role | string | true | none | none |
| »»»» name | string | false | none | none |
or
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| »»» anonymous | object | false | none | none |
| »»»» content | any | false | none | none |
anyOf
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| »»»»» anonymous | string | false | none | none |
or
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| »»»»» anonymous | [anyOf] | false | none | none |
anyOf
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| »»»»»» anonymous | object | false | none | none |
| »»»»»»» text | string | false | none | none |
| »»»»»»» type | string | true | none | none |
or
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| »»»»»» anonymous | object | false | none | none |
| »»»»»»» image_url | object | true | none | none |
| »»»»»»»» url | string | true | none | none |
| »»»»»»»» detail | any | false | none | none |
anyOf
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| »»»»»»»»» anonymous | string | false | none | none |
or
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| »»»»»»»»» anonymous | string | false | none | none |
or
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| »»»»»»»»» anonymous | string | false | none | none |
continued
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| »»»»»»» type | string | true | none | none |
| »»»» role | string | true | none | none |
| »»»» name | string | false | none | none |
or
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| »»» anonymous | object | false | none | none |
| »»»» role | string | true | none | none |
| »»»» content | string¦null | false | none | none |
| »»»» function_call | object | false | none | none |
| »»»»» arguments | string | true | none | none |
| »»»»» name | string | true | none | none |
| »»»» name | string | false | none | none |
| »»»» tool_calls | [object] | false | none | none |
| »»»»» id | string | true | none | none |
| »»»»» function | object | true | none | none |
| »»»»»» arguments | string | true | none | none |
| »»»»»» name | string | true | none | none |
| »»»»» type | string | true | none | none |
or
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| »»» anonymous | object | false | none | none |
| »»»» content | string | false | none | none |
| »»»» role | string | true | none | none |
| »»»» tool_call_id | string | true | none | none |
or
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| »»» anonymous | object | false | none | none |
| »»»» content | string | false | none | none |
| »»»» name | string | true | none | none |
| »»»» role | string | true | none | none |
continued
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| »» tools | string | false | none | none |
or
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| » anonymous | object¦null | false | none | none |
continued
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| options | object¦null | false | none | none |
| » model | string | false | none | none |
| » params | any | false | none | none |
anyOf
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| »» anonymous | object | false | none | none |
| »»» use_cache | boolean | false | none | none |
| »»» temperature | number | false | none | none |
| »»» top_p | number | false | none | none |
| »»» max_tokens | number | false | none | none |
| »»» frequency_penalty | number | false | none | none |
| »»» presence_penalty | number | false | none | none |
| »»» response_format | object¦null | false | none | none |
| »»»» type | string | true | none | none |
| »»» tool_choice | any | false | none | none |
anyOf
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| »»»» anonymous | string | false | none | none |
or
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| »»»» anonymous | string | false | none | none |
or
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| »»»» anonymous | object | false | none | none |
| »»»»» type | string | true | none | none |
| »»»»» function | object | true | none | none |
| »»»»»» name | string | true | none | none |
continued
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| »»» function_call | any | false | none | none |
anyOf
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| »»»» anonymous | string | false | none | none |
or
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| »»»» anonymous | string | false | none | none |
or
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| »»»» anonymous | object | false | none | none |
| »»»»» name | string | true | none | none |
continued
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| »»» n | number | false | none | none |
| »»» stop | [string] | false | none | none |
or
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| »» anonymous | object | false | none | none |
| »»» use_cache | boolean | false | none | none |
| »»» max_tokens | number | true | none | none |
| »»» temperature | number | true | none | none |
| »»» top_p | number | false | none | none |
| »»» top_k | number | false | none | none |
| »»» stop_sequences | [string] | false | none | none |
| »»» max_tokens_to_sample | number | false | none | This is a legacy parameter that should not be used. |
or
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| »» anonymous | object | false | none | none |
| »»» use_cache | boolean | false | none | none |
| »»» temperature | number | true | none | none |
| »»» maxOutputTokens | number | false | none | none |
| »»» topP | number | false | none | none |
| »»» topK | number | false | none | none |
or
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| »» anonymous | object | false | none | none |
| »»» use_cache | boolean | false | none | none |
continued
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| » position | string | false | none | none |
| origin | object¦null | false | none | none |
| » prompt_id | string | false | none | none |
| » project_id | string | false | none | none |
| » prompt_version | string | false | none | none |
Enumerated Values
| Property | Value |
|---|---|
| type | completion |
| type | chat |
| role | system |
| type | text |
| anonymous | auto |
| anonymous | low |
| anonymous | high |
| type | image_url |
| role | user |
| role | assistant |
| type | function |
| role | tool |
| role | function |
| type | json_object |
| anonymous | auto |
| anonymous | none |
| type | function |
| anonymous | auto |
| anonymous | none |
Prompt
{
"id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
"_xact_id": "string",
"project_id": "405d8375-3514-403b-8c43-83ae74cfe0e9",
"log_id": "p",
"org_id": "a40f5d1f-d889-42e9-94ea-b9b33585fc6b",
"name": "string",
"slug": "string",
"description": "string",
"created": "2019-08-24T14:15:22Z",
"prompt_data": {
"prompt": {
"type": "completion",
"content": "string"
},
"options": {
"model": "string",
"params": {
"use_cache": true,
"temperature": 0,
"top_p": 0,
"max_tokens": 0,
"frequency_penalty": 0,
"presence_penalty": 0,
"response_format": {
"type": "json_object"
},
"tool_choice": "auto",
"function_call": "auto",
"n": 0,
"stop": ["string"]
},
"position": "string"
},
"origin": {
"prompt_id": "string",
"project_id": "string",
"prompt_version": "string"
}
},
"tags": ["string"],
"metadata": {
"property1": null,
"property2": null
}
}Properties
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| id | string(uuid) | true | none | Unique identifier for the prompt |
| _xact_id | string | true | none | The transaction id of an event is unique to the network operation that processed the event insertion. Transaction ids are monotonically increasing over time and can be used to retrieve a versioned snapshot of the prompt (see the version parameter) |
| project_id | string(uuid) | true | none | Unique identifier for the project that the prompt belongs under |
| log_id | string | true | none | A literal 'p' which identifies the object as a project prompt |
| org_id | string(uuid) | true | none | Unique identifier for the organization |
| name | string | true | none | Name of the prompt |
| slug | string | true | none | Unique identifier for the prompt |
| description | string¦null | false | none | Textual description of the prompt |
| created | string(date-time)¦null | false | none | Date of prompt creation |
| prompt_data | PromptData | false | none | The prompt, model, and its parameters |
| tags | [string]¦null | false | none | A list of tags for the prompt |
| metadata | object¦null | false | none | User-controlled metadata about the prompt |
| » additionalProperties | any | false | none | none |
Enumerated Values
| Property | Value |
|---|---|
| log_id | p |
CreatePrompt
{
"project_id": "405d8375-3514-403b-8c43-83ae74cfe0e9",
"name": "string",
"slug": "string",
"description": "string",
"prompt_data": {
"prompt": {
"type": "completion",
"content": "string"
},
"options": {
"model": "string",
"params": {
"use_cache": true,
"temperature": 0,
"top_p": 0,
"max_tokens": 0,
"frequency_penalty": 0,
"presence_penalty": 0,
"response_format": {
"type": "json_object"
},
"tool_choice": "auto",
"function_call": "auto",
"n": 0,
"stop": ["string"]
},
"position": "string"
},
"origin": {
"prompt_id": "string",
"project_id": "string",
"prompt_version": "string"
}
},
"tags": ["string"]
}Properties
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| project_id | string(uuid) | true | none | Unique identifier for the project that the prompt belongs under |
| name | string | true | none | Name of the prompt |
| slug | string | true | none | Unique identifier for the prompt |
| description | string¦null | false | none | Textual description of the prompt |
| prompt_data | PromptData | false | none | The prompt, model, and its parameters |
| tags | [string]¦null | false | none | A list of tags for the prompt |
PatchPrompt
{
"name": "string",
"description": "string",
"prompt_data": {
"prompt": {
"type": "completion",
"content": "string"
},
"options": {
"model": "string",
"params": {
"use_cache": true,
"temperature": 0,
"top_p": 0,
"max_tokens": 0,
"frequency_penalty": 0,
"presence_penalty": 0,
"response_format": {
"type": "json_object"
},
"tool_choice": "auto",
"function_call": "auto",
"n": 0,
"stop": ["string"]
},
"position": "string"
},
"origin": {
"prompt_id": "string",
"project_id": "string",
"prompt_version": "string"
}
},
"tags": ["string"]
}Properties
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| name | string¦null | false | none | Name of the prompt |
| description | string¦null | false | none | Textual description of the prompt |
| prompt_data | PromptData | false | none | The prompt, model, and its parameters |
| tags | [string]¦null | false | none | A list of tags for the prompt |
FeedbackPromptItem
{
"id": "string",
"comment": "string",
"metadata": {
"property1": null,
"property2": null
},
"source": "app"
}Properties
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| id | string | true | none | The id of the prompt event to log feedback for. This is the row id returned by POST /v1/prompt/{prompt_id}/insert |
| comment | string¦null | false | none | An optional comment string to log about the prompt event |
| metadata | object¦null | false | none | A dictionary with additional data about the feedback. If you have a user_id, you can log it here and access it in the Braintrust UI. |
| » additionalProperties | any | false | none | none |
| source | string¦null | false | none | The source of the feedback. Must be one of "external" (default), "app", or "api" |
Enumerated Values
| Property | Value |
|---|---|
| source | app |
| source | api |
| source | external |
| source | null |
FeedbackPromptEventRequest
{
"feedback": [
{
"id": "string",
"comment": "string",
"metadata": {
"property1": null,
"property2": null
},
"source": "app"
}
]
}Properties
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| feedback | [FeedbackPromptItem] | true | none | A list of prompt feedback items |
Role
{
"id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
"org_id": "a40f5d1f-d889-42e9-94ea-b9b33585fc6b",
"user_id": "a169451c-8525-4352-b8ca-070dd449a1a5",
"created": "2019-08-24T14:15:22Z",
"name": "string",
"description": "string",
"deleted_at": "2019-08-24T14:15:22Z",
"member_permissions": ["create"],
"member_roles": ["497f6eca-6276-4993-bfeb-53cbbbba6f08"]
}A role is a collection of permissions which can be granted as part of an ACL
Roles can consist of individual permissions, as well as a set of roles they inherit from
Properties
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| id | string(uuid) | true | none | Unique identifier for the role |
| org_id | string(uuid)¦null | false | none | Unique id for the organization that the role belongs under A null org_id indicates a system role, which may be assigned to anybody and inherited by any other role, but cannot be edited. It is forbidden to change the org after creating a role |
| user_id | string(uuid)¦null | false | none | Identifies the user who created the role |
| created | string(date-time)¦null | false | none | Date of role creation |
| name | string | true | none | Name of the role |
| description | string¦null | false | none | Textual description of the role |
| deleted_at | string(date-time)¦null | false | none | Date of role deletion, or null if the role is still active |
| member_permissions | [string]¦null | false | none | Permissions which belong to this role |
| member_roles | [string]¦null | false | none | Ids of the roles this role inherits from An inheriting role has all the permissions contained in its member roles, as well as all of their inherited permissions |
CreateRole
{
"name": "string",
"description": "string",
"member_permissions": ["create"],
"member_roles": ["497f6eca-6276-4993-bfeb-53cbbbba6f08"],
"org_name": "string"
}Properties
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| name | string | true | none | Name of the role |
| description | string¦null | false | none | Textual description of the role |
| member_permissions | [string]¦null | false | none | Permissions which belong to this role |
| member_roles | [string]¦null | false | none | Ids of the roles this role inherits from An inheriting role has all the permissions contained in its member roles, as well as all of their inherited permissions |
| org_name | string¦null | false | none | For nearly all users, this parameter should be unnecessary. But in the rare case that your API key belongs to multiple organizations, you may specify the name of the organization the role belongs in. |
PatchRole
{
"description": "string",
"member_permissions": ["create"],
"member_roles": ["497f6eca-6276-4993-bfeb-53cbbbba6f08"],
"name": "string"
}Properties
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| description | string¦null | false | none | Textual description of the role |
| member_permissions | [string]¦null | false | none | Permissions which belong to this role |
| member_roles | [string]¦null | false | none | Ids of the roles this role inherits from An inheriting role has all the permissions contained in its member roles, as well as all of their inherited permissions |
| name | string¦null | false | none | Name of the role |
Group
{
"id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
"org_id": "a40f5d1f-d889-42e9-94ea-b9b33585fc6b",
"user_id": "a169451c-8525-4352-b8ca-070dd449a1a5",
"created": "2019-08-24T14:15:22Z",
"name": "string",
"description": "string",
"deleted_at": "2019-08-24T14:15:22Z",
"member_users": ["497f6eca-6276-4993-bfeb-53cbbbba6f08"],
"member_groups": ["497f6eca-6276-4993-bfeb-53cbbbba6f08"]
}A group is a collection of users which can be assigned an ACL
Groups can consist of individual users, as well as a set of groups they inherit from
Properties
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| id | string(uuid) | true | none | Unique identifier for the group |
| org_id | string(uuid) | true | none | Unique id for the organization that the group belongs under It is forbidden to change the org after creating a group |
| user_id | string(uuid)¦null | false | none | Identifies the user who created the group |
| created | string(date-time)¦null | false | none | Date of group creation |
| name | string | true | none | Name of the group |
| description | string¦null | false | none | Textual description of the group |
| deleted_at | string(date-time)¦null | false | none | Date of group deletion, or null if the group is still active |
| member_users | [string]¦null | false | none | Ids of users which belong to this group |
| member_groups | [string]¦null | false | none | Ids of the groups this group inherits from An inheriting group has all the users contained in its member groups, as well as all of their inherited users |
CreateGroup
{
"name": "string",
"description": "string",
"member_users": ["497f6eca-6276-4993-bfeb-53cbbbba6f08"],
"member_groups": ["497f6eca-6276-4993-bfeb-53cbbbba6f08"],
"org_name": "string"
}Properties
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| name | string | true | none | Name of the group |
| description | string¦null | false | none | Textual description of the group |
| member_users | [string]¦null | false | none | Ids of users which belong to this group |
| member_groups | [string]¦null | false | none | Ids of the groups this group inherits from An inheriting group has all the users contained in its member groups, as well as all of their inherited users |
| org_name | string¦null | false | none | For nearly all users, this parameter should be unnecessary. But in the rare case that your API key belongs to multiple organizations, you may specify the name of the organization the group belongs in. |
PatchGroup
{
"description": "string",
"member_users": ["497f6eca-6276-4993-bfeb-53cbbbba6f08"],
"member_groups": ["497f6eca-6276-4993-bfeb-53cbbbba6f08"],
"name": "string"
}Properties
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| description | string¦null | false | none | Textual description of the group |
| member_users | [string]¦null | false | none | Ids of users which belong to this group |
| member_groups | [string]¦null | false | none | Ids of the groups this group inherits from An inheriting group has all the users contained in its member groups, as well as all of their inherited users |
| name | string¦null | false | none | Name of the group |
Acl
{
"id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
"object_type": "organization",
"object_id": "463a83d0-a816-4902-abba-2486e0c0a0bb",
"restrict_object_type": "organization",
"_object_org_id": "4d272dc1-1d6f-4a99-8c76-7bcbfc11ce4e",
"created": "2019-08-24T14:15:22Z",
"user_id": "a169451c-8525-4352-b8ca-070dd449a1a5",
"group_id": "306db4e0-7449-4501-b76f-075576fe2d8f",
"permission": "create",
"role_id": "ac4e70c8-d5be-48af-93eb-760f58fc91a9"
}An ACL grants a certain permission or role to a certain user or group on an object.
ACLs are inherited across the object hierarchy. So for example, if a user has read permissions on a project, they will also have read permissions on any experiment, dataset, etc. created within that project.
To restrict a grant to a particular sub-object, you may specify restrict_object_type in the ACL.
Properties
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| id | string(uuid) | true | none | Unique identifier for the acl |
| object_type | string | true | none | The object type that the ACL applies to |
| object_id | string(uuid) | true | none | The id of the object the ACL applies to |
| restrict_object_type | string¦null | false | none | Optionally restricts the permission grant to just the specified object type |
| _object_org_id | string(uuid) | true | none | The organization the ACL's referred object belongs to |
| created | string(date-time)¦null | false | none | Date of acl creation |
| user_id | string(uuid)¦null | false | none | Id of the user the ACL applies to. Exactly one of user_id and group_id will be provided |
| group_id | string(uuid)¦null | false | none | Id of the group the ACL applies to. Exactly one of user_id and group_id will be provided |
| permission | string¦null | false | none | Permission the ACL grants. Exactly one of permission and role_id will be provided |
| role_id | string(uuid)¦null | false | none | Id of the role the ACL grants. Exactly one of permission and role_id will be provided |
Enumerated Values
| Property | Value |
|---|---|
| object_type | organization |
| object_type | project |
| object_type | experiment |
| object_type | dataset |
| object_type | prompt |
| object_type | prompt_session |
| object_type | project_score |
| object_type | project_tag |
| object_type | group |
| object_type | role |
| object_type | null |
| restrict_object_type | organization |
| restrict_object_type | project |
| restrict_object_type | experiment |
| restrict_object_type | dataset |
| restrict_object_type | prompt |
| restrict_object_type | prompt_session |
| restrict_object_type | project_score |
| restrict_object_type | project_tag |
| restrict_object_type | group |
| restrict_object_type | role |
| restrict_object_type | null |
| permission | create |
| permission | read |
| permission | update |
| permission | delete |
| permission | create_acls |
| permission | read_acls |
| permission | update_acls |
| permission | delete_acls |
| permission | null |
CreateAcl
{
"object_type": "organization",
"object_id": "463a83d0-a816-4902-abba-2486e0c0a0bb",
"restrict_object_type": "organization",
"user_id": "a169451c-8525-4352-b8ca-070dd449a1a5",
"group_id": "306db4e0-7449-4501-b76f-075576fe2d8f",
"permission": "create",
"role_id": "ac4e70c8-d5be-48af-93eb-760f58fc91a9"
}An ACL grants a certain permission or role to a certain user or group on an object.
ACLs are inherited across the object hierarchy. So for example, if a user has read permissions on a project, they will also have read permissions on any experiment, dataset, etc. created within that project.
To restrict a grant to a particular sub-object, you may specify restrict_object_type in the ACL.
Properties
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| object_type | string | true | none | The object type that the ACL applies to |
| object_id | string(uuid) | true | none | The id of the object the ACL applies to |
| restrict_object_type | string¦null | false | none | Optionally restricts the permission grant to just the specified object type |
| user_id | string(uuid)¦null | false | none | Id of the user the ACL applies to. Exactly one of user_id and group_id will be provided |
| group_id | string(uuid)¦null | false | none | Id of the group the ACL applies to. Exactly one of user_id and group_id will be provided |
| permission | string¦null | false | none | Permission the ACL grants. Exactly one of permission and role_id will be provided |
| role_id | string(uuid)¦null | false | none | Id of the role the ACL grants. Exactly one of permission and role_id will be provided |
Enumerated Values
| Property | Value |
|---|---|
| object_type | organization |
| object_type | project |
| object_type | experiment |
| object_type | dataset |
| object_type | prompt |
| object_type | prompt_session |
| object_type | project_score |
| object_type | project_tag |
| object_type | group |
| object_type | role |
| object_type | null |
| restrict_object_type | organization |
| restrict_object_type | project |
| restrict_object_type | experiment |
| restrict_object_type | dataset |
| restrict_object_type | prompt |
| restrict_object_type | prompt_session |
| restrict_object_type | project_score |
| restrict_object_type | project_tag |
| restrict_object_type | group |
| restrict_object_type | role |
| restrict_object_type | null |
| permission | create |
| permission | read |
| permission | update |
| permission | delete |
| permission | create_acls |
| permission | read_acls |
| permission | update_acls |
| permission | delete_acls |
| permission | null |
User
{
"id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
"given_name": "string",
"family_name": "string",
"email": "string",
"avatar_url": "string",
"created": "2019-08-24T14:15:22Z"
}Properties
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| id | string(uuid) | true | none | Unique identifier for the user |
| given_name | string¦null | false | none | Given name of the user |
| family_name | string¦null | false | none | Family name of the user |
| string¦null | false | none | The user's email | |
| avatar_url | string¦null | false | none | URL of the user's Avatar image |
| created | string(date-time)¦null | false | none | Date of user creation |
CrossObjectInsertResponse
{
"experiment": {
"property1": {
"row_ids": ["string"]
},
"property2": {
"row_ids": ["string"]
}
},
"dataset": {
"property1": {
"row_ids": ["string"]
},
"property2": {
"row_ids": ["string"]
}
},
"project_logs": {
"property1": {
"row_ids": ["string"]
},
"property2": {
"row_ids": ["string"]
}
}
}Properties
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| experiment | object¦null | false | none | A mapping from experiment id to row ids for inserted events |
| » additionalProperties | InsertEventsResponse | false | none | none |
| dataset | object¦null | false | none | A mapping from dataset id to row ids for inserted events |
| » additionalProperties | InsertEventsResponse | false | none | none |
| project_logs | object¦null | false | none | A mapping from project id to row ids for inserted events |
| » additionalProperties | InsertEventsResponse | false | none | none |
CrossObjectInsertRequest
{
"experiment": {
"property1": {
"events": [
{
"input": null,
"output": null,
"expected": null,
"scores": {
"property1": 1,
"property2": 1
},
"metadata": {
"property1": null,
"property2": null
},
"tags": ["string"],
"metrics": {
"start": 0,
"end": 0,
"prompt_tokens": 0,
"completion_tokens": 0,
"tokens": 0,
"property1": null,
"property2": null
},
"context": {
"caller_functionname": "string",
"caller_filename": "string",
"caller_lineno": 0,
"property1": null,
"property2": null
},
"span_attributes": {
"name": "string",
"type": "llm",
"property1": null,
"property2": null
},
"id": "string",
"dataset_record_id": "string",
"_object_delete": true,
"_is_merge": false,
"_parent_id": "string"
}
],
"feedback": [
{
"id": "string",
"scores": {
"property1": 1,
"property2": 1
},
"expected": null,
"comment": "string",
"metadata": {
"property1": null,
"property2": null
},
"source": "app"
}
]
},
"property2": {
"events": [
{
"input": null,
"output": null,
"expected": null,
"scores": {
"property1": 1,
"property2": 1
},
"metadata": {
"property1": null,
"property2": null
},
"tags": ["string"],
"metrics": {
"start": 0,
"end": 0,
"prompt_tokens": 0,
"completion_tokens": 0,
"tokens": 0,
"property1": null,
"property2": null
},
"context": {
"caller_functionname": "string",
"caller_filename": "string",
"caller_lineno": 0,
"property1": null,
"property2": null
},
"span_attributes": {
"name": "string",
"type": "llm",
"property1": null,
"property2": null
},
"id": "string",
"dataset_record_id": "string",
"_object_delete": true,
"_is_merge": false,
"_parent_id": "string"
}
],
"feedback": [
{
"id": "string",
"scores": {
"property1": 1,
"property2": 1
},
"expected": null,
"comment": "string",
"metadata": {
"property1": null,
"property2": null
},
"source": "app"
}
]
}
},
"dataset": {
"property1": {
"events": [
{
"input": null,
"expected": null,
"metadata": {
"property1": null,
"property2": null
},
"tags": ["string"],
"id": "string",
"_object_delete": true,
"_is_merge": false,
"_parent_id": "string"
}
],
"feedback": [
{
"id": "string",
"comment": "string",
"metadata": {
"property1": null,
"property2": null
},
"source": "app"
}
]
},
"property2": {
"events": [
{
"input": null,
"expected": null,
"metadata": {
"property1": null,
"property2": null
},
"tags": ["string"],
"id": "string",
"_object_delete": true,
"_is_merge": false,
"_parent_id": "string"
}
],
"feedback": [
{
"id": "string",
"comment": "string",
"metadata": {
"property1": null,
"property2": null
},
"source": "app"
}
]
}
},
"project_logs": {
"property1": {
"events": [
{
"input": null,
"output": null,
"expected": null,
"scores": {
"property1": 1,
"property2": 1
},
"metadata": {
"property1": null,
"property2": null
},
"tags": ["string"],
"metrics": {
"start": 0,
"end": 0,
"prompt_tokens": 0,
"completion_tokens": 0,
"tokens": 0,
"property1": null,
"property2": null
},
"context": {
"caller_functionname": "string",
"caller_filename": "string",
"caller_lineno": 0,
"property1": null,
"property2": null
},
"span_attributes": {
"name": "string",
"type": "llm",
"property1": null,
"property2": null
},
"id": "string",
"_object_delete": true,
"_is_merge": false,
"_parent_id": "string"
}
],
"feedback": [
{
"id": "string",
"scores": {
"property1": 1,
"property2": 1
},
"expected": null,
"comment": "string",
"metadata": {
"property1": null,
"property2": null
},
"source": "app"
}
]
},
"property2": {
"events": [
{
"input": null,
"output": null,
"expected": null,
"scores": {
"property1": 1,
"property2": 1
},
"metadata": {
"property1": null,
"property2": null
},
"tags": ["string"],
"metrics": {
"start": 0,
"end": 0,
"prompt_tokens": 0,
"completion_tokens": 0,
"tokens": 0,
"property1": null,
"property2": null
},
"context": {
"caller_functionname": "string",
"caller_filename": "string",
"caller_lineno": 0,
"property1": null,
"property2": null
},
"span_attributes": {
"name": "string",
"type": "llm",
"property1": null,
"property2": null
},
"id": "string",
"_object_delete": true,
"_is_merge": false,
"_parent_id": "string"
}
],
"feedback": [
{
"id": "string",
"scores": {
"property1": 1,
"property2": 1
},
"expected": null,
"comment": "string",
"metadata": {
"property1": null,
"property2": null
},
"source": "app"
}
]
}
}
}Properties
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
| experiment | object¦null | false | none | A mapping from experiment id to a set of log events and feedback items to insert |
| » additionalProperties | object | false | none | none |
| »» events | [InsertExperimentEvent]¦null | false | none | A list of experiment events to insert |
| »» feedback | [FeedbackExperimentItem]¦null | false | none | A list of experiment feedback items |
| dataset | object¦null | false | none | A mapping from dataset id to a set of log events and feedback items to insert |
| » additionalProperties | object | false | none | none |
| »» events | [InsertDatasetEvent]¦null | false | none | A list of dataset events to insert |
| »» feedback | [FeedbackDatasetItem]¦null | false | none | A list of dataset feedback items |
| project_logs | object¦null | false | none | A mapping from project id to a set of log events and feedback items to insert |
| » additionalProperties | object | false | none | none |
| »» events | [InsertProjectLogsEvent]¦null | false | none | A list of project logs events to insert |
| »» feedback | [FeedbackProjectLogsItem]¦null | false | none | A list of project logs feedback items |