experiments

Creates, updates, deletes, gets or lists an experiments resource.

Overview

Name	`experiments`
Type	Resource
Id	`datadog.llm_observability.experiments`

Fields

The following fields are returned by SELECT queries:

list_llmobs_experiments

Name	Datatype	Description
`id`	`string`	Unique identifier of the experiment. (example: 3fd6b5e0-8910-4b1c-a7d0-5b84de329012)
`attributes`	`object`	Attributes of an Agent Observability experiment.
`type`	`string`	Resource type of an Agent Observability experiment. (experiments) (example: experiments)

Methods

The following methods are available for this resource:

Name	Accessible by	Required Params	Optional Params	Description
`list_llmobs_experiments`	`select`		`filter[project_id]`, `filter[dataset_id]`, `filter[id]`, `filter[name]`, `filter[experiment]`, `filter[metadata]`, `filter[parent_experiment_id]`, `filter[is_deleted]`, `include[user_data]`, `include[dataset_names]`, `page[cursor]`, `page[limit]`	List all Agent Observability experiments sorted by creation date, newest first.
`create_llmobs_experiment`	`insert`	`data`		Create a new Agent Observability experiment.
`update_llmobs_experiment`	`update`	`experiment_id`, `data`		Partially update an existing Agent Observability experiment.
`aggregate_llmobs_experimentation`	`exec`	`data`		Execute an analytics aggregation over Agent Observability experimentation data.<br />Use this endpoint to compute metrics (for example average eval scores) grouped by fields such as `span_id` or `experiment_id`.<br /><br />At least one `compute` definition and one `index` must be provided.
`search_llmobs_experimentation`	`exec`	`data`		Search across Agent Observability experimentation entities — projects, datasets, dataset records, experiments, and experiment runs — using cursor-based pagination.<br /><br />The `filter.scope` field controls which entity types are returned. At least one valid scope must be provided.<br /><br />Returns `200 OK` when all results fit in a single page. Returns `206 Partial Content` with a cursor in `meta.after` when additional pages are available.
`simple_search_llmobs_experimentation`	`exec`	`data`		Search across Agent Observability experimentation entities using offset-based (page-number) pagination.<br />Use this endpoint when you need total page count or want to navigate to a specific page number.<br /><br />The `filter.scope` field controls which entity types are returned. At least one valid scope must be provided.
`delete_llmobs_experiments`	`exec`	`data`		Delete one or more Agent Observability experiments.

Parameters

Parameters can be passed in the WHERE clause of a query. Check the Methods section to see which parameters are required or optional for each operation.

Name	Datatype	Description
`experiment_id`	`string`	The ID of the Agent Observability experiment. (example: 3fd6b5e0-8910-4b1c-a7d0-5b84de329012)
`site`	`string`	The Datadog site (region) for your organization, for example datadoghq.com, us3.datadoghq.com, us5.datadoghq.com, ap1.datadoghq.com, ap2.datadoghq.com, datadoghq.eu, ddog-gov.com. Resolved from the DD_SITE environment variable when set. Optional: defaults to datadoghq.com, or the value of the DD_SITE environment variable when set; a WHERE value overrides both.
`filter[dataset_id]`	`string`	Filter experiments by dataset ID.
`filter[experiment]`	`string`	Filter by logical experiment name. This is the `name` field set when creating an experiment through `POST /experiments`. Returns all experiment runs that share the same name, enabling cross-commit and cross-branch comparisons.
`filter[id]`	`string`	Filter experiments by experiment ID. Can be specified multiple times.
`filter[is_deleted]`	`boolean`	When `true`, return only soft-deleted experiments. Defaults to `false`.
`filter[metadata]`	`string`	Filter by JSONB metadata containment. Provide a JSON object string where experiments whose metadata contains all specified key-value pairs are returned. For example: `{"commit":"abc123","branch":"main"}`.
`filter[name]`	`string`	Filter experiments by their exact run name.
`filter[parent_experiment_id]`	`string`	Filter experiments by the ID of their parent (baseline) experiment. Returns all experiments that were run against the given baseline. Can be specified multiple times.
`filter[project_id]`	`string`	Filter experiments by project ID. Required if `filter[dataset_id]` is not provided.
`include[dataset_names]`	`boolean`	When `true`, enrich each experiment with its dataset name in the `dataset_name` field.
`include[user_data]`	`boolean`	When `true`, enrich each experiment with its author's user data in the `author` field.
`page[cursor]`	`string`	Use the pagination cursor returned in `meta.after` to retrieve the next page of results.
`page[limit]`	`integer (int64)`	Maximum number of results to return per page. Values above 5000 are clamped to 5000. Defaults to 5000.

`SELECT` examples

list_llmobs_experiments

List all Agent Observability experiments sorted by creation date, newest first.

SELECT
id,
attributes,
type
FROM datadog.llm_observability.experiments
WHERE filter[project_id] = '{{ filter[project_id] }}'
AND filter[dataset_id] = '{{ filter[dataset_id] }}'
AND filter[id] = '{{ filter[id] }}'
AND filter[name] = '{{ filter[name] }}'
AND filter[experiment] = '{{ filter[experiment] }}'
AND filter[metadata] = '{{ filter[metadata] }}'
AND filter[parent_experiment_id] = '{{ filter[parent_experiment_id] }}'
AND filter[is_deleted] = '{{ filter[is_deleted] }}'
AND include[user_data] = '{{ include[user_data] }}'
AND include[dataset_names] = '{{ include[dataset_names] }}'
AND page[cursor] = '{{ page[cursor] }}'
AND page[limit] = '{{ page[limit] }}'
;

`INSERT` examples

create_llmobs_experiment
Manifest

Create a new Agent Observability experiment.

INSERT INTO datadog.llm_observability.experiments (
data
)
SELECT 
'{{ data }}' /* required */
RETURNING
data
;

# Description fields are for documentation purposes
- name: experiments
props:
  - name: data
    description: |
      Data object for creating an Agent Observability experiment.
    value:
      attributes:
        config: "{{ config }}"
        dataset_id: "{{ dataset_id }}"
        dataset_version: {{ dataset_version }}
        description: "{{ description }}"
        ensure_unique: {{ ensure_unique }}
        metadata: "{{ metadata }}"
        name: "{{ name }}"
        parent_experiment_id: "{{ parent_experiment_id }}"
        project_id: "{{ project_id }}"
        run_count: {{ run_count }}
      type: "{{ type }}"

`UPDATE` examples

update_llmobs_experiment

Partially update an existing Agent Observability experiment.

UPDATE datadog.llm_observability.experiments
SET 
data = '{{ data }}'
WHERE 
experiment_id = '{{ experiment_id }}' --required
AND data = '{{ data }}' --required
RETURNING
data;

Lifecycle Methods

EXEC variables use wire (API) names.

aggregate_llmobs_experimentation
search_llmobs_experimentation
simple_search_llmobs_experimentation
delete_llmobs_experiments

Execute an analytics aggregation over Agent Observability experimentation data. Use this endpoint to compute metrics (for example average eval scores) grouped by fields such as span_id or experiment_id. At least one compute definition and one index must be provided.

EXEC datadog.llm_observability.experiments.aggregate_llmobs_experimentation 
@@json=
'{
"data": "{{ data }}"
}'
;

Search across Agent Observability experimentation entities — projects, datasets, dataset records, experiments, and experiment runs — using cursor-based pagination. The filter.scope field controls which entity types are returned. At least one valid scope must be provided. Returns 200 OK when all results fit in a single page. Returns 206 Partial Content with a cursor in meta.after when additional pages are available.

EXEC datadog.llm_observability.experiments.search_llmobs_experimentation 
@@json=
'{
"data": "{{ data }}"
}'
;

Search across Agent Observability experimentation entities using offset-based (page-number) pagination. Use this endpoint when you need total page count or want to navigate to a specific page number. The filter.scope field controls which entity types are returned. At least one valid scope must be provided.

EXEC datadog.llm_observability.experiments.simple_search_llmobs_experimentation 
@@json=
'{
"data": "{{ data }}"
}'
;

Delete one or more Agent Observability experiments.

EXEC datadog.llm_observability.experiments.delete_llmobs_experiments 
@@json=
'{
"data": "{{ data }}"
}'
;

Overview​

Fields​

Methods​

Parameters​

SELECT examples​

INSERT examples​

UPDATE examples​

Lifecycle Methods​