evaluator_customs

Creates, updates, deletes, gets or lists an evaluator_customs resource.

Overview

Name	evaluator_customs
Type	Resource
Id	datadog.llm_observability.evaluator_customs

Fields

The following fields are returned by SELECT queries:

get_llmobs_custom_eval_config

Name	Datatype	Description
id	string	Unique name identifier of the evaluator configuration. (example: my-custom-evaluator)
attributes	object	Attributes of a custom Agent Observability evaluator configuration.
type	string	Type of the custom Agent Observability evaluator configuration resource. (evaluator_config) (example: evaluator_config)

list_llmobs_custom_eval_configs

Name	Datatype	Description
id	string	Unique name identifier of the evaluator configuration. (example: my-custom-evaluator)
attributes	object	Attributes of a custom Agent Observability evaluator configuration.
type	string	Type of the custom Agent Observability evaluator configuration resource. (evaluator_config) (example: evaluator_config)

Methods

The following methods are available for this resource:

Name	Accessible by	Required Params	Optional Params	Description
get_llmobs_custom_eval_config	select	eval_name		Retrieve a custom Agent Observability evaluator configuration by its name.
list_llmobs_custom_eval_configs	select			List all custom Agent Observability evaluator configurations for the organization.
update_llmobs_custom_eval_config	replace	eval_name, data		Create or update a custom Agent Observability evaluator configuration by its name.
delete_llmobs_custom_eval_config	delete	eval_name		Delete a custom Agent Observability evaluator configuration by its name.

Parameters

Parameters can be passed in the WHERE clause of a query. Check the Methods section to see which parameters are required or optional for each operation.

Name	Datatype	Description
eval_name	string	The name of the custom Agent Observability evaluator configuration. (example: my-custom-evaluator)
site	string	The Datadog site (region) for your organization, for example datadoghq.com, us3.datadoghq.com, us5.datadoghq.com, ap1.datadoghq.com, ap2.datadoghq.com, datadoghq.eu, ddog-gov.com. Resolved from the DD_SITE environment variable when set. Optional: defaults to datadoghq.com, or the value of the DD_SITE environment variable when set; a WHERE value overrides both.

SELECT examples

get_llmobs_custom_eval_config

Retrieve a custom Agent Observability evaluator configuration by its name.

SELECT
id,
attributes,
type
FROM datadog.llm_observability.evaluator_customs
WHERE eval_name = '{{ eval_name }}' -- required
;

list_llmobs_custom_eval_configs

List all custom Agent Observability evaluator configurations for the organization.

SELECT
id,
attributes,
type
FROM datadog.llm_observability.evaluator_customs
;

REPLACE examples

update_llmobs_custom_eval_config

Create or update a custom Agent Observability evaluator configuration by its name.

REPLACE datadog.llm_observability.evaluator_customs
SET 
data = '{{ data }}'
WHERE 
eval_name = '{{ eval_name }}' --required
AND data = '{{ data }}' --required;

DELETE examples

delete_llmobs_custom_eval_config

Delete a custom Agent Observability evaluator configuration by its name.

DELETE FROM datadog.llm_observability.evaluator_customs
WHERE eval_name = '{{ eval_name }}' --required
;