integration_inferences

Creates, updates, deletes, gets or lists an integration_inferences resource.

Overview

Name	integration_inferences
Type	Resource
Id	datadog.llm_observability.integration_inferences

Fields

The following fields are returned by SELECT queries:

SELECT not supported for this resource, use SHOW METHODS to view available operations for the resource.

Methods

The following methods are available for this resource:

Name	Accessible by	Required Params	Optional Params	Description
create_llmobs_integration_inference	insert	integration, account_id, model_id, messages		Run an LLM inference request through the specified integration and account, returning the model response and token usage.

Parameters

Parameters can be passed in the WHERE clause of a query. Check the Methods section to see which parameters are required or optional for each operation.

Name	Datatype	Description
account_id	string	The ID of the integration account. (example: account-abc123)
integration	string	The name of the LLM integration.
site	string	The Datadog site (region) for your organization, for example datadoghq.com, us3.datadoghq.com, us5.datadoghq.com, ap1.datadoghq.com, ap2.datadoghq.com, datadoghq.eu, ddog-gov.com. Resolved from the DD_SITE environment variable when set. Optional: defaults to datadoghq.com, or the value of the DD_SITE environment variable when set; a WHERE value overrides both.

INSERT examples

create_llmobs_integration_inference

Run an LLM inference request through the specified integration and account, returning the model response and token usage.

INSERT INTO datadog.llm_observability.integration_inferences (
anthropic_metadata,
azure_openai_metadata,
bedrock_metadata,
frequency_penalty,
json_schema,
max_completion_tokens,
max_tokens,
messages,
model_id,
openai_metadata,
presence_penalty,
temperature,
tools,
top_k,
top_p,
vertex_ai_metadata,
integration,
account_id
)
SELECT 
'{{ anthropic_metadata }}',
'{{ azure_openai_metadata }}',
'{{ bedrock_metadata }}',
{{ frequency_penalty }},
'{{ json_schema }}',
{{ max_completion_tokens }},
{{ max_tokens }},
'{{ messages }}' /* required */,
'{{ model_id }}' /* required */,
'{{ openai_metadata }}',
{{ presence_penalty }},
{{ temperature }},
'{{ tools }}',
{{ top_k }},
{{ top_p }},
'{{ vertex_ai_metadata }}',
'{{ integration }}',
'{{ account_id }}'
RETURNING
model_id,
anthropic_metadata,
azure_openai_metadata,
bedrock_metadata,
error_response,
frequency_penalty,
json_schema,
max_completion_tokens,
max_tokens,
messages,
openai_metadata,
presence_penalty,
response,
temperature,
tools,
top_k,
top_p,
vertex_ai_metadata
;

Manifest

# Description fields are for documentation purposes
- name: integration_inferences
props:
  - name: integration
    value: "{{ integration }}"
    description: Required parameter for the integration_inferences resource.
  - name: account_id
    value: "{{ account_id }}"
    description: Required parameter for the integration_inferences resource.
  - name: anthropic_metadata
    description: |
      Anthropic-specific metadata for an inference request.
    value:
      effort: "{{ effort }}"
      thinking:
        budget_tokens: {{ budget_tokens }}
        type: "{{ type }}"
  - name: azure_openai_metadata
    description: |
      Azure OpenAI-specific metadata for an integration account or inference request.
    value:
      deployment_id: "{{ deployment_id }}"
      model_version: "{{ model_version }}"
      resource_name: "{{ resource_name }}"
  - name: bedrock_metadata
    description: |
      Amazon Bedrock-specific metadata for an inference request.
    value:
      region: "{{ region }}"
  - name: frequency_penalty
    value: {{ frequency_penalty }}
    description: |
      Penalty for token frequency to reduce repetition.
  - name: json_schema
    value: "{{ json_schema }}"
    description: |
      JSON schema for structured output, if supported by the model.
  - name: max_completion_tokens
    value: {{ max_completion_tokens }}
    description: |
      Maximum number of completion tokens to generate (alternative to max_tokens for some providers).
  - name: max_tokens
    value: {{ max_tokens }}
    description: |
      Maximum number of tokens to generate.
  - name: messages
    description: |
      List of messages in an inference conversation.
    value:
      - content: "{{ content }}"
        contents: "{{ contents }}"
        id: "{{ id }}"
        role: "{{ role }}"
        tool_calls: "{{ tool_calls }}"
        tool_results: "{{ tool_results }}"
  - name: model_id
    value: "{{ model_id }}"
    description: |
      The model identifier to use for inference.
  - name: openai_metadata
    description: |
      OpenAI-specific metadata for an inference request.
    value:
      reasoning_effort: "{{ reasoning_effort }}"
      reasoning_summary: "{{ reasoning_summary }}"
  - name: presence_penalty
    value: {{ presence_penalty }}
    description: |
      Penalty for token presence to encourage topic diversity.
  - name: temperature
    value: {{ temperature }}
    description: |
      Sampling temperature between 0 and 2. Higher values produce more random output.
  - name: tools
    description: |
      List of tools available to the model.
    value:
      - function:
          description: "{{ description }}"
          name: "{{ name }}"
          parameters: "{{ parameters }}"
        type: "{{ type }}"
  - name: top_k
    value: {{ top_k }}
    description: |
      Top-K sampling parameter.
  - name: top_p
    value: {{ top_p }}
    description: |
      Nucleus sampling probability mass.
  - name: vertex_ai_metadata
    description: |
      Vertex AI-specific metadata for an integration account or inference request.
    value:
      location: "{{ location }}"
      project: "{{ project }}"
      project_ids:
        - "{{ project_ids }}"