Skip to content
AWS for Query.Farm

aws.lexv2_models.list_test_execution_result_items

Example SQL Queries

SELECT * FROM
aws.lexv2_models.list_test_execution_result_items
WHERE
"test_execution_id" = 'VALUE'
AND "result_filter_by" = 'VALUE';

Description

Gets a list of test execution result items.

Table Definition

Column NameColumn Data Type
result_filter_by Required Input Column

The filter for the list of results from the test set execution.

STRUCT(
    "result_type_filter" VARCHAR,
    "conversation_level_test_results_filter_by" STRUCT(
        "end_to_end_result" VARCHAR
    )
)
Show child fields
result_filter_by.conversation_level_test_results_filter_by

Contains information about the method for filtering Conversation level test results.

Show child fields
result_filter_by.conversation_level_test_results_filter_by.end_to_end_result

The selection of matched or mismatched end-to-end status to filter test set results data at the conversation level.

result_filter_by.result_type_filter

Specifies which results to filter. See Test result details">Test results details for details about different types of results.

test_execution_id Required Input Column

The unique identifier of the test execution to list the result items.

VARCHAR
max_results Input Column

The maximum number of test execution result items to return in each page. If there are fewer results than the max page size, only the actual number of results are returned.

BIGINT
next_token Input Column

A token that indicates whether there are more results to return in a response to the ListTestExecutionResultItems operation. If the nextToken field is present, you send the contents as the nextToken parameter of a ListTestExecutionResultItems operation request to get the next page of results.

VARCHAR
_aws_profile Input Column

The AWS profile defines the AWS identity used. It can be defined via credentials or by assuming a IAM role.

STRUCT(
    "type" VARCHAR,
    "name" VARCHAR,
    "account_id" VARCHAR,
    "via_profile_name" VARCHAR,
    "assumed_role_arn" VARCHAR,
    "organization" STRUCT(
        "account_name" VARCHAR,
        "id" VARCHAR,
        "tags" STRUCT(
            "key" VARCHAR,
            "value" VARCHAR
        )[],
        "master_account" STRUCT(
            "id" VARCHAR,
            "email" VARCHAR
        ),
        "parents" STRUCT(
            "type" VARCHAR,
            "id" VARCHAR,
            "name" VARCHAR,
            "tags" STRUCT(
                "key" VARCHAR,
                "value" VARCHAR
            )[]
        )[]
    )
)
Show child fields
_aws_profile.account_id

The AWS account id

_aws_profile.assumed_role_arn

The ARN of the assumed role

_aws_profile.name

The unique name of the profile.

_aws_profile.organization

Information about this profile's membership in the AWS organization.

Show child fields
_aws_profile.organization.account_name

The name of account speciifed by the organization

_aws_profile.organization.id

The organization id

_aws_profile.organization.master_account
Show child fields
_aws_profile.organization.master_account.email

The organization master account email address

_aws_profile.organization.master_account.id

The organization master account id

_aws_profile.organization.parents[]
Show child fields
_aws_profile.organization.parents[].id

The id of the parent

_aws_profile.organization.parents[].name

The name of the parent

_aws_profile.organization.parents[].tags[]
Show child fields
_aws_profile.organization.parents[].tags[].key
_aws_profile.organization.parents[].tags[].value
_aws_profile.organization.parents[].type

The type of parent can be an organization unit or a root

_aws_profile.organization.tags[]
Show child fields
_aws_profile.organization.tags[].key
_aws_profile.organization.tags[].value
_aws_profile.type

The type of profile, either 'credentials' or 'assumed_role'

_aws_profile.via_profile_name

This IAM role for this profile is assumed by first utilizing another profile with this name to obtain credentials.

test_execution_results

The list of results from the test execution.

STRUCT(
    "overall_test_results" STRUCT(
        "items" STRUCT(
            "multi_turn_conversation" BOOLEAN,
            "total_result_count" BIGINT,
            "speech_transcription_result_counts" MAP(VARCHAR, BIGINT),
            "end_to_end_result_counts" MAP(VARCHAR, BIGINT)
        )[]
    ),
    "conversation_level_test_results" STRUCT(
        "items" STRUCT(
            "conversation_id" VARCHAR,
            "end_to_end_result" VARCHAR,
            "speech_transcription_result" VARCHAR,
            "intent_classification_results" STRUCT(
                "intent_name" VARCHAR,
                "match_result" VARCHAR
            )[],
            "slot_resolution_results" STRUCT(
                "intent_name" VARCHAR,
                "slot_name" VARCHAR,
                "match_result" VARCHAR
            )[]
        )[]
    ),
    "intent_classification_test_results" STRUCT(
        "items" STRUCT(
            "intent_name" VARCHAR,
            "multi_turn_conversation" BOOLEAN,
            "result_counts" STRUCT(
                "total_result_count" BIGINT,
                "speech_transcription_result_counts" MAP(VARCHAR, BIGINT),
                "intent_match_result_counts" MAP(VARCHAR, BIGINT)
            )
        )[]
    ),
    "intent_level_slot_resolution_test_results" STRUCT(
        "items" STRUCT(
            "intent_name" VARCHAR,
            "multi_turn_conversation" BOOLEAN,
            "slot_resolution_results" STRUCT(
                "slot_name" VARCHAR,
                "result_counts" STRUCT(
                    "total_result_count" BIGINT,
                    "speech_transcription_result_counts" MAP(VARCHAR, BIGINT),
                    "slot_match_result_counts" MAP(VARCHAR, BIGINT)
                )
            )[]
        )[]
    ),
    "utterance_level_test_results" STRUCT(
        "items" STRUCT(
            "record_number" BIGINT,
            "conversation_id" VARCHAR,
            "turn_result" STRUCT(
                "agent" STRUCT(
                    "expected_agent_prompt" VARCHAR,
                    "actual_agent_prompt" VARCHAR,
                    "error_details" STRUCT(
                        "error_code" VARCHAR,
                        "error_message" VARCHAR
                    ),
                    "actual_elicited_slot" VARCHAR,
                    "actual_intent" VARCHAR
                ),
                "user" STRUCT(
                    "input" STRUCT(
                        "utterance_input" STRUCT(
                            "text_input" VARCHAR,
                            "audio_input" STRUCT(
                                "audio_file_s3_location" VARCHAR
                            )
                        ),
                        "request_attributes" MAP(VARCHAR, VARCHAR),
                        "session_state" STRUCT(
                            "session_attributes" MAP(VARCHAR, VARCHAR),
                            "active_contexts" STRUCT(
                                "name" VARCHAR
                            )[],
                            "runtime_hints" STRUCT(
                                "slot_hints" MAP(VARCHAR, MAP(VARCHAR, STRUCT(
                                            "runtime_hint_values" STRUCT(
                                                "phrase" VARCHAR
                                            )[]
                                        )))
                            )
                        )
                    ),
                    "expected_output" STRUCT(
                        "intent" STRUCT(
                            "name" VARCHAR,
                            "slots" MAP(VARCHAR, STRUCT(
                                    "value" VARCHAR
                                ))
                        ),
                        "active_contexts" STRUCT(
                            "name" VARCHAR
                        )[],
                        "transcript" VARCHAR
                    ),
                    "actual_output" STRUCT(
                        "intent" STRUCT(
                            "name" VARCHAR,
                            "slots" MAP(VARCHAR, STRUCT(
                                    "value" VARCHAR
                                ))
                        ),
                        "active_contexts" STRUCT(
                            "name" VARCHAR
                        )[],
                        "transcript" VARCHAR
                    ),
                    "error_details" STRUCT(
                        "error_code" VARCHAR,
                        "error_message" VARCHAR
                    ),
                    "end_to_end_result" VARCHAR,
                    "intent_match_result" VARCHAR,
                    "slot_match_result" VARCHAR,
                    "speech_transcription_result" VARCHAR,
                    "conversation_level_result" STRUCT(
                        "end_to_end_result" VARCHAR,
                        "speech_transcription_result" VARCHAR
                    )
                )
            )
        )[]
    )
)
Show child fields
test_execution_results.conversation_level_test_results

Results related to conversations in the test set, including metrics about success and failure of conversations and intent and slot failures.

Show child fields
test_execution_results.conversation_level_test_results.items[]
Show child fields
test_execution_results.conversation_level_test_results.items[].conversation_id

The conversation Id of the test result evaluation item.

test_execution_results.conversation_level_test_results.items[].end_to_end_result

The end-to-end success or failure of the test result evaluation item.

test_execution_results.conversation_level_test_results.items[].intent_classification_results[]
Show child fields
test_execution_results.conversation_level_test_results.items[].intent_classification_results[].intent_name

The intent name used in the evaluation of intent level success or failure.

test_execution_results.conversation_level_test_results.items[].intent_classification_results[].match_result

The number of times the specific intent is used in the evaluation of intent level success or failure.

test_execution_results.conversation_level_test_results.items[].slot_resolution_results[]
Show child fields
test_execution_results.conversation_level_test_results.items[].slot_resolution_results[].intent_name

The intents used in the slots list for the slot resolution details.

test_execution_results.conversation_level_test_results.items[].slot_resolution_results[].match_result

The number of matching slots used in the slots listings for the slot resolution evaluation.

test_execution_results.conversation_level_test_results.items[].slot_resolution_results[].slot_name

The slot name in the slots list for the slot resolution details.

test_execution_results.conversation_level_test_results.items[].speech_transcription_result

The speech transcription success or failure of the test result evaluation item.

test_execution_results.intent_classification_test_results

Intent recognition results aggregated by intent name. The aggregated results contain success and failure rates of intent recognition, speech transcriptions, and end-to-end conversations.

Show child fields
test_execution_results.intent_classification_test_results.items[]
Show child fields
test_execution_results.intent_classification_test_results.items[].intent_name

The name of the intent.

test_execution_results.intent_classification_test_results.items[].multi_turn_conversation

Indicates whether the conversation involves multiple turns or not.

test_execution_results.intent_classification_test_results.items[].result_counts

The result of the intent classification test.

Show child fields
test_execution_results.intent_classification_test_results.items[].result_counts.intent_match_result_counts

The number of matched and mismatched results for intent recognition for the intent.

test_execution_results.intent_classification_test_results.items[].result_counts.speech_transcription_result_counts

The number of matched, mismatched, and execution error results for speech transcription for the intent.

test_execution_results.intent_classification_test_results.items[].result_counts.total_result_count

The total number of results in the intent classification test.

test_execution_results.intent_level_slot_resolution_test_results

Slot resolution results aggregated by intent and slot name. The aggregated results contain success and failure rates of slot resolution, speech transcriptions, and end-to-end conversations

Show child fields
test_execution_results.intent_level_slot_resolution_test_results.items[]
Show child fields
test_execution_results.intent_level_slot_resolution_test_results.items[].intent_name

The name of the intent that was recognized.

test_execution_results.intent_level_slot_resolution_test_results.items[].multi_turn_conversation

Indicates whether the conversation involves multiple turns or not.

test_execution_results.intent_level_slot_resolution_test_results.items[].slot_resolution_results[]
Show child fields
test_execution_results.intent_level_slot_resolution_test_results.items[].slot_resolution_results[].result_counts

A result for slot resolution in the results of a test execution.

Show child fields
test_execution_results.intent_level_slot_resolution_test_results.items[].slot_resolution_results[].result_counts.slot_match_result_counts

The number of matched and mismatched results for slot resolution for the slot.

test_execution_results.intent_level_slot_resolution_test_results.items[].slot_resolution_results[].result_counts.speech_transcription_result_counts

The number of matched, mismatched and execution error results for speech transcription for the slot.

test_execution_results.intent_level_slot_resolution_test_results.items[].slot_resolution_results[].result_counts.total_result_count

The total number of results.

test_execution_results.intent_level_slot_resolution_test_results.items[].slot_resolution_results[].slot_name

The name of the slot.

test_execution_results.overall_test_results

Overall results for the test execution, including the breakdown of conversations and single-input utterances.

Show child fields
test_execution_results.overall_test_results.items[]
Show child fields
test_execution_results.overall_test_results.items[].end_to_end_result_counts

The number of results that succeeded.

test_execution_results.overall_test_results.items[].multi_turn_conversation

Indicates whether the conversation contains multiple turns or not.

test_execution_results.overall_test_results.items[].speech_transcription_result_counts

The number of speech transcription results in the overall test.

test_execution_results.overall_test_results.items[].total_result_count

The total number of overall results in the result of the test execution.

test_execution_results.utterance_level_test_results

Results related to utterances in the test set.

Show child fields
test_execution_results.utterance_level_test_results.items[]
Show child fields
test_execution_results.utterance_level_test_results.items[].conversation_id

The unique identifier for the conversation associated with the result.

test_execution_results.utterance_level_test_results.items[].record_number

The record number of the result.

test_execution_results.utterance_level_test_results.items[].turn_result

Contains information about the turn associated with the result.

Show child fields
test_execution_results.utterance_level_test_results.items[].turn_result.agent

Contains information about the agent messages in the turn.

Show child fields
test_execution_results.utterance_level_test_results.items[].turn_result.agent.actual_agent_prompt

The actual agent prompt for the agent turn in a test set execution.

test_execution_results.utterance_level_test_results.items[].turn_result.agent.actual_elicited_slot

The actual elicited slot for the agent turn in a test set execution.

test_execution_results.utterance_level_test_results.items[].turn_result.agent.actual_intent

The actual intent for the agent turn in a test set execution.

test_execution_results.utterance_level_test_results.items[].turn_result.agent.error_details

Details about an error in an execution of a test set.

Show child fields
test_execution_results.utterance_level_test_results.items[].turn_result.agent.error_details.error_code

The error code for the error.

test_execution_results.utterance_level_test_results.items[].turn_result.agent.error_details.error_message

The message describing the error.

test_execution_results.utterance_level_test_results.items[].turn_result.agent.expected_agent_prompt

The expected agent prompt for the agent turn in a test set execution.

test_execution_results.utterance_level_test_results.items[].turn_result.user

Contains information about the user messages in the turn.

Show child fields
test_execution_results.utterance_level_test_results.items[].turn_result.user.actual_output

Contains information about the actual output for the user turn.

Show child fields
test_execution_results.utterance_level_test_results.items[].turn_result.user.actual_output.active_contexts[]
Show child fields
test_execution_results.utterance_level_test_results.items[].turn_result.user.actual_output.active_contexts[].name

The name of active context.

test_execution_results.utterance_level_test_results.items[].turn_result.user.actual_output.intent

Contains information about the intent.

Show child fields
test_execution_results.utterance_level_test_results.items[].turn_result.user.actual_output.intent.name

The name of the intent.

test_execution_results.utterance_level_test_results.items[].turn_result.user.actual_output.intent.slots

The slots associated with the intent.

test_execution_results.utterance_level_test_results.items[].turn_result.user.actual_output.transcript

The transcript that is output for the user turn by the test execution.

test_execution_results.utterance_level_test_results.items[].turn_result.user.conversation_level_result

Contains information about the results related to the conversation associated with the user turn.

Show child fields
test_execution_results.utterance_level_test_results.items[].turn_result.user.conversation_level_result.end_to_end_result

The success or failure of the streaming of the conversation.

test_execution_results.utterance_level_test_results.items[].turn_result.user.conversation_level_result.speech_transcription_result

The speech transcription success or failure details of the conversation.

test_execution_results.utterance_level_test_results.items[].turn_result.user.end_to_end_result

Specifies whether the expected and actual outputs match or not, or if there is an error in execution.

test_execution_results.utterance_level_test_results.items[].turn_result.user.error_details

Details about an error in an execution of a test set.

Show child fields
test_execution_results.utterance_level_test_results.items[].turn_result.user.error_details.error_code

The error code for the error.

test_execution_results.utterance_level_test_results.items[].turn_result.user.error_details.error_message

The message describing the error.

test_execution_results.utterance_level_test_results.items[].turn_result.user.expected_output

Contains information about the expected output for the user turn.

Show child fields
test_execution_results.utterance_level_test_results.items[].turn_result.user.expected_output.active_contexts[]
Show child fields
test_execution_results.utterance_level_test_results.items[].turn_result.user.expected_output.active_contexts[].name

The name of active context.

test_execution_results.utterance_level_test_results.items[].turn_result.user.expected_output.intent

Contains information about the intent.

Show child fields
test_execution_results.utterance_level_test_results.items[].turn_result.user.expected_output.intent.name

The name of the intent.

test_execution_results.utterance_level_test_results.items[].turn_result.user.expected_output.intent.slots

The slots associated with the intent.

test_execution_results.utterance_level_test_results.items[].turn_result.user.expected_output.transcript

The transcript that is output for the user turn by the test execution.

test_execution_results.utterance_level_test_results.items[].turn_result.user.input

Contains information about the user messages in the turn in the input.

Show child fields
test_execution_results.utterance_level_test_results.items[].turn_result.user.input.request_attributes

Request attributes of the user turn.

test_execution_results.utterance_level_test_results.items[].turn_result.user.input.session_state

Contains information about the session state in the input.

Show child fields
test_execution_results.utterance_level_test_results.items[].turn_result.user.input.session_state.active_contexts[]
Show child fields
test_execution_results.utterance_level_test_results.items[].turn_result.user.input.session_state.active_contexts[].name

The name of active context.

test_execution_results.utterance_level_test_results.items[].turn_result.user.input.session_state.runtime_hints

Runtime hints for the session state.

Show child fields
test_execution_results.utterance_level_test_results.items[].turn_result.user.input.session_state.runtime_hints.slot_hints

A list of the slots in the intent that should have runtime hints added, and the phrases that should be added for each slot.

The first level of the slotHints map is the name of the intent. The second level is the name of the slot within the intent. For more information, see Using hints to improve accuracy.

The intent name and slot name must exist.

test_execution_results.utterance_level_test_results.items[].turn_result.user.input.session_state.session_attributes

Session attributes for the session state.

test_execution_results.utterance_level_test_results.items[].turn_result.user.input.utterance_input

The utterance input in the user turn.

Show child fields
test_execution_results.utterance_level_test_results.items[].turn_result.user.input.utterance_input.audio_input

Contains information about the audio input for an utterance.

Show child fields
test_execution_results.utterance_level_test_results.items[].turn_result.user.input.utterance_input.audio_input.audio_file_s3_location

Amazon S3 file pointing to the audio.

test_execution_results.utterance_level_test_results.items[].turn_result.user.input.utterance_input.text_input

A text input transcription of the utterance. It is only applicable for test-sets containing text data.

test_execution_results.utterance_level_test_results.items[].turn_result.user.intent_match_result

Specifies whether the expected and actual intents match or not.

test_execution_results.utterance_level_test_results.items[].turn_result.user.slot_match_result

Specifies whether the expected and actual slots match or not.

test_execution_results.utterance_level_test_results.items[].turn_result.user.speech_transcription_result

Specifies whether the expected and actual speech transcriptions match or not, or if there is an error in execution.