| Column Name | Column Data Type |
|---|
job_identifier Required Input Column
The Amazon Resource Name (ARN) of the model evaluation job. | VARCHAR |
_aws_profile Input Column
The AWS profile defines the AWS identity used. It can be defined via credentials or by assuming a IAM role. | STRUCT(
"type" VARCHAR,
"name" VARCHAR,
"account_id" VARCHAR,
"via_profile_name" VARCHAR,
"assumed_role_arn" VARCHAR,
"organization" STRUCT(
"account_name" VARCHAR,
"id" VARCHAR,
"tags" STRUCT(
"key" VARCHAR,
"value" VARCHAR
)[],
"master_account" STRUCT(
"id" VARCHAR,
"email" VARCHAR
),
"parents" STRUCT(
"type" VARCHAR,
"id" VARCHAR,
"name" VARCHAR,
"tags" STRUCT(
"key" VARCHAR,
"value" VARCHAR
)[]
)[]
)
) |
Show child fields- _aws_profile.account_id
The AWS account id - _aws_profile.assumed_role_arn
The ARN of the assumed role - _aws_profile.name
The unique name of the profile. - _aws_profile.organization
Information about this profile's membership in the AWS organization. Show child fields- _aws_profile.organization.account_name
The name of account speciifed by the organization - _aws_profile.organization.id
The organization id - _aws_profile.organization.master_account
Show child fields- _aws_profile.organization.master_account.email
The organization master account email address - _aws_profile.organization.master_account.id
The organization master account id
- _aws_profile.organization.parents[]
Show child fields- _aws_profile.organization.parents[].id
The id of the parent - _aws_profile.organization.parents[].name
The name of the parent - _aws_profile.organization.parents[].tags[]
Show child fields- _aws_profile.organization.parents[].tags[].key
- _aws_profile.organization.parents[].tags[].value
- _aws_profile.organization.parents[].type
The type of parent can be an organization unit or a root
- _aws_profile.organization.tags[]
Show child fields- _aws_profile.organization.tags[].key
- _aws_profile.organization.tags[].value
- _aws_profile.type
The type of profile, either 'credentials' or 'assumed_role' - _aws_profile.via_profile_name
This IAM role for this profile is assumed by first utilizing another profile with this name to obtain credentials.
|
_aws_region Input Column
The AWS region to use. | VARCHAR |
creation_time
When the model evaluation job was created. | TIMESTAMP_S |
customer_encryption_key_id
The Amazon Resource Name (ARN) of the customer managed key specified when the model evaluation job was created. | VARCHAR |
evaluation_config
Contains details about the type of model evaluation job, the metrics used, the task type selected, the datasets used, and any custom metrics you defined. | STRUCT(
"automated" STRUCT(
"dataset_metric_configs" STRUCT(
"task_type" VARCHAR,
"dataset" STRUCT(
"name" VARCHAR,
"dataset_location" STRUCT(
"s3_uri" VARCHAR
)
),
"metric_names" VARCHAR[]
)[]
),
"human" STRUCT(
"human_workflow_config" STRUCT(
"flow_definition_arn" VARCHAR,
"instructions" VARCHAR
),
"custom_metrics" STRUCT(
"name" VARCHAR,
"description" VARCHAR,
"rating_method" VARCHAR
)[],
"dataset_metric_configs" STRUCT(
"task_type" VARCHAR,
"dataset" STRUCT(
"name" VARCHAR,
"dataset_location" STRUCT(
"s3_uri" VARCHAR
)
),
"metric_names" VARCHAR[]
)[]
)
) |
Show child fields- evaluation_config.automated
Used to specify an automated model evaluation job. See AutomatedEvaluationConfig to view the required parameters. Show child fields- evaluation_config.automated.dataset_metric_configs[]
Show child fields- evaluation_config.automated.dataset_metric_configs[].dataset
Specifies the prompt dataset. Show child fields- evaluation_config.automated.dataset_metric_configs[].dataset.dataset_location
For custom prompt datasets, you must specify the location in Amazon S3 where the prompt dataset is saved. Show child fields- evaluation_config.automated.dataset_metric_configs[].dataset.dataset_location.s3_uri
The S3 URI of the S3 bucket specified in the job.
- evaluation_config.automated.dataset_metric_configs[].dataset.name
Used to specify supported built-in prompt datasets. Valid values are Builtin.Bold, Builtin.BoolQ, Builtin.NaturalQuestions, Builtin.Gigaword, Builtin.RealToxicityPrompts, Builtin.TriviaQA, Builtin.T-Rex, Builtin.WomensEcommerceClothingReviews and Builtin.Wikitext2.
- evaluation_config.automated.dataset_metric_configs[].metric_names[]
- evaluation_config.automated.dataset_metric_configs[].task_type
The task type you want the model to carry out.
- evaluation_config.human
Used to specify a model evaluation job that uses human workers.See HumanEvaluationConfig to view the required parameters. Show child fields- evaluation_config.human.custom_metrics[]
Show child fields- evaluation_config.human.custom_metrics[].description
An optional description of the metric. Use this parameter to provide more details about the metric. - evaluation_config.human.custom_metrics[].name
The name of the metric. Your human evaluators will see this name in the evaluation UI. - evaluation_config.human.custom_metrics[].rating_method
Choose how you want your human workers to evaluation your model. Valid values for rating methods are ThumbsUpDown, IndividualLikertScale,ComparisonLikertScale, ComparisonChoice, and ComparisonRank
- evaluation_config.human.dataset_metric_configs[]
Show child fields- evaluation_config.human.dataset_metric_configs[].dataset
Specifies the prompt dataset. Show child fields- evaluation_config.human.dataset_metric_configs[].dataset.dataset_location
For custom prompt datasets, you must specify the location in Amazon S3 where the prompt dataset is saved. Show child fields- evaluation_config.human.dataset_metric_configs[].dataset.dataset_location.s3_uri
The S3 URI of the S3 bucket specified in the job.
- evaluation_config.human.dataset_metric_configs[].dataset.name
Used to specify supported built-in prompt datasets. Valid values are Builtin.Bold, Builtin.BoolQ, Builtin.NaturalQuestions, Builtin.Gigaword, Builtin.RealToxicityPrompts, Builtin.TriviaQA, Builtin.T-Rex, Builtin.WomensEcommerceClothingReviews and Builtin.Wikitext2.
- evaluation_config.human.dataset_metric_configs[].metric_names[]
- evaluation_config.human.dataset_metric_configs[].task_type
The task type you want the model to carry out.
- evaluation_config.human.human_workflow_config
The parameters of the human workflow. Show child fields- evaluation_config.human.human_workflow_config.flow_definition_arn
The Amazon Resource Number (ARN) for the flow definition - evaluation_config.human.human_workflow_config.instructions
Instructions for the flow definition
|
failure_messages
An array of strings the specify why the model evaluation job has failed. | VARCHAR[] |
Show child fields- failure_messages[]
|
inference_config
Details about the models you specified in your model evaluation job. | STRUCT(
"models" STRUCT(
"bedrock_model" STRUCT(
"model_identifier" VARCHAR,
"inference_params" VARCHAR
)
)[]
) |
Show child fields- inference_config.models[]
Show child fields- inference_config.models[].bedrock_model
Defines the Amazon Bedrock model and inference parameters you want used. Show child fields- inference_config.models[].bedrock_model.inference_params
Each Amazon Bedrock support different inference parameters that change how the model behaves during inference. - inference_config.models[].bedrock_model.model_identifier
The ARN of the Amazon Bedrock model specified.
|
job_arn
The Amazon Resource Name (ARN) of the model evaluation job. | VARCHAR |
job_description
The description of the model evaluation job. | VARCHAR |
job_name
The name of the model evaluation job. | VARCHAR |
job_type
The type of model evaluation job. | VARCHAR |
last_modified_time
When the model evaluation job was last modified. | TIMESTAMP_S |
output_data_config
Amazon S3 location for where output data is saved. | STRUCT(
"s3_uri" VARCHAR
) |
Show child fields- output_data_config.s3_uri
The Amazon S3 URI where the results of model evaluation job are saved.
|
role_arn
The Amazon Resource Name (ARN) of the IAM service role used in the model evaluation job. | VARCHAR |
status
The status of the model evaluation job. | VARCHAR |