| Column Name | Column Data Type |
|---|
job_definition_name Required Input Column
The name of the data quality monitoring job definition. | VARCHAR |
_aws_profile Input Column
The AWS profile defines the AWS identity used. It can be defined via credentials or by assuming a IAM role. | STRUCT(
"type" VARCHAR,
"name" VARCHAR,
"account_id" VARCHAR,
"via_profile_name" VARCHAR,
"assumed_role_arn" VARCHAR,
"organization" STRUCT(
"account_name" VARCHAR,
"id" VARCHAR,
"tags" STRUCT(
"key" VARCHAR,
"value" VARCHAR
)[],
"master_account" STRUCT(
"id" VARCHAR,
"email" VARCHAR
),
"parents" STRUCT(
"type" VARCHAR,
"id" VARCHAR,
"name" VARCHAR,
"tags" STRUCT(
"key" VARCHAR,
"value" VARCHAR
)[]
)[]
)
) |
Show child fields- _aws_profile.account_id
The AWS account id - _aws_profile.assumed_role_arn
The ARN of the assumed role - _aws_profile.name
The unique name of the profile. - _aws_profile.organization
Information about this profile's membership in the AWS organization. Show child fields- _aws_profile.organization.account_name
The name of account speciifed by the organization - _aws_profile.organization.id
The organization id - _aws_profile.organization.master_account
Show child fields- _aws_profile.organization.master_account.email
The organization master account email address - _aws_profile.organization.master_account.id
The organization master account id
- _aws_profile.organization.parents[]
Show child fields- _aws_profile.organization.parents[].id
The id of the parent - _aws_profile.organization.parents[].name
The name of the parent - _aws_profile.organization.parents[].tags[]
Show child fields- _aws_profile.organization.parents[].tags[].key
- _aws_profile.organization.parents[].tags[].value
- _aws_profile.organization.parents[].type
The type of parent can be an organization unit or a root
- _aws_profile.organization.tags[]
Show child fields- _aws_profile.organization.tags[].key
- _aws_profile.organization.tags[].value
- _aws_profile.type
The type of profile, either 'credentials' or 'assumed_role' - _aws_profile.via_profile_name
This IAM role for this profile is assumed by first utilizing another profile with this name to obtain credentials.
|
creation_time
The time that the data quality monitoring job definition was created. | TIMESTAMP_S |
data_quality_app_specification
Information about the container that runs the data quality monitoring job. | STRUCT(
"image_uri" VARCHAR,
"container_entrypoint" VARCHAR[],
"container_arguments" VARCHAR[],
"record_preprocessor_source_uri" VARCHAR,
"post_analytics_processor_source_uri" VARCHAR,
"environment" MAP(VARCHAR, VARCHAR)
) |
Show child fields- data_quality_app_specification.container_arguments[]
- data_quality_app_specification.container_entrypoint[]
- data_quality_app_specification.environment
Sets the environment variables in the container that the monitoring job runs. - data_quality_app_specification.image_uri
The container image that the data quality monitoring job runs. - data_quality_app_specification.post_analytics_processor_source_uri
An Amazon S3 URI to a script that is called after analysis has been performed. Applicable only for the built-in (first party) containers. - data_quality_app_specification.record_preprocessor_source_uri
An Amazon S3 URI to a script that is called per row prior to running analysis. It can base64 decode the payload and convert it into a flattened JSON so that the built-in container can use the converted data. Applicable only for the built-in (first party) containers.
|
data_quality_baseline_config
The constraints and baselines for the data quality monitoring job definition. | STRUCT(
"baselining_job_name" VARCHAR,
"constraints_resource" STRUCT(
"s3_uri" VARCHAR
),
"statistics_resource" STRUCT(
"s3_uri" VARCHAR
)
) |
Show child fields- data_quality_baseline_config.baselining_job_name
The name of the job that performs baselining for the data quality monitoring job. - data_quality_baseline_config.constraints_resource
The constraints resource for a monitoring job. Show child fields- data_quality_baseline_config.constraints_resource.s3_uri
The Amazon S3 URI for the constraints resource.
- data_quality_baseline_config.statistics_resource
The statistics resource for a monitoring job. Show child fields- data_quality_baseline_config.statistics_resource.s3_uri
The Amazon S3 URI for the statistics resource.
|
data_quality_job_input
The list of inputs for the data quality monitoring job. Currently endpoints are supported. | STRUCT(
"endpoint_input" STRUCT(
"endpoint_name" VARCHAR,
"local_path" VARCHAR,
"s3_input_mode" VARCHAR,
"s3_data_distribution_type" VARCHAR,
"features_attribute" VARCHAR,
"inference_attribute" VARCHAR,
"probability_attribute" VARCHAR,
"probability_threshold_attribute" DOUBLE,
"start_time_offset" VARCHAR,
"end_time_offset" VARCHAR,
"exclude_features_attribute" VARCHAR
),
"batch_transform_input" STRUCT(
"data_captured_destination_s3_uri" VARCHAR,
"dataset_format" STRUCT(
"csv" STRUCT(
"header" BOOLEAN
),
"json" STRUCT(
"line" BOOLEAN
),
"parquet" BOOLEAN
),
"local_path" VARCHAR,
"s3_input_mode" VARCHAR,
"s3_data_distribution_type" VARCHAR,
"features_attribute" VARCHAR,
"inference_attribute" VARCHAR,
"probability_attribute" VARCHAR,
"probability_threshold_attribute" DOUBLE,
"start_time_offset" VARCHAR,
"end_time_offset" VARCHAR,
"exclude_features_attribute" VARCHAR
)
) |
Show child fields- data_quality_job_input.batch_transform_input
Input object for the batch transform job. Show child fields- data_quality_job_input.batch_transform_input.data_captured_destination_s3_uri
The Amazon S3 location being used to capture the data. - data_quality_job_input.batch_transform_input.dataset_format
The dataset format for your batch transform job. Show child fields- data_quality_job_input.batch_transform_input.dataset_format.csv
The CSV dataset used in the monitoring job. Show child fields- data_quality_job_input.batch_transform_input.dataset_format.csv.header
Indicates if the CSV data has a header.
- data_quality_job_input.batch_transform_input.dataset_format.json
The JSON dataset used in the monitoring job Show child fields- data_quality_job_input.batch_transform_input.dataset_format.json.line
Indicates if the file should be read as a JSON object per line.
- data_quality_job_input.batch_transform_input.dataset_format.parquet
The Parquet dataset used in the monitoring job
- data_quality_job_input.batch_transform_input.end_time_offset
If specified, monitoring jobs subtract this time from the end time. For information about using offsets for scheduling monitoring jobs, see Schedule Model Quality Monitoring Jobs. - data_quality_job_input.batch_transform_input.exclude_features_attribute
The attributes of the input data to exclude from the analysis. - data_quality_job_input.batch_transform_input.features_attribute
The attributes of the input data that are the input features. - data_quality_job_input.batch_transform_input.inference_attribute
The attribute of the input data that represents the ground truth label. - data_quality_job_input.batch_transform_input.local_path
Path to the filesystem where the batch transform data is available to the container. - data_quality_job_input.batch_transform_input.probability_attribute
In a classification problem, the attribute that represents the class probability. - data_quality_job_input.batch_transform_input.probability_threshold_attribute
The threshold for the class probability to be evaluated as a positive result. - data_quality_job_input.batch_transform_input.s3_data_distribution_type
Whether input data distributed in Amazon S3 is fully replicated or sharded by an S3 key. Defaults to FullyReplicated - data_quality_job_input.batch_transform_input.s3_input_mode
Whether the Pipe or File is used as the input mode for transferring data for the monitoring job. Pipe mode is recommended for large datasets. File mode is useful for small files that fit in memory. Defaults to File. - data_quality_job_input.batch_transform_input.start_time_offset
If specified, monitoring jobs substract this time from the start time. For information about using offsets for scheduling monitoring jobs, see Schedule Model Quality Monitoring Jobs.
- data_quality_job_input.endpoint_input
Input object for the endpoint Show child fields- data_quality_job_input.endpoint_input.end_time_offset
If specified, monitoring jobs substract this time from the end time. For information about using offsets for scheduling monitoring jobs, see Schedule Model Quality Monitoring Jobs. - data_quality_job_input.endpoint_input.endpoint_name
An endpoint in customer's account which has enabled DataCaptureConfig enabled. - data_quality_job_input.endpoint_input.exclude_features_attribute
The attributes of the input data to exclude from the analysis. - data_quality_job_input.endpoint_input.features_attribute
The attributes of the input data that are the input features. - data_quality_job_input.endpoint_input.inference_attribute
The attribute of the input data that represents the ground truth label. - data_quality_job_input.endpoint_input.local_path
Path to the filesystem where the endpoint data is available to the container. - data_quality_job_input.endpoint_input.probability_attribute
In a classification problem, the attribute that represents the class probability. - data_quality_job_input.endpoint_input.probability_threshold_attribute
The threshold for the class probability to be evaluated as a positive result. - data_quality_job_input.endpoint_input.s3_data_distribution_type
Whether input data distributed in Amazon S3 is fully replicated or sharded by an Amazon S3 key. Defaults to FullyReplicated - data_quality_job_input.endpoint_input.s3_input_mode
Whether the Pipe or File is used as the input mode for transferring data for the monitoring job. Pipe mode is recommended for large datasets. File mode is useful for small files that fit in memory. Defaults to File. - data_quality_job_input.endpoint_input.start_time_offset
If specified, monitoring jobs substract this time from the start time. For information about using offsets for scheduling monitoring jobs, see Schedule Model Quality Monitoring Jobs.
|
data_quality_job_output_config
The output configuration for monitoring jobs. | STRUCT(
"monitoring_outputs" STRUCT(
"s3_output" STRUCT(
"s3_uri" VARCHAR,
"local_path" VARCHAR,
"s3_upload_mode" VARCHAR
)
)[],
"kms_key_id" VARCHAR
) |
Show child fields- data_quality_job_output_config.kms_key_id
The Key Management Service (KMS) key that Amazon SageMaker uses to encrypt the model artifacts at rest using Amazon S3 server-side encryption. - data_quality_job_output_config.monitoring_outputs[]
Show child fields- data_quality_job_output_config.monitoring_outputs[].s3_output
The Amazon S3 storage location where the results of a monitoring job are saved. Show child fields- data_quality_job_output_config.monitoring_outputs[].s3_output.local_path
The local path to the Amazon S3 storage location where Amazon SageMaker saves the results of a monitoring job. LocalPath is an absolute path for the output data. - data_quality_job_output_config.monitoring_outputs[].s3_output.s3_upload_mode
Whether to upload the results of the monitoring job continuously or after the job completes. - data_quality_job_output_config.monitoring_outputs[].s3_output.s3_uri
A URI that identifies the Amazon S3 storage location where Amazon SageMaker saves the results of a monitoring job.
|
job_definition_arn
The Amazon Resource Name (ARN) of the data quality monitoring job definition. | VARCHAR |
job_resources
Identifies the resources to deploy for a monitoring job. | STRUCT(
"cluster_config" STRUCT(
"instance_count" BIGINT,
"instance_type" VARCHAR,
"volume_size_in_gb" BIGINT,
"volume_kms_key_id" VARCHAR
)
) |
Show child fields- job_resources.cluster_config
The configuration for the cluster resources used to run the processing job. Show child fields- job_resources.cluster_config.instance_count
The number of ML compute instances to use in the model monitoring job. For distributed processing jobs, specify a value greater than 1. The default value is 1. - job_resources.cluster_config.instance_type
The ML compute instance type for the processing job. - job_resources.cluster_config.volume_kms_key_id
The Key Management Service (KMS) key that Amazon SageMaker uses to encrypt data on the storage volume attached to the ML compute instance(s) that run the model monitoring job. - job_resources.cluster_config.volume_size_in_gb
The size of the ML storage volume, in gigabytes, that you want to provision. You must specify sufficient ML storage for your scenario.
|
network_config
The networking configuration for the data quality monitoring job. | STRUCT(
"enable_inter_container_traffic_encryption" BOOLEAN,
"enable_network_isolation" BOOLEAN,
"vpc_config" STRUCT(
"security_group_ids" VARCHAR[],
"subnets" VARCHAR[]
)
) |
Show child fields- network_config.enable_inter_container_traffic_encryption
Whether to encrypt all communications between the instances used for the monitoring jobs. Choose True to encrypt communications. Encryption provides greater security for distributed jobs, but the processing might take longer. - network_config.enable_network_isolation
Whether to allow inbound and outbound network calls to and from the containers used for the monitoring job. - network_config.vpc_config
Specifies an Amazon Virtual Private Cloud (VPC) that your SageMaker jobs, hosted models, and compute resources have access to. You can control access to and from your resources by configuring a VPC. For more information, see Give SageMaker Access to Resources in your Amazon VPC. Show child fields- network_config.vpc_config.security_group_ids[]
- network_config.vpc_config.subnets[]
|
role_arn
The Amazon Resource Name (ARN) of an IAM role that Amazon SageMaker can assume to perform tasks on your behalf. | VARCHAR |
stopping_condition
A time limit for how long the monitoring job is allowed to run before stopping. | STRUCT(
"max_runtime_in_seconds" BIGINT
) |
Show child fields- stopping_condition.max_runtime_in_seconds
The maximum runtime allowed in seconds. The MaxRuntimeInSeconds cannot exceed the frequency of the job. For data quality and model explainability, this can be up to 3600 seconds for an hourly schedule. For model bias and model quality hourly schedules, this can be up to 1800 seconds.
|