aws.sagemaker.describe_hyper_parameter_tuning_job

Example SQL Queries

SELECT * FROM
aws.sagemaker.describe_hyper_parameter_tuning_job
WHERE
"hyper_parameter_tuning_job_name" = 'VALUE';

Description

Returns a description of a hyperparameter tuning job, depending on the fields selected. These fields can include the name, Amazon Resource Name (ARN), job status of your tuning job and more.

Table Definition

Column Name	Column Data Type
`hyper_parameter_tuning_job_name` Required Input Column The name of the hyperparameter tuning job.	VARCHAR
`_aws_profile` Input Column The AWS profile defines the AWS identity used. It can be defined via credentials or by assuming a IAM role.	STRUCT( "type" VARCHAR, "name" VARCHAR, "account_id" VARCHAR, "via_profile_name" VARCHAR, "assumed_role_arn" VARCHAR, "organization" STRUCT( "account_name" VARCHAR, "id" VARCHAR, "tags" STRUCT( "key" VARCHAR, "value" VARCHAR )[], "master_account" STRUCT( "id" VARCHAR, "email" VARCHAR ), "parents" STRUCT( "type" VARCHAR, "id" VARCHAR, "name" VARCHAR, "tags" STRUCT( "key" VARCHAR, "value" VARCHAR )[] )[] ) )
Show child fields `_aws_profile.account_id` The AWS account id `_aws_profile.assumed_role_arn` The ARN of the assumed role `_aws_profile.name` The unique name of the profile. `_aws_profile.organization` Information about this profile's membership in the AWS organization. Show child fields `_aws_profile.organization.account_name` The name of account speciifed by the organization `_aws_profile.organization.id` The organization id `_aws_profile.organization.master_account` Show child fields `_aws_profile.organization.master_account.email` The organization master account email address `_aws_profile.organization.master_account.id` The organization master account id `_aws_profile.organization.parents[]` Show child fields `_aws_profile.organization.parents[].id` The id of the parent `_aws_profile.organization.parents[].name` The name of the parent `_aws_profile.organization.parents[].tags[]` Show child fields `_aws_profile.organization.parents[].tags[].key` `_aws_profile.organization.parents[].tags[].value` `_aws_profile.organization.parents[].type` The type of parent can be an organization unit or a root `_aws_profile.organization.tags[]` Show child fields `_aws_profile.organization.tags[].key` `_aws_profile.organization.tags[].value` `_aws_profile.type` The type of profile, either 'credentials' or 'assumed_role' `_aws_profile.via_profile_name` This IAM role for this profile is assumed by first utilizing another profile with this name to obtain credentials.
`autotune` A flag to indicate if autotune is enabled for the hyperparameter tuning job.	STRUCT( "mode" VARCHAR )
Show child fields `autotune.mode` Set `Mode` to `Enabled` if you want to use Autotune.
`best_training_job` A TrainingJobSummary object that describes the training job that completed with the best current HyperParameterTuningJobObjective.	STRUCT( "training_job_definition_name" VARCHAR, "training_job_name" VARCHAR, "training_job_arn" VARCHAR, "tuning_job_name" VARCHAR, "creation_time" TIMESTAMP_S, "training_start_time" TIMESTAMP_S, "training_end_time" TIMESTAMP_S, "training_job_status" VARCHAR, "tuned_hyper_parameters" MAP(VARCHAR, VARCHAR), "failure_reason" VARCHAR, "final_hyper_parameter_tuning_job_objective_metric" STRUCT( "type" VARCHAR, "metric_name" VARCHAR, "value" DOUBLE ), "objective_status" VARCHAR )
Show child fields `best_training_job.creation_time` The date and time that the training job was created. `best_training_job.failure_reason` The reason that the training job failed. `best_training_job.final_hyper_parameter_tuning_job_objective_metric` The FinalHyperParameterTuningJobObjectiveMetric object that specifies the value of the objective metric of the tuning job that launched this training job. Show child fields `best_training_job.final_hyper_parameter_tuning_job_objective_metric.metric_name` The name of the objective metric. For SageMaker built-in algorithms, metrics are defined per algorithm. See the metrics for XGBoost as an example. You can also use a custom algorithm for training and define your own metrics. For more information, see Define metrics and environment variables. `best_training_job.final_hyper_parameter_tuning_job_objective_metric.type` Select if you want to minimize or maximize the objective metric during hyperparameter tuning. `best_training_job.final_hyper_parameter_tuning_job_objective_metric.value` The value of the objective metric. `best_training_job.objective_status` The status of the objective metric for the training job: Succeeded: The final objective metric for the training job was evaluated by the hyperparameter tuning job and used in the hyperparameter tuning process. Pending: The training job is in progress and evaluation of its final objective metric is pending. Failed: The final objective metric for the training job was not evaluated, and was not used in the hyperparameter tuning process. This typically occurs when the training job failed or did not emit an objective metric. `best_training_job.training_end_time` Specifies the time when the training job ends on training instances. You are billed for the time interval between the value of `TrainingStartTime` and this time. For successful jobs and stopped jobs, this is the time after model artifacts are uploaded. For failed jobs, this is the time when SageMaker detects a job failure. `best_training_job.training_job_arn` The Amazon Resource Name (ARN) of the training job. `best_training_job.training_job_definition_name` The training job definition name. `best_training_job.training_job_name` The name of the training job. `best_training_job.training_job_status` The status of the training job. `best_training_job.training_start_time` The date and time that the training job started. `best_training_job.tuned_hyper_parameters` A list of the hyperparameters for which you specified ranges to search. `best_training_job.tuning_job_name` The HyperParameter tuning job that launched the training job.
`consumed_resources` The total resources consumed by your hyperparameter tuning job.	STRUCT( "runtime_in_seconds" BIGINT )
Show child fields `consumed_resources.runtime_in_seconds` The wall clock runtime in seconds used by your hyperparameter tuning job.
`creation_time` The date and time that the tuning job started.	TIMESTAMP_S
`failure_reason` If the tuning job failed, the reason it failed.	VARCHAR
`hyper_parameter_tuning_end_time` The date and time that the tuning job ended.	TIMESTAMP_S
`hyper_parameter_tuning_job_arn` The Amazon Resource Name (ARN) of the tuning job.	VARCHAR
`hyper_parameter_tuning_job_config` The HyperParameterTuningJobConfig object that specifies the configuration of the tuning job.	STRUCT( "strategy" VARCHAR, "strategy_config" STRUCT( "hyperband_strategy_config" STRUCT( "min_resource" BIGINT, "max_resource" BIGINT ) ), "hyper_parameter_tuning_job_objective" STRUCT( "type" VARCHAR, "metric_name" VARCHAR ), "resource_limits" STRUCT( "max_number_of_training_jobs" BIGINT, "max_parallel_training_jobs" BIGINT, "max_runtime_in_seconds" BIGINT ), "parameter_ranges" STRUCT( "integer_parameter_ranges" STRUCT( "name" VARCHAR, "min_value" VARCHAR, "max_value" VARCHAR, "scaling_type" VARCHAR )[], "continuous_parameter_ranges" STRUCT( "name" VARCHAR, "min_value" VARCHAR, "max_value" VARCHAR, "scaling_type" VARCHAR )[], "categorical_parameter_ranges" STRUCT( "name" VARCHAR, "values" VARCHAR[] )[], "auto_parameters" STRUCT( "name" VARCHAR, "value_hint" VARCHAR )[] ), "training_job_early_stopping_type" VARCHAR, "tuning_job_completion_criteria" STRUCT( "target_objective_metric_value" DOUBLE, "best_objective_not_improving" STRUCT( "max_number_of_training_jobs_not_improving" BIGINT ), "convergence_detected" STRUCT( "complete_on_convergence" VARCHAR ) ), "random_seed" BIGINT )
Show child fields `hyper_parameter_tuning_job_config.hyper_parameter_tuning_job_objective` The HyperParameterTuningJobObjective specifies the objective metric used to evaluate the performance of training jobs launched by this tuning job. Show child fields `hyper_parameter_tuning_job_config.hyper_parameter_tuning_job_objective.metric_name` The name of the metric to use for the objective metric. `hyper_parameter_tuning_job_config.hyper_parameter_tuning_job_objective.type` Whether to minimize or maximize the objective metric. `hyper_parameter_tuning_job_config.parameter_ranges` The ParameterRanges object that specifies the ranges of hyperparameters that this tuning job searches over to find the optimal configuration for the highest model performance against your chosen objective metric. Show child fields `hyper_parameter_tuning_job_config.parameter_ranges.auto_parameters[]` Show child fields `hyper_parameter_tuning_job_config.parameter_ranges.auto_parameters[].name` The name of the hyperparameter to optimize using Autotune. `hyper_parameter_tuning_job_config.parameter_ranges.auto_parameters[].value_hint` An example value of the hyperparameter to optimize using Autotune. `hyper_parameter_tuning_job_config.parameter_ranges.categorical_parameter_ranges[]` Show child fields `hyper_parameter_tuning_job_config.parameter_ranges.categorical_parameter_ranges[].name` The name of the categorical hyperparameter to tune. `hyper_parameter_tuning_job_config.parameter_ranges.categorical_parameter_ranges[].values[]` `hyper_parameter_tuning_job_config.parameter_ranges.continuous_parameter_ranges[]` Show child fields `hyper_parameter_tuning_job_config.parameter_ranges.continuous_parameter_ranges[].max_value` The maximum value for the hyperparameter. The tuning job uses floating-point values between `MinValue` value and this value for tuning. `hyper_parameter_tuning_job_config.parameter_ranges.continuous_parameter_ranges[].min_value` The minimum value for the hyperparameter. The tuning job uses floating-point values between this value and `MaxValue`for tuning. `hyper_parameter_tuning_job_config.parameter_ranges.continuous_parameter_ranges[].name` The name of the continuous hyperparameter to tune. `hyper_parameter_tuning_job_config.parameter_ranges.continuous_parameter_ranges[].scaling_type` The scale that hyperparameter tuning uses to search the hyperparameter range. For information about choosing a hyperparameter scale, see Hyperparameter Scaling. One of the following values: Auto SageMaker hyperparameter tuning chooses the best scale for the hyperparameter. Linear Hyperparameter tuning searches the values in the hyperparameter range by using a linear scale. Logarithmic Hyperparameter tuning searches the values in the hyperparameter range by using a logarithmic scale. Logarithmic scaling works only for ranges that have only values greater than 0. ReverseLogarithmic Hyperparameter tuning searches the values in the hyperparameter range by using a reverse logarithmic scale. Reverse logarithmic scaling works only for ranges that are entirely within the range 0<=x<1.0. `hyper_parameter_tuning_job_config.parameter_ranges.integer_parameter_ranges[]` Show child fields `hyper_parameter_tuning_job_config.parameter_ranges.integer_parameter_ranges[].max_value` The maximum value of the hyperparameter to search. `hyper_parameter_tuning_job_config.parameter_ranges.integer_parameter_ranges[].min_value` The minimum value of the hyperparameter to search. `hyper_parameter_tuning_job_config.parameter_ranges.integer_parameter_ranges[].name` The name of the hyperparameter to search. `hyper_parameter_tuning_job_config.parameter_ranges.integer_parameter_ranges[].scaling_type` The scale that hyperparameter tuning uses to search the hyperparameter range. For information about choosing a hyperparameter scale, see Hyperparameter Scaling. One of the following values: Auto SageMaker hyperparameter tuning chooses the best scale for the hyperparameter. Linear Hyperparameter tuning searches the values in the hyperparameter range by using a linear scale. Logarithmic Hyperparameter tuning searches the values in the hyperparameter range by using a logarithmic scale. Logarithmic scaling works only for ranges that have only values greater than 0. `hyper_parameter_tuning_job_config.random_seed` A value used to initialize a pseudo-random number generator. Setting a random seed and using the same seed later for the same tuning job will allow hyperparameter optimization to find more a consistent hyperparameter configuration between the two runs. `hyper_parameter_tuning_job_config.resource_limits` The ResourceLimits object that specifies the maximum number of training and parallel training jobs that can be used for this hyperparameter tuning job. Show child fields `hyper_parameter_tuning_job_config.resource_limits.max_number_of_training_jobs` The maximum number of training jobs that a hyperparameter tuning job can launch. `hyper_parameter_tuning_job_config.resource_limits.max_parallel_training_jobs` The maximum number of concurrent training jobs that a hyperparameter tuning job can launch. `hyper_parameter_tuning_job_config.resource_limits.max_runtime_in_seconds` The maximum time in seconds that a hyperparameter tuning job can run. `hyper_parameter_tuning_job_config.strategy` Specifies how hyperparameter tuning chooses the combinations of hyperparameter values to use for the training job it launches. For information about search strategies, see How Hyperparameter Tuning Works. `hyper_parameter_tuning_job_config.strategy_config` The configuration for the `Hyperband` optimization strategy. This parameter should be provided only if `Hyperband` is selected as the strategy for `HyperParameterTuningJobConfig`. Show child fields `hyper_parameter_tuning_job_config.strategy_config.hyperband_strategy_config` The configuration for the object that specifies the `Hyperband` strategy. This parameter is only supported for the `Hyperband` selection for `Strategy` within the `HyperParameterTuningJobConfig` API. Show child fields `hyper_parameter_tuning_job_config.strategy_config.hyperband_strategy_config.max_resource` The maximum number of resources (such as epochs) that can be used by a training job launched by a hyperparameter tuning job. Once a job reaches the `MaxResource` value, it is stopped. If a value for `MaxResource` is not provided, and `Hyperband` is selected as the hyperparameter tuning strategy, `HyperbandTraining` attempts to infer `MaxResource` from the following keys (if present) in StaticsHyperParameters: `epochs` `numepochs` `n-epochs` `n_epochs` `num_epochs` If `HyperbandStrategyConfig` is unable to infer a value for `MaxResource`, it generates a validation error. The maximum value is 20,000 epochs. All metrics that correspond to an objective metric are used to derive early stopping decisions. For distributed training jobs, ensure that duplicate metrics are not printed in the logs across the individual nodes in a training job. If multiple nodes are publishing duplicate or incorrect metrics, training jobs may make an incorrect stopping decision and stop the job prematurely. `hyper_parameter_tuning_job_config.strategy_config.hyperband_strategy_config.min_resource` The minimum number of resources (such as epochs) that can be used by a training job launched by a hyperparameter tuning job. If the value for `MinResource` has not been reached, the training job is not stopped by `Hyperband`. `hyper_parameter_tuning_job_config.training_job_early_stopping_type` Specifies whether to use early stopping for training jobs launched by the hyperparameter tuning job. Because the `Hyperband` strategy has its own advanced internal early stopping mechanism, `TrainingJobEarlyStoppingType` must be `OFF` to use `Hyperband`. This parameter can take on one of the following values (the default value is `OFF`): OFF Training jobs launched by the hyperparameter tuning job do not use early stopping. AUTO SageMaker stops training jobs launched by the hyperparameter tuning job when they are unlikely to perform better than previously completed training jobs. For more information, see Stop Training Jobs Early. `hyper_parameter_tuning_job_config.tuning_job_completion_criteria` The tuning job's completion criteria. Show child fields `hyper_parameter_tuning_job_config.tuning_job_completion_criteria.best_objective_not_improving` A flag to stop your hyperparameter tuning job if model performance fails to improve as evaluated against an objective function. Show child fields `hyper_parameter_tuning_job_config.tuning_job_completion_criteria.best_objective_not_improving.max_number_of_training_jobs_not_improving` The number of training jobs that have failed to improve model performance by 1% or greater over prior training jobs as evaluated against an objective function. `hyper_parameter_tuning_job_config.tuning_job_completion_criteria.convergence_detected` A flag to top your hyperparameter tuning job if automatic model tuning (AMT) has detected that your model has converged as evaluated against your objective function. Show child fields `hyper_parameter_tuning_job_config.tuning_job_completion_criteria.convergence_detected.complete_on_convergence` A flag to stop a tuning job once AMT has detected that the job has converged. `hyper_parameter_tuning_job_config.tuning_job_completion_criteria.target_objective_metric_value` The value of the objective metric.
`hyper_parameter_tuning_job_status` The status of the tuning job.	VARCHAR
`last_modified_time` The date and time that the status of the tuning job was modified.	TIMESTAMP_S
`objective_status_counters` The ObjectiveStatusCounters object that specifies the number of training jobs, categorized by the status of their final objective metric, that this tuning job launched.	STRUCT( "succeeded" BIGINT, "pending" BIGINT, "failed" BIGINT )
Show child fields `objective_status_counters.failed` The number of training jobs whose final objective metric was not evaluated and used in the hyperparameter tuning process. This typically occurs when the training job failed or did not emit an objective metric. `objective_status_counters.pending` The number of training jobs that are in progress and pending evaluation of their final objective metric. `objective_status_counters.succeeded` The number of training jobs whose final objective metric was evaluated by the hyperparameter tuning job and used in the hyperparameter tuning process.
`overall_best_training_job` If the hyperparameter tuning job is an warm start tuning job with a `WarmStartType` of `IDENTICAL_DATA_AND_ALGORITHM`, this is the TrainingJobSummary for the training job with the best objective metric value of all training jobs launched by this tuning job and all parent jobs specified for the warm start tuning job.	STRUCT( "training_job_definition_name" VARCHAR, "training_job_name" VARCHAR, "training_job_arn" VARCHAR, "tuning_job_name" VARCHAR, "creation_time" TIMESTAMP_S, "training_start_time" TIMESTAMP_S, "training_end_time" TIMESTAMP_S, "training_job_status" VARCHAR, "tuned_hyper_parameters" MAP(VARCHAR, VARCHAR), "failure_reason" VARCHAR, "final_hyper_parameter_tuning_job_objective_metric" STRUCT( "type" VARCHAR, "metric_name" VARCHAR, "value" DOUBLE ), "objective_status" VARCHAR )
Show child fields `overall_best_training_job.creation_time` The date and time that the training job was created. `overall_best_training_job.failure_reason` The reason that the training job failed. `overall_best_training_job.final_hyper_parameter_tuning_job_objective_metric` The FinalHyperParameterTuningJobObjectiveMetric object that specifies the value of the objective metric of the tuning job that launched this training job. Show child fields `overall_best_training_job.final_hyper_parameter_tuning_job_objective_metric.metric_name` The name of the objective metric. For SageMaker built-in algorithms, metrics are defined per algorithm. See the metrics for XGBoost as an example. You can also use a custom algorithm for training and define your own metrics. For more information, see Define metrics and environment variables. `overall_best_training_job.final_hyper_parameter_tuning_job_objective_metric.type` Select if you want to minimize or maximize the objective metric during hyperparameter tuning. `overall_best_training_job.final_hyper_parameter_tuning_job_objective_metric.value` The value of the objective metric. `overall_best_training_job.objective_status` The status of the objective metric for the training job: Succeeded: The final objective metric for the training job was evaluated by the hyperparameter tuning job and used in the hyperparameter tuning process. Pending: The training job is in progress and evaluation of its final objective metric is pending. Failed: The final objective metric for the training job was not evaluated, and was not used in the hyperparameter tuning process. This typically occurs when the training job failed or did not emit an objective metric. `overall_best_training_job.training_end_time` Specifies the time when the training job ends on training instances. You are billed for the time interval between the value of `TrainingStartTime` and this time. For successful jobs and stopped jobs, this is the time after model artifacts are uploaded. For failed jobs, this is the time when SageMaker detects a job failure. `overall_best_training_job.training_job_arn` The Amazon Resource Name (ARN) of the training job. `overall_best_training_job.training_job_definition_name` The training job definition name. `overall_best_training_job.training_job_name` The name of the training job. `overall_best_training_job.training_job_status` The status of the training job. `overall_best_training_job.training_start_time` The date and time that the training job started. `overall_best_training_job.tuned_hyper_parameters` A list of the hyperparameters for which you specified ranges to search. `overall_best_training_job.tuning_job_name` The HyperParameter tuning job that launched the training job.
`training_job_definition` The HyperParameterTrainingJobDefinition object that specifies the definition of the training jobs that this tuning job launches.	STRUCT( "definition_name" VARCHAR, "tuning_objective" STRUCT( "type" VARCHAR, "metric_name" VARCHAR ), "hyper_parameter_ranges" STRUCT( "integer_parameter_ranges" STRUCT( "name" VARCHAR, "min_value" VARCHAR, "max_value" VARCHAR, "scaling_type" VARCHAR )[], "continuous_parameter_ranges" STRUCT( "name" VARCHAR, "min_value" VARCHAR, "max_value" VARCHAR, "scaling_type" VARCHAR )[], "categorical_parameter_ranges" STRUCT( "name" VARCHAR, "values" VARCHAR[] )[], "auto_parameters" STRUCT( "name" VARCHAR, "value_hint" VARCHAR )[] ), "static_hyper_parameters" MAP(VARCHAR, VARCHAR), "algorithm_specification" STRUCT( "training_image" VARCHAR, "training_input_mode" VARCHAR, "algorithm_name" VARCHAR, "metric_definitions" STRUCT( "name" VARCHAR, "regex" VARCHAR )[] ), "role_arn" VARCHAR, "input_data_config" STRUCT( "channel_name" VARCHAR, "data_source" STRUCT( "s3_data_source" STRUCT( "s3_data_type" VARCHAR, "s3_uri" VARCHAR, "s3_data_distribution_type" VARCHAR, "attribute_names" VARCHAR[], "instance_group_names" VARCHAR[] ), "file_system_data_source" STRUCT( "file_system_id" VARCHAR, "file_system_access_mode" VARCHAR, "file_system_type" VARCHAR, "directory_path" VARCHAR ) ), "content_type" VARCHAR, "compression_type" VARCHAR, "record_wrapper_type" VARCHAR, "input_mode" VARCHAR, "shuffle_config" STRUCT( "seed" BIGINT ) )[], "vpc_config" STRUCT( "security_group_ids" VARCHAR[], "subnets" VARCHAR[] ), "output_data_config" STRUCT( "kms_key_id" VARCHAR, "s3_output_path" VARCHAR, "compression_type" VARCHAR ), "resource_config" STRUCT( "instance_type" VARCHAR, "instance_count" BIGINT, "volume_size_in_gb" BIGINT, "volume_kms_key_id" VARCHAR, "keep_alive_period_in_seconds" BIGINT, "instance_groups" STRUCT( "instance_type" VARCHAR, "instance_count" BIGINT, "instance_group_name" VARCHAR )[] ), "hyper_parameter_tuning_resource_config" STRUCT( "instance_type" VARCHAR, "instance_count" BIGINT, "volume_size_in_gb" BIGINT, "volume_kms_key_id" VARCHAR, "allocation_strategy" VARCHAR, "instance_configs" STRUCT( "instance_type" VARCHAR, "instance_count" BIGINT, "volume_size_in_gb" BIGINT )[] ), "stopping_condition" STRUCT( "max_runtime_in_seconds" BIGINT, "max_wait_time_in_seconds" BIGINT, "max_pending_time_in_seconds" BIGINT ), "enable_network_isolation" BOOLEAN, "enable_inter_container_traffic_encryption" BOOLEAN, "enable_managed_spot_training" BOOLEAN, "checkpoint_config" STRUCT( "s3_uri" VARCHAR, "local_path" VARCHAR ), "retry_strategy" STRUCT( "maximum_retry_attempts" BIGINT ), "environment" MAP(VARCHAR, VARCHAR) )
Show child fields `training_job_definition.algorithm_specification` The HyperParameterAlgorithmSpecification object that specifies the resource algorithm to use for the training jobs that the tuning job launches. Show child fields `training_job_definition.algorithm_specification.algorithm_name` The name of the resource algorithm to use for the hyperparameter tuning job. If you specify a value for this parameter, do not specify a value for `TrainingImage`. `training_job_definition.algorithm_specification.metric_definitions[]` Show child fields `training_job_definition.algorithm_specification.metric_definitions[].name` The name of the metric. `training_job_definition.algorithm_specification.metric_definitions[].regex` A regular expression that searches the output of a training job and gets the value of the metric. For more information about using regular expressions to define metrics, see Defining metrics and environment variables. `training_job_definition.algorithm_specification.training_image` The registry path of the Docker image that contains the training algorithm. For information about Docker registry paths for built-in algorithms, see Algorithms Provided by Amazon SageMaker: Common Parameters. SageMaker supports both `registry/repository[:tag]` and `registry/repository[@digest]` image path formats. For more information, see Using Your Own Algorithms with Amazon SageMaker. `training_job_definition.algorithm_specification.training_input_mode` The training input mode that the algorithm supports. For more information about input modes, see Algorithms. Pipe mode If an algorithm supports `Pipe` mode, Amazon SageMaker streams data directly from Amazon S3 to the container. File mode If an algorithm supports `File` mode, SageMaker downloads the training data from S3 to the provisioned ML storage volume, and mounts the directory to the Docker volume for the training container. You must provision the ML storage volume with sufficient capacity to accommodate the data downloaded from S3. In addition to the training data, the ML storage volume also stores the output model. The algorithm container uses the ML storage volume to also store intermediate information, if any. For distributed algorithms, training data is distributed uniformly. Your training duration is predictable if the input data objects sizes are approximately the same. SageMaker does not split the files any further for model training. If the object sizes are skewed, training won't be optimal as the data distribution is also skewed when one host in a training cluster is overloaded, thus becoming a bottleneck in training. FastFile mode If an algorithm supports `FastFile` mode, SageMaker streams data directly from S3 to the container with no code changes, and provides file system access to the data. Users can author their training script to interact with these files as if they were stored on disk. `FastFile` mode works best when the data is read sequentially. Augmented manifest files aren't supported. The startup time is lower when there are fewer files in the S3 bucket provided. `training_job_definition.checkpoint_config` Contains information about the output location for managed spot training checkpoint data. Show child fields `training_job_definition.checkpoint_config.local_path` (Optional) The local directory where checkpoints are written. The default directory is `/opt/ml/checkpoints/`. `training_job_definition.checkpoint_config.s3_uri` Identifies the S3 path where you want SageMaker to store checkpoints. For example, `s3://bucket-name/key-name-prefix`. `training_job_definition.definition_name` The job definition name. `training_job_definition.enable_inter_container_traffic_encryption` To encrypt all communications between ML compute instances in distributed training, choose `True`. Encryption provides greater security for distributed training, but training might take longer. How long it takes depends on the amount of communication between compute instances, especially if you use a deep learning algorithm in distributed training. `training_job_definition.enable_managed_spot_training` A Boolean indicating whether managed spot training is enabled (`True`) or not (`False`). `training_job_definition.enable_network_isolation` Isolates the training container. No inbound or outbound network calls can be made, except for calls between peers within a training cluster for distributed training. If network isolation is used for training jobs that are configured to use a VPC, SageMaker downloads and uploads customer data and model artifacts through the specified VPC, but the training container does not have network access. `training_job_definition.environment` An environment variable that you can pass into the SageMaker CreateTrainingJob API. You can use an existing environment variable from the training container or use your own. See Define metrics and variables for more information. The maximum number of items specified for `Map Entries` refers to the maximum number of environment variables for each `TrainingJobDefinition` and also the maximum for the hyperparameter tuning job itself. That is, the sum of the number of environment variables for all the training job definitions can't exceed the maximum number specified. `training_job_definition.hyper_parameter_ranges` Specifies ranges of integer, continuous, and categorical hyperparameters that a hyperparameter tuning job searches. The hyperparameter tuning job launches training jobs with hyperparameter values within these ranges to find the combination of values that result in the training job with the best performance as measured by the objective metric of the hyperparameter tuning job. The maximum number of items specified for `Array Members` refers to the maximum number of hyperparameters for each range and also the maximum for the hyperparameter tuning job itself. That is, the sum of the number of hyperparameters for all the ranges can't exceed the maximum number specified. Show child fields `training_job_definition.hyper_parameter_ranges.auto_parameters[]` Show child fields `training_job_definition.hyper_parameter_ranges.auto_parameters[].name` The name of the hyperparameter to optimize using Autotune. `training_job_definition.hyper_parameter_ranges.auto_parameters[].value_hint` An example value of the hyperparameter to optimize using Autotune. `training_job_definition.hyper_parameter_ranges.categorical_parameter_ranges[]` Show child fields `training_job_definition.hyper_parameter_ranges.categorical_parameter_ranges[].name` The name of the categorical hyperparameter to tune. `training_job_definition.hyper_parameter_ranges.categorical_parameter_ranges[].values[]` `training_job_definition.hyper_parameter_ranges.continuous_parameter_ranges[]` Show child fields `training_job_definition.hyper_parameter_ranges.continuous_parameter_ranges[].max_value` The maximum value for the hyperparameter. The tuning job uses floating-point values between `MinValue` value and this value for tuning. `training_job_definition.hyper_parameter_ranges.continuous_parameter_ranges[].min_value` The minimum value for the hyperparameter. The tuning job uses floating-point values between this value and `MaxValue`for tuning. `training_job_definition.hyper_parameter_ranges.continuous_parameter_ranges[].name` The name of the continuous hyperparameter to tune. `training_job_definition.hyper_parameter_ranges.continuous_parameter_ranges[].scaling_type` The scale that hyperparameter tuning uses to search the hyperparameter range. For information about choosing a hyperparameter scale, see Hyperparameter Scaling. One of the following values: Auto SageMaker hyperparameter tuning chooses the best scale for the hyperparameter. Linear Hyperparameter tuning searches the values in the hyperparameter range by using a linear scale. Logarithmic Hyperparameter tuning searches the values in the hyperparameter range by using a logarithmic scale. Logarithmic scaling works only for ranges that have only values greater than 0. ReverseLogarithmic Hyperparameter tuning searches the values in the hyperparameter range by using a reverse logarithmic scale. Reverse logarithmic scaling works only for ranges that are entirely within the range 0<=x<1.0. `training_job_definition.hyper_parameter_ranges.integer_parameter_ranges[]` Show child fields `training_job_definition.hyper_parameter_ranges.integer_parameter_ranges[].max_value` The maximum value of the hyperparameter to search. `training_job_definition.hyper_parameter_ranges.integer_parameter_ranges[].min_value` The minimum value of the hyperparameter to search. `training_job_definition.hyper_parameter_ranges.integer_parameter_ranges[].name` The name of the hyperparameter to search. `training_job_definition.hyper_parameter_ranges.integer_parameter_ranges[].scaling_type` The scale that hyperparameter tuning uses to search the hyperparameter range. For information about choosing a hyperparameter scale, see Hyperparameter Scaling. One of the following values: Auto SageMaker hyperparameter tuning chooses the best scale for the hyperparameter. Linear Hyperparameter tuning searches the values in the hyperparameter range by using a linear scale. Logarithmic Hyperparameter tuning searches the values in the hyperparameter range by using a logarithmic scale. Logarithmic scaling works only for ranges that have only values greater than 0. `training_job_definition.hyper_parameter_tuning_resource_config` The configuration for the hyperparameter tuning resources, including the compute instances and storage volumes, used for training jobs launched by the tuning job. By default, storage volumes hold model artifacts and incremental states. Choose `File` for `TrainingInputMode` in the `AlgorithmSpecification` parameter to additionally store training data in the storage volume (optional). Show child fields `training_job_definition.hyper_parameter_tuning_resource_config.allocation_strategy` The strategy that determines the order of preference for resources specified in `InstanceConfigs` used in hyperparameter optimization. `training_job_definition.hyper_parameter_tuning_resource_config.instance_configs[]` Show child fields `training_job_definition.hyper_parameter_tuning_resource_config.instance_configs[].instance_count` The number of instances of the type specified by `InstanceType`. Choose an instance count larger than 1 for distributed training algorithms. See Step 2: Launch a SageMaker Distributed Training Job Using the SageMaker Python SDK for more information. `training_job_definition.hyper_parameter_tuning_resource_config.instance_configs[].instance_type` The instance type used for processing of hyperparameter optimization jobs. Choose from general purpose (no GPUs) instance types: ml.m5.xlarge, ml.m5.2xlarge, and ml.m5.4xlarge or compute optimized (no GPUs) instance types: ml.c5.xlarge and ml.c5.2xlarge. For more information about instance types, see instance type descriptions. `training_job_definition.hyper_parameter_tuning_resource_config.instance_configs[].volume_size_in_gb` The volume size in GB of the data to be processed for hyperparameter optimization (optional). `training_job_definition.hyper_parameter_tuning_resource_config.instance_count` The number of compute instances of type `InstanceType` to use. For distributed training, select a value greater than 1. `training_job_definition.hyper_parameter_tuning_resource_config.instance_type` The instance type used to run hyperparameter optimization tuning jobs. See descriptions of instance types for more information. `training_job_definition.hyper_parameter_tuning_resource_config.volume_kms_key_id` A key used by Amazon Web Services Key Management Service to encrypt data on the storage volume attached to the compute instances used to run the training job. You can use either of the following formats to specify a key. KMS Key ID: `"1234abcd-12ab-34cd-56ef-1234567890ab"` Amazon Resource Name (ARN) of a KMS key: `"arn:aws:kms:us-west-2:111122223333:key/1234abcd-12ab-34cd-56ef-1234567890ab"` Some instances use local storage, which use a hardware module to encrypt storage volumes. If you choose one of these instance types, you cannot request a `VolumeKmsKeyId`. For a list of instance types that use local storage, see instance store volumes. For more information about Amazon Web Services Key Management Service, see KMS encryption for more information. `training_job_definition.hyper_parameter_tuning_resource_config.volume_size_in_gb` The volume size in GB for the storage volume to be used in processing hyperparameter optimization jobs (optional). These volumes store model artifacts, incremental states and optionally, scratch space for training algorithms. Do not provide a value for this parameter if a value for `InstanceConfigs` is also specified. Some instance types have a fixed total local storage size. If you select one of these instances for training, `VolumeSizeInGB` cannot be greater than this total size. For a list of instance types with local instance storage and their sizes, see instance store volumes. SageMaker supports only the General Purpose SSD (gp2) storage volume type. `training_job_definition.input_data_config[]` Show child fields `training_job_definition.input_data_config[].channel_name` The name of the channel. `training_job_definition.input_data_config[].compression_type` If training data is compressed, the compression type. The default value is `None`. `CompressionType` is used only in Pipe input mode. In File mode, leave this field unset or set it to None. `training_job_definition.input_data_config[].content_type` The MIME type of the data. `training_job_definition.input_data_config[].data_source` The location of the channel data. Show child fields `training_job_definition.input_data_config[].data_source.file_system_data_source` The file system that is associated with a channel. Show child fields `training_job_definition.input_data_config[].data_source.file_system_data_source.directory_path` The full path to the directory to associate with the channel. `training_job_definition.input_data_config[].data_source.file_system_data_source.file_system_access_mode` The access mode of the mount of the directory associated with the channel. A directory can be mounted either in `ro` (read-only) or `rw` (read-write) mode. `training_job_definition.input_data_config[].data_source.file_system_data_source.file_system_id` The file system id. `training_job_definition.input_data_config[].data_source.file_system_data_source.file_system_type` The file system type. `training_job_definition.input_data_config[].data_source.s3_data_source` The S3 location of the data source that is associated with a channel. Show child fields `training_job_definition.input_data_config[].data_source.s3_data_source.attribute_names[]` `training_job_definition.input_data_config[].data_source.s3_data_source.instance_group_names[]` `training_job_definition.input_data_config[].data_source.s3_data_source.s3_data_distribution_type` If you want SageMaker to replicate the entire dataset on each ML compute instance that is launched for model training, specify `FullyReplicated`. If you want SageMaker to replicate a subset of data on each ML compute instance that is launched for model training, specify `ShardedByS3Key`. If there are n ML compute instances launched for a training job, each instance gets approximately 1/n of the number of S3 objects. In this case, model training on each machine uses only the subset of training data. Don't choose more ML compute instances for training than available S3 objects. If you do, some nodes won't get any data and you will pay for nodes that aren't getting any training data. This applies in both File and Pipe modes. Keep this in mind when developing algorithms. In distributed training, where you use multiple ML compute EC2 instances, you might choose `ShardedByS3Key`. If the algorithm requires copying training data to the ML storage volume (when `TrainingInputMode` is set to `File`), this copies 1/n of the number of objects. `training_job_definition.input_data_config[].data_source.s3_data_source.s3_data_type` If you choose `S3Prefix`, `S3Uri` identifies a key name prefix. SageMaker uses all objects that match the specified key name prefix for model training. If you choose `ManifestFile`, `S3Uri` identifies an object that is a manifest file containing a list of object keys that you want SageMaker to use for model training. If you choose `AugmentedManifestFile`, S3Uri identifies an object that is an augmented manifest file in JSON lines format. This file contains the data you want to use for model training. `AugmentedManifestFile` can only be used if the Channel's input mode is `Pipe`. `training_job_definition.input_data_config[].data_source.s3_data_source.s3_uri` Depending on the value specified for the `S3DataType`, identifies either a key name prefix or a manifest. For example: A key name prefix might look like this: `s3://bucketname/exampleprefix/` A manifest might look like this: `s3://bucketname/example.manifest` A manifest is an S3 object which is a JSON file consisting of an array of elements. The first element is a prefix which is followed by one or more suffixes. SageMaker appends the suffix elements to the prefix to get a full set of `S3Uri`. Note that the prefix must be a valid non-empty `S3Uri` that precludes users from specifying a manifest whose individual `S3Uri` is sourced from different S3 buckets. The following code example shows a valid manifest format: `[ {"prefix": "s3://customer_bucket/some/prefix/"},` `"relative/path/to/custdata-1",` `"relative/path/custdata-2",` `...` `"relative/path/custdata-N"` `]` This JSON is equivalent to the following `S3Uri` list: `s3://customer_bucket/some/prefix/relative/path/to/custdata-1` `s3://customer_bucket/some/prefix/relative/path/custdata-2` `...` `s3://customer_bucket/some/prefix/relative/path/custdata-N` The complete set of `S3Uri` in this manifest is the input data for the channel for this data source. The object that each `S3Uri` points to must be readable by the IAM role that SageMaker uses to perform tasks on your behalf. Your input bucket must be located in same Amazon Web Services region as your training job. `training_job_definition.input_data_config[].input_mode` (Optional) The input mode to use for the data channel in a training job. If you don't set a value for `InputMode`, SageMaker uses the value set for `TrainingInputMode`. Use this parameter to override the `TrainingInputMode` setting in a AlgorithmSpecification request when you have a channel that needs a different input mode from the training job's general setting. To download the data from Amazon Simple Storage Service (Amazon S3) to the provisioned ML storage volume, and mount the directory to a Docker volume, use `File` input mode. To stream data directly from Amazon S3 to the container, choose `Pipe` input mode. To use a model for incremental training, choose `File` input model. `training_job_definition.input_data_config[].record_wrapper_type` Specify RecordIO as the value when input data is in raw format but the training algorithm requires the RecordIO format. In this case, SageMaker wraps each individual S3 object in a RecordIO record. If the input data is already in RecordIO format, you don't need to set this attribute. For more information, see Create a Dataset Using RecordIO. In File mode, leave this field unset or set it to None. `training_job_definition.input_data_config[].shuffle_config` A configuration for a shuffle option for input data in a channel. If you use `S3Prefix` for `S3DataType`, this shuffles the results of the S3 key prefix matches. If you use `ManifestFile`, the order of the S3 object references in the `ManifestFile` is shuffled. If you use `AugmentedManifestFile`, the order of the JSON lines in the `AugmentedManifestFile` is shuffled. The shuffling order is determined using the `Seed` value. For Pipe input mode, shuffling is done at the start of every epoch. With large datasets this ensures that the order of the training data is different for each epoch, it helps reduce bias and possible overfitting. In a multi-node training job when ShuffleConfig is combined with `S3DataDistributionType` of `ShardedByS3Key`, the data is shuffled across nodes so that the content sent to a particular node on the first epoch might be sent to a different node on the second epoch. Show child fields `training_job_definition.input_data_config[].shuffle_config.seed` Determines the shuffling order in `ShuffleConfig` value. `training_job_definition.output_data_config` Specifies the path to the Amazon S3 bucket where you store model artifacts from the training jobs that the tuning job launches. Show child fields `training_job_definition.output_data_config.compression_type` The model output compression type. Select `None` to output an uncompressed model, recommended for large model outputs. Defaults to gzip. `training_job_definition.output_data_config.kms_key_id` The Amazon Web Services Key Management Service (Amazon Web Services KMS) key that SageMaker uses to encrypt the model artifacts at rest using Amazon S3 server-side encryption. The `KmsKeyId` can be any of the following formats: // KMS Key ID `"1234abcd-12ab-34cd-56ef-1234567890ab"` // Amazon Resource Name (ARN) of a KMS Key `"arn:aws:kms:us-west-2:111122223333:key/1234abcd-12ab-34cd-56ef-1234567890ab"` // KMS Key Alias `"alias/ExampleAlias"` // Amazon Resource Name (ARN) of a KMS Key Alias `"arn:aws:kms:us-west-2:111122223333:alias/ExampleAlias"` If you use a KMS key ID or an alias of your KMS key, the SageMaker execution role must include permissions to call `kms:Encrypt`. If you don't provide a KMS key ID, SageMaker uses the default KMS key for Amazon S3 for your role's account. For more information, see KMS-Managed Encryption Keys in the Amazon Simple Storage Service Developer Guide. If the output data is stored in Amazon S3 Express One Zone, it is encrypted with server-side encryption with Amazon S3 managed keys (SSE-S3). KMS key is not supported for Amazon S3 Express One Zone The KMS key policy must grant permission to the IAM role that you specify in your `CreateTrainingJob`, `CreateTransformJob`, or `CreateHyperParameterTuningJob` requests. For more information, see Using Key Policies in Amazon Web Services KMS in the Amazon Web Services Key Management Service Developer Guide. `training_job_definition.output_data_config.s3_output_path` Identifies the S3 path where you want SageMaker to store the model artifacts. For example, `s3://bucket-name/key-name-prefix`. `training_job_definition.resource_config` The resources, including the compute instances and storage volumes, to use for the training jobs that the tuning job launches. Storage volumes store model artifacts and incremental states. Training algorithms might also use storage volumes for scratch space. If you want SageMaker to use the storage volume to store the training data, choose `File` as the `TrainingInputMode` in the algorithm specification. For distributed training algorithms, specify an instance count greater than 1. If you want to use hyperparameter optimization with instance type flexibility, use `HyperParameterTuningResourceConfig` instead. Show child fields `training_job_definition.resource_config.instance_count` The number of ML compute instances to use. For distributed training, provide a value greater than 1. `training_job_definition.resource_config.instance_groups[]` Show child fields `training_job_definition.resource_config.instance_groups[].instance_count` Specifies the number of instances of the instance group. `training_job_definition.resource_config.instance_groups[].instance_group_name` Specifies the name of the instance group. `training_job_definition.resource_config.instance_groups[].instance_type` Specifies the instance type of the instance group. `training_job_definition.resource_config.instance_type` The ML compute instance type. SageMaker Training on Amazon Elastic Compute Cloud (EC2) P4de instances is in preview release starting December 9th, 2022. Amazon EC2 P4de instances (currently in preview) are powered by 8 NVIDIA A100 GPUs with 80GB high-performance HBM2e GPU memory, which accelerate the speed of training ML models that need to be trained on large datasets of high-resolution data. In this preview release, Amazon SageMaker supports ML training jobs on P4de instances (`ml.p4de.24xlarge`) to reduce model training time. The `ml.p4de.24xlarge` instances are available in the following Amazon Web Services Regions. US East (N. Virginia) (us-east-1) US West (Oregon) (us-west-2) To request quota limit increase and start using P4de instances, contact the SageMaker Training service team through your account team. `training_job_definition.resource_config.keep_alive_period_in_seconds` The duration of time in seconds to retain configured resources in a warm pool for subsequent training jobs. `training_job_definition.resource_config.volume_kms_key_id` The Amazon Web Services KMS key that SageMaker uses to encrypt data on the storage volume attached to the ML compute instance(s) that run the training job. Certain Nitro-based instances include local storage, dependent on the instance type. Local storage volumes are encrypted using a hardware module on the instance. You can't request a `VolumeKmsKeyId` when using an instance type with local storage. For a list of instance types that support local instance storage, see Instance Store Volumes. For more information about local instance storage encryption, see SSD Instance Store Volumes. The `VolumeKmsKeyId` can be in any of the following formats: // KMS Key ID `"1234abcd-12ab-34cd-56ef-1234567890ab"` // Amazon Resource Name (ARN) of a KMS Key `"arn:aws:kms:us-west-2:111122223333:key/1234abcd-12ab-34cd-56ef-1234567890ab"` `training_job_definition.resource_config.volume_size_in_gb` The size of the ML storage volume that you want to provision. ML storage volumes store model artifacts and incremental states. Training algorithms might also use the ML storage volume for scratch space. If you want to store the training data in the ML storage volume, choose `File` as the `TrainingInputMode` in the algorithm specification. When using an ML instance with NVMe SSD volumes, SageMaker doesn't provision Amazon EBS General Purpose SSD (gp2) storage. Available storage is fixed to the NVMe-type instance's storage capacity. SageMaker configures storage paths for training datasets, checkpoints, model artifacts, and outputs to use the entire capacity of the instance storage. For example, ML instance families with the NVMe-type instance storage include `ml.p4d`, `ml.g4dn`, and `ml.g5`. When using an ML instance with the EBS-only storage option and without instance storage, you must define the size of EBS volume through `VolumeSizeInGB` in the `ResourceConfig` API. For example, ML instance families that use EBS volumes include `ml.c5` and `ml.p2`. To look up instance types and their instance storage types and volumes, see Amazon EC2 Instance Types. To find the default local paths defined by the SageMaker training platform, see Amazon SageMaker Training Storage Folders for Training Datasets, Checkpoints, Model Artifacts, and Outputs. `training_job_definition.retry_strategy` The number of times to retry the job when the job fails due to an `InternalServerError`. Show child fields `training_job_definition.retry_strategy.maximum_retry_attempts` The number of times to retry the job. When the job is retried, it's `SecondaryStatus` is changed to `STARTING`. `training_job_definition.role_arn` The Amazon Resource Name (ARN) of the IAM role associated with the training jobs that the tuning job launches. `training_job_definition.static_hyper_parameters` Specifies the values of hyperparameters that do not change for the tuning job. `training_job_definition.stopping_condition` Specifies a limit to how long a model hyperparameter training job can run. It also specifies how long a managed spot training job has to complete. When the job reaches the time limit, SageMaker ends the training job. Use this API to cap model training costs. Show child fields `training_job_definition.stopping_condition.max_pending_time_in_seconds` The maximum length of time, in seconds, that a training or compilation job can be pending before it is stopped. `training_job_definition.stopping_condition.max_runtime_in_seconds` The maximum length of time, in seconds, that a training or compilation job can run before it is stopped. For compilation jobs, if the job does not complete during this time, a `TimeOut` error is generated. We recommend starting with 900 seconds and increasing as necessary based on your model. For all other jobs, if the job does not complete during this time, SageMaker ends the job. When `RetryStrategy` is specified in the job request, `MaxRuntimeInSeconds` specifies the maximum time for all of the attempts in total, not each individual attempt. The default value is 1 day. The maximum value is 28 days. The maximum time that a `TrainingJob` can run in total, including any time spent publishing metrics or archiving and uploading models after it has been stopped, is 30 days. `training_job_definition.stopping_condition.max_wait_time_in_seconds` The maximum length of time, in seconds, that a managed Spot training job has to complete. It is the amount of time spent waiting for Spot capacity plus the amount of time the job can run. It must be equal to or greater than `MaxRuntimeInSeconds`. If the job does not complete during this time, SageMaker ends the job. When `RetryStrategy` is specified in the job request, `MaxWaitTimeInSeconds` specifies the maximum time for all of the attempts in total, not each individual attempt. `training_job_definition.tuning_objective` Defines the objective metric for a hyperparameter tuning job. Hyperparameter tuning uses the value of this metric to evaluate the training jobs it launches, and returns the training job that results in either the highest or lowest value for this metric, depending on the value you specify for the `Type` parameter. If you want to define a custom objective metric, see Define metrics and environment variables. Show child fields `training_job_definition.tuning_objective.metric_name` The name of the metric to use for the objective metric. `training_job_definition.tuning_objective.type` Whether to minimize or maximize the objective metric. `training_job_definition.vpc_config` The VpcConfig object that specifies the VPC that you want the training jobs that this hyperparameter tuning job launches to connect to. Control access to and from your training container by configuring the VPC. For more information, see Protect Training Jobs by Using an Amazon Virtual Private Cloud. Show child fields `training_job_definition.vpc_config.security_group_ids[]` `training_job_definition.vpc_config.subnets[]`
`training_job_definitions` A list of the HyperParameterTrainingJobDefinition objects launched for this tuning job.	STRUCT( "definition_name" VARCHAR, "tuning_objective" STRUCT( "type" VARCHAR, "metric_name" VARCHAR ), "hyper_parameter_ranges" STRUCT( "integer_parameter_ranges" STRUCT( "name" VARCHAR, "min_value" VARCHAR, "max_value" VARCHAR, "scaling_type" VARCHAR )[], "continuous_parameter_ranges" STRUCT( "name" VARCHAR, "min_value" VARCHAR, "max_value" VARCHAR, "scaling_type" VARCHAR )[], "categorical_parameter_ranges" STRUCT( "name" VARCHAR, "values" VARCHAR[] )[], "auto_parameters" STRUCT( "name" VARCHAR, "value_hint" VARCHAR )[] ), "static_hyper_parameters" MAP(VARCHAR, VARCHAR), "algorithm_specification" STRUCT( "training_image" VARCHAR, "training_input_mode" VARCHAR, "algorithm_name" VARCHAR, "metric_definitions" STRUCT( "name" VARCHAR, "regex" VARCHAR )[] ), "role_arn" VARCHAR, "input_data_config" STRUCT( "channel_name" VARCHAR, "data_source" STRUCT( "s3_data_source" STRUCT( "s3_data_type" VARCHAR, "s3_uri" VARCHAR, "s3_data_distribution_type" VARCHAR, "attribute_names" VARCHAR[], "instance_group_names" VARCHAR[] ), "file_system_data_source" STRUCT( "file_system_id" VARCHAR, "file_system_access_mode" VARCHAR, "file_system_type" VARCHAR, "directory_path" VARCHAR ) ), "content_type" VARCHAR, "compression_type" VARCHAR, "record_wrapper_type" VARCHAR, "input_mode" VARCHAR, "shuffle_config" STRUCT( "seed" BIGINT ) )[], "vpc_config" STRUCT( "security_group_ids" VARCHAR[], "subnets" VARCHAR[] ), "output_data_config" STRUCT( "kms_key_id" VARCHAR, "s3_output_path" VARCHAR, "compression_type" VARCHAR ), "resource_config" STRUCT( "instance_type" VARCHAR, "instance_count" BIGINT, "volume_size_in_gb" BIGINT, "volume_kms_key_id" VARCHAR, "keep_alive_period_in_seconds" BIGINT, "instance_groups" STRUCT( "instance_type" VARCHAR, "instance_count" BIGINT, "instance_group_name" VARCHAR )[] ), "hyper_parameter_tuning_resource_config" STRUCT( "instance_type" VARCHAR, "instance_count" BIGINT, "volume_size_in_gb" BIGINT, "volume_kms_key_id" VARCHAR, "allocation_strategy" VARCHAR, "instance_configs" STRUCT( "instance_type" VARCHAR, "instance_count" BIGINT, "volume_size_in_gb" BIGINT )[] ), "stopping_condition" STRUCT( "max_runtime_in_seconds" BIGINT, "max_wait_time_in_seconds" BIGINT, "max_pending_time_in_seconds" BIGINT ), "enable_network_isolation" BOOLEAN, "enable_inter_container_traffic_encryption" BOOLEAN, "enable_managed_spot_training" BOOLEAN, "checkpoint_config" STRUCT( "s3_uri" VARCHAR, "local_path" VARCHAR ), "retry_strategy" STRUCT( "maximum_retry_attempts" BIGINT ), "environment" MAP(VARCHAR, VARCHAR) )[]
Show child fields `training_job_definitions[]` Show child fields `training_job_definitions[].algorithm_specification` The HyperParameterAlgorithmSpecification object that specifies the resource algorithm to use for the training jobs that the tuning job launches. Show child fields `training_job_definitions[].algorithm_specification.algorithm_name` The name of the resource algorithm to use for the hyperparameter tuning job. If you specify a value for this parameter, do not specify a value for `TrainingImage`. `training_job_definitions[].algorithm_specification.metric_definitions[]` Show child fields `training_job_definitions[].algorithm_specification.metric_definitions[].name` The name of the metric. `training_job_definitions[].algorithm_specification.metric_definitions[].regex` A regular expression that searches the output of a training job and gets the value of the metric. For more information about using regular expressions to define metrics, see Defining metrics and environment variables. `training_job_definitions[].algorithm_specification.training_image` The registry path of the Docker image that contains the training algorithm. For information about Docker registry paths for built-in algorithms, see Algorithms Provided by Amazon SageMaker: Common Parameters. SageMaker supports both `registry/repository[:tag]` and `registry/repository[@digest]` image path formats. For more information, see Using Your Own Algorithms with Amazon SageMaker. `training_job_definitions[].algorithm_specification.training_input_mode` The training input mode that the algorithm supports. For more information about input modes, see Algorithms. Pipe mode If an algorithm supports `Pipe` mode, Amazon SageMaker streams data directly from Amazon S3 to the container. File mode If an algorithm supports `File` mode, SageMaker downloads the training data from S3 to the provisioned ML storage volume, and mounts the directory to the Docker volume for the training container. You must provision the ML storage volume with sufficient capacity to accommodate the data downloaded from S3. In addition to the training data, the ML storage volume also stores the output model. The algorithm container uses the ML storage volume to also store intermediate information, if any. For distributed algorithms, training data is distributed uniformly. Your training duration is predictable if the input data objects sizes are approximately the same. SageMaker does not split the files any further for model training. If the object sizes are skewed, training won't be optimal as the data distribution is also skewed when one host in a training cluster is overloaded, thus becoming a bottleneck in training. FastFile mode If an algorithm supports `FastFile` mode, SageMaker streams data directly from S3 to the container with no code changes, and provides file system access to the data. Users can author their training script to interact with these files as if they were stored on disk. `FastFile` mode works best when the data is read sequentially. Augmented manifest files aren't supported. The startup time is lower when there are fewer files in the S3 bucket provided. `training_job_definitions[].checkpoint_config` Contains information about the output location for managed spot training checkpoint data. Show child fields `training_job_definitions[].checkpoint_config.local_path` (Optional) The local directory where checkpoints are written. The default directory is `/opt/ml/checkpoints/`. `training_job_definitions[].checkpoint_config.s3_uri` Identifies the S3 path where you want SageMaker to store checkpoints. For example, `s3://bucket-name/key-name-prefix`. `training_job_definitions[].definition_name` The job definition name. `training_job_definitions[].enable_inter_container_traffic_encryption` To encrypt all communications between ML compute instances in distributed training, choose `True`. Encryption provides greater security for distributed training, but training might take longer. How long it takes depends on the amount of communication between compute instances, especially if you use a deep learning algorithm in distributed training. `training_job_definitions[].enable_managed_spot_training` A Boolean indicating whether managed spot training is enabled (`True`) or not (`False`). `training_job_definitions[].enable_network_isolation` Isolates the training container. No inbound or outbound network calls can be made, except for calls between peers within a training cluster for distributed training. If network isolation is used for training jobs that are configured to use a VPC, SageMaker downloads and uploads customer data and model artifacts through the specified VPC, but the training container does not have network access. `training_job_definitions[].environment` An environment variable that you can pass into the SageMaker CreateTrainingJob API. You can use an existing environment variable from the training container or use your own. See Define metrics and variables for more information. The maximum number of items specified for `Map Entries` refers to the maximum number of environment variables for each `TrainingJobDefinition` and also the maximum for the hyperparameter tuning job itself. That is, the sum of the number of environment variables for all the training job definitions can't exceed the maximum number specified. `training_job_definitions[].hyper_parameter_ranges` Specifies ranges of integer, continuous, and categorical hyperparameters that a hyperparameter tuning job searches. The hyperparameter tuning job launches training jobs with hyperparameter values within these ranges to find the combination of values that result in the training job with the best performance as measured by the objective metric of the hyperparameter tuning job. The maximum number of items specified for `Array Members` refers to the maximum number of hyperparameters for each range and also the maximum for the hyperparameter tuning job itself. That is, the sum of the number of hyperparameters for all the ranges can't exceed the maximum number specified. Show child fields `training_job_definitions[].hyper_parameter_ranges.auto_parameters[]` Show child fields `training_job_definitions[].hyper_parameter_ranges.auto_parameters[].name` The name of the hyperparameter to optimize using Autotune. `training_job_definitions[].hyper_parameter_ranges.auto_parameters[].value_hint` An example value of the hyperparameter to optimize using Autotune. `training_job_definitions[].hyper_parameter_ranges.categorical_parameter_ranges[]` Show child fields `training_job_definitions[].hyper_parameter_ranges.categorical_parameter_ranges[].name` The name of the categorical hyperparameter to tune. `training_job_definitions[].hyper_parameter_ranges.categorical_parameter_ranges[].values[]` `training_job_definitions[].hyper_parameter_ranges.continuous_parameter_ranges[]` Show child fields `training_job_definitions[].hyper_parameter_ranges.continuous_parameter_ranges[].max_value` The maximum value for the hyperparameter. The tuning job uses floating-point values between `MinValue` value and this value for tuning. `training_job_definitions[].hyper_parameter_ranges.continuous_parameter_ranges[].min_value` The minimum value for the hyperparameter. The tuning job uses floating-point values between this value and `MaxValue`for tuning. `training_job_definitions[].hyper_parameter_ranges.continuous_parameter_ranges[].name` The name of the continuous hyperparameter to tune. `training_job_definitions[].hyper_parameter_ranges.continuous_parameter_ranges[].scaling_type` The scale that hyperparameter tuning uses to search the hyperparameter range. For information about choosing a hyperparameter scale, see Hyperparameter Scaling. One of the following values: Auto SageMaker hyperparameter tuning chooses the best scale for the hyperparameter. Linear Hyperparameter tuning searches the values in the hyperparameter range by using a linear scale. Logarithmic Hyperparameter tuning searches the values in the hyperparameter range by using a logarithmic scale. Logarithmic scaling works only for ranges that have only values greater than 0. ReverseLogarithmic Hyperparameter tuning searches the values in the hyperparameter range by using a reverse logarithmic scale. Reverse logarithmic scaling works only for ranges that are entirely within the range 0<=x<1.0. `training_job_definitions[].hyper_parameter_ranges.integer_parameter_ranges[]` Show child fields `training_job_definitions[].hyper_parameter_ranges.integer_parameter_ranges[].max_value` The maximum value of the hyperparameter to search. `training_job_definitions[].hyper_parameter_ranges.integer_parameter_ranges[].min_value` The minimum value of the hyperparameter to search. `training_job_definitions[].hyper_parameter_ranges.integer_parameter_ranges[].name` The name of the hyperparameter to search. `training_job_definitions[].hyper_parameter_ranges.integer_parameter_ranges[].scaling_type` The scale that hyperparameter tuning uses to search the hyperparameter range. For information about choosing a hyperparameter scale, see Hyperparameter Scaling. One of the following values: Auto SageMaker hyperparameter tuning chooses the best scale for the hyperparameter. Linear Hyperparameter tuning searches the values in the hyperparameter range by using a linear scale. Logarithmic Hyperparameter tuning searches the values in the hyperparameter range by using a logarithmic scale. Logarithmic scaling works only for ranges that have only values greater than 0. `training_job_definitions[].hyper_parameter_tuning_resource_config` The configuration for the hyperparameter tuning resources, including the compute instances and storage volumes, used for training jobs launched by the tuning job. By default, storage volumes hold model artifacts and incremental states. Choose `File` for `TrainingInputMode` in the `AlgorithmSpecification` parameter to additionally store training data in the storage volume (optional). Show child fields `training_job_definitions[].hyper_parameter_tuning_resource_config.allocation_strategy` The strategy that determines the order of preference for resources specified in `InstanceConfigs` used in hyperparameter optimization. `training_job_definitions[].hyper_parameter_tuning_resource_config.instance_configs[]` Show child fields `training_job_definitions[].hyper_parameter_tuning_resource_config.instance_configs[].instance_count` The number of instances of the type specified by `InstanceType`. Choose an instance count larger than 1 for distributed training algorithms. See Step 2: Launch a SageMaker Distributed Training Job Using the SageMaker Python SDK for more information. `training_job_definitions[].hyper_parameter_tuning_resource_config.instance_configs[].instance_type` The instance type used for processing of hyperparameter optimization jobs. Choose from general purpose (no GPUs) instance types: ml.m5.xlarge, ml.m5.2xlarge, and ml.m5.4xlarge or compute optimized (no GPUs) instance types: ml.c5.xlarge and ml.c5.2xlarge. For more information about instance types, see instance type descriptions. `training_job_definitions[].hyper_parameter_tuning_resource_config.instance_configs[].volume_size_in_gb` The volume size in GB of the data to be processed for hyperparameter optimization (optional). `training_job_definitions[].hyper_parameter_tuning_resource_config.instance_count` The number of compute instances of type `InstanceType` to use. For distributed training, select a value greater than 1. `training_job_definitions[].hyper_parameter_tuning_resource_config.instance_type` The instance type used to run hyperparameter optimization tuning jobs. See descriptions of instance types for more information. `training_job_definitions[].hyper_parameter_tuning_resource_config.volume_kms_key_id` A key used by Amazon Web Services Key Management Service to encrypt data on the storage volume attached to the compute instances used to run the training job. You can use either of the following formats to specify a key. KMS Key ID: `"1234abcd-12ab-34cd-56ef-1234567890ab"` Amazon Resource Name (ARN) of a KMS key: `"arn:aws:kms:us-west-2:111122223333:key/1234abcd-12ab-34cd-56ef-1234567890ab"` Some instances use local storage, which use a hardware module to encrypt storage volumes. If you choose one of these instance types, you cannot request a `VolumeKmsKeyId`. For a list of instance types that use local storage, see instance store volumes. For more information about Amazon Web Services Key Management Service, see KMS encryption for more information. `training_job_definitions[].hyper_parameter_tuning_resource_config.volume_size_in_gb` The volume size in GB for the storage volume to be used in processing hyperparameter optimization jobs (optional). These volumes store model artifacts, incremental states and optionally, scratch space for training algorithms. Do not provide a value for this parameter if a value for `InstanceConfigs` is also specified. Some instance types have a fixed total local storage size. If you select one of these instances for training, `VolumeSizeInGB` cannot be greater than this total size. For a list of instance types with local instance storage and their sizes, see instance store volumes. SageMaker supports only the General Purpose SSD (gp2) storage volume type. `training_job_definitions[].input_data_config[]` Show child fields `training_job_definitions[].input_data_config[].channel_name` The name of the channel. `training_job_definitions[].input_data_config[].compression_type` If training data is compressed, the compression type. The default value is `None`. `CompressionType` is used only in Pipe input mode. In File mode, leave this field unset or set it to None. `training_job_definitions[].input_data_config[].content_type` The MIME type of the data. `training_job_definitions[].input_data_config[].data_source` The location of the channel data. Show child fields `training_job_definitions[].input_data_config[].data_source.file_system_data_source` The file system that is associated with a channel. Show child fields `training_job_definitions[].input_data_config[].data_source.file_system_data_source.directory_path` The full path to the directory to associate with the channel. `training_job_definitions[].input_data_config[].data_source.file_system_data_source.file_system_access_mode` The access mode of the mount of the directory associated with the channel. A directory can be mounted either in `ro` (read-only) or `rw` (read-write) mode. `training_job_definitions[].input_data_config[].data_source.file_system_data_source.file_system_id` The file system id. `training_job_definitions[].input_data_config[].data_source.file_system_data_source.file_system_type` The file system type. `training_job_definitions[].input_data_config[].data_source.s3_data_source` The S3 location of the data source that is associated with a channel. Show child fields `training_job_definitions[].input_data_config[].data_source.s3_data_source.attribute_names[]` `training_job_definitions[].input_data_config[].data_source.s3_data_source.instance_group_names[]` `training_job_definitions[].input_data_config[].data_source.s3_data_source.s3_data_distribution_type` If you want SageMaker to replicate the entire dataset on each ML compute instance that is launched for model training, specify `FullyReplicated`. If you want SageMaker to replicate a subset of data on each ML compute instance that is launched for model training, specify `ShardedByS3Key`. If there are n ML compute instances launched for a training job, each instance gets approximately 1/n of the number of S3 objects. In this case, model training on each machine uses only the subset of training data. Don't choose more ML compute instances for training than available S3 objects. If you do, some nodes won't get any data and you will pay for nodes that aren't getting any training data. This applies in both File and Pipe modes. Keep this in mind when developing algorithms. In distributed training, where you use multiple ML compute EC2 instances, you might choose `ShardedByS3Key`. If the algorithm requires copying training data to the ML storage volume (when `TrainingInputMode` is set to `File`), this copies 1/n of the number of objects. `training_job_definitions[].input_data_config[].data_source.s3_data_source.s3_data_type` If you choose `S3Prefix`, `S3Uri` identifies a key name prefix. SageMaker uses all objects that match the specified key name prefix for model training. If you choose `ManifestFile`, `S3Uri` identifies an object that is a manifest file containing a list of object keys that you want SageMaker to use for model training. If you choose `AugmentedManifestFile`, S3Uri identifies an object that is an augmented manifest file in JSON lines format. This file contains the data you want to use for model training. `AugmentedManifestFile` can only be used if the Channel's input mode is `Pipe`. `training_job_definitions[].input_data_config[].data_source.s3_data_source.s3_uri` Depending on the value specified for the `S3DataType`, identifies either a key name prefix or a manifest. For example: A key name prefix might look like this: `s3://bucketname/exampleprefix/` A manifest might look like this: `s3://bucketname/example.manifest` A manifest is an S3 object which is a JSON file consisting of an array of elements. The first element is a prefix which is followed by one or more suffixes. SageMaker appends the suffix elements to the prefix to get a full set of `S3Uri`. Note that the prefix must be a valid non-empty `S3Uri` that precludes users from specifying a manifest whose individual `S3Uri` is sourced from different S3 buckets. The following code example shows a valid manifest format: `[ {"prefix": "s3://customer_bucket/some/prefix/"},` `"relative/path/to/custdata-1",` `"relative/path/custdata-2",` `...` `"relative/path/custdata-N"` `]` This JSON is equivalent to the following `S3Uri` list: `s3://customer_bucket/some/prefix/relative/path/to/custdata-1` `s3://customer_bucket/some/prefix/relative/path/custdata-2` `...` `s3://customer_bucket/some/prefix/relative/path/custdata-N` The complete set of `S3Uri` in this manifest is the input data for the channel for this data source. The object that each `S3Uri` points to must be readable by the IAM role that SageMaker uses to perform tasks on your behalf. Your input bucket must be located in same Amazon Web Services region as your training job. `training_job_definitions[].input_data_config[].input_mode` (Optional) The input mode to use for the data channel in a training job. If you don't set a value for `InputMode`, SageMaker uses the value set for `TrainingInputMode`. Use this parameter to override the `TrainingInputMode` setting in a AlgorithmSpecification request when you have a channel that needs a different input mode from the training job's general setting. To download the data from Amazon Simple Storage Service (Amazon S3) to the provisioned ML storage volume, and mount the directory to a Docker volume, use `File` input mode. To stream data directly from Amazon S3 to the container, choose `Pipe` input mode. To use a model for incremental training, choose `File` input model. `training_job_definitions[].input_data_config[].record_wrapper_type` Specify RecordIO as the value when input data is in raw format but the training algorithm requires the RecordIO format. In this case, SageMaker wraps each individual S3 object in a RecordIO record. If the input data is already in RecordIO format, you don't need to set this attribute. For more information, see Create a Dataset Using RecordIO. In File mode, leave this field unset or set it to None. `training_job_definitions[].input_data_config[].shuffle_config` A configuration for a shuffle option for input data in a channel. If you use `S3Prefix` for `S3DataType`, this shuffles the results of the S3 key prefix matches. If you use `ManifestFile`, the order of the S3 object references in the `ManifestFile` is shuffled. If you use `AugmentedManifestFile`, the order of the JSON lines in the `AugmentedManifestFile` is shuffled. The shuffling order is determined using the `Seed` value. For Pipe input mode, shuffling is done at the start of every epoch. With large datasets this ensures that the order of the training data is different for each epoch, it helps reduce bias and possible overfitting. In a multi-node training job when ShuffleConfig is combined with `S3DataDistributionType` of `ShardedByS3Key`, the data is shuffled across nodes so that the content sent to a particular node on the first epoch might be sent to a different node on the second epoch. Show child fields `training_job_definitions[].input_data_config[].shuffle_config.seed` Determines the shuffling order in `ShuffleConfig` value. `training_job_definitions[].output_data_config` Specifies the path to the Amazon S3 bucket where you store model artifacts from the training jobs that the tuning job launches. Show child fields `training_job_definitions[].output_data_config.compression_type` The model output compression type. Select `None` to output an uncompressed model, recommended for large model outputs. Defaults to gzip. `training_job_definitions[].output_data_config.kms_key_id` The Amazon Web Services Key Management Service (Amazon Web Services KMS) key that SageMaker uses to encrypt the model artifacts at rest using Amazon S3 server-side encryption. The `KmsKeyId` can be any of the following formats: // KMS Key ID `"1234abcd-12ab-34cd-56ef-1234567890ab"` // Amazon Resource Name (ARN) of a KMS Key `"arn:aws:kms:us-west-2:111122223333:key/1234abcd-12ab-34cd-56ef-1234567890ab"` // KMS Key Alias `"alias/ExampleAlias"` // Amazon Resource Name (ARN) of a KMS Key Alias `"arn:aws:kms:us-west-2:111122223333:alias/ExampleAlias"` If you use a KMS key ID or an alias of your KMS key, the SageMaker execution role must include permissions to call `kms:Encrypt`. If you don't provide a KMS key ID, SageMaker uses the default KMS key for Amazon S3 for your role's account. For more information, see KMS-Managed Encryption Keys in the Amazon Simple Storage Service Developer Guide. If the output data is stored in Amazon S3 Express One Zone, it is encrypted with server-side encryption with Amazon S3 managed keys (SSE-S3). KMS key is not supported for Amazon S3 Express One Zone The KMS key policy must grant permission to the IAM role that you specify in your `CreateTrainingJob`, `CreateTransformJob`, or `CreateHyperParameterTuningJob` requests. For more information, see Using Key Policies in Amazon Web Services KMS in the Amazon Web Services Key Management Service Developer Guide. `training_job_definitions[].output_data_config.s3_output_path` Identifies the S3 path where you want SageMaker to store the model artifacts. For example, `s3://bucket-name/key-name-prefix`. `training_job_definitions[].resource_config` The resources, including the compute instances and storage volumes, to use for the training jobs that the tuning job launches. Storage volumes store model artifacts and incremental states. Training algorithms might also use storage volumes for scratch space. If you want SageMaker to use the storage volume to store the training data, choose `File` as the `TrainingInputMode` in the algorithm specification. For distributed training algorithms, specify an instance count greater than 1. If you want to use hyperparameter optimization with instance type flexibility, use `HyperParameterTuningResourceConfig` instead. Show child fields `training_job_definitions[].resource_config.instance_count` The number of ML compute instances to use. For distributed training, provide a value greater than 1. `training_job_definitions[].resource_config.instance_groups[]` Show child fields `training_job_definitions[].resource_config.instance_groups[].instance_count` Specifies the number of instances of the instance group. `training_job_definitions[].resource_config.instance_groups[].instance_group_name` Specifies the name of the instance group. `training_job_definitions[].resource_config.instance_groups[].instance_type` Specifies the instance type of the instance group. `training_job_definitions[].resource_config.instance_type` The ML compute instance type. SageMaker Training on Amazon Elastic Compute Cloud (EC2) P4de instances is in preview release starting December 9th, 2022. Amazon EC2 P4de instances (currently in preview) are powered by 8 NVIDIA A100 GPUs with 80GB high-performance HBM2e GPU memory, which accelerate the speed of training ML models that need to be trained on large datasets of high-resolution data. In this preview release, Amazon SageMaker supports ML training jobs on P4de instances (`ml.p4de.24xlarge`) to reduce model training time. The `ml.p4de.24xlarge` instances are available in the following Amazon Web Services Regions. US East (N. Virginia) (us-east-1) US West (Oregon) (us-west-2) To request quota limit increase and start using P4de instances, contact the SageMaker Training service team through your account team. `training_job_definitions[].resource_config.keep_alive_period_in_seconds` The duration of time in seconds to retain configured resources in a warm pool for subsequent training jobs. `training_job_definitions[].resource_config.volume_kms_key_id` The Amazon Web Services KMS key that SageMaker uses to encrypt data on the storage volume attached to the ML compute instance(s) that run the training job. Certain Nitro-based instances include local storage, dependent on the instance type. Local storage volumes are encrypted using a hardware module on the instance. You can't request a `VolumeKmsKeyId` when using an instance type with local storage. For a list of instance types that support local instance storage, see Instance Store Volumes. For more information about local instance storage encryption, see SSD Instance Store Volumes. The `VolumeKmsKeyId` can be in any of the following formats: // KMS Key ID `"1234abcd-12ab-34cd-56ef-1234567890ab"` // Amazon Resource Name (ARN) of a KMS Key `"arn:aws:kms:us-west-2:111122223333:key/1234abcd-12ab-34cd-56ef-1234567890ab"` `training_job_definitions[].resource_config.volume_size_in_gb` The size of the ML storage volume that you want to provision. ML storage volumes store model artifacts and incremental states. Training algorithms might also use the ML storage volume for scratch space. If you want to store the training data in the ML storage volume, choose `File` as the `TrainingInputMode` in the algorithm specification. When using an ML instance with NVMe SSD volumes, SageMaker doesn't provision Amazon EBS General Purpose SSD (gp2) storage. Available storage is fixed to the NVMe-type instance's storage capacity. SageMaker configures storage paths for training datasets, checkpoints, model artifacts, and outputs to use the entire capacity of the instance storage. For example, ML instance families with the NVMe-type instance storage include `ml.p4d`, `ml.g4dn`, and `ml.g5`. When using an ML instance with the EBS-only storage option and without instance storage, you must define the size of EBS volume through `VolumeSizeInGB` in the `ResourceConfig` API. For example, ML instance families that use EBS volumes include `ml.c5` and `ml.p2`. To look up instance types and their instance storage types and volumes, see Amazon EC2 Instance Types. To find the default local paths defined by the SageMaker training platform, see Amazon SageMaker Training Storage Folders for Training Datasets, Checkpoints, Model Artifacts, and Outputs. `training_job_definitions[].retry_strategy` The number of times to retry the job when the job fails due to an `InternalServerError`. Show child fields `training_job_definitions[].retry_strategy.maximum_retry_attempts` The number of times to retry the job. When the job is retried, it's `SecondaryStatus` is changed to `STARTING`. `training_job_definitions[].role_arn` The Amazon Resource Name (ARN) of the IAM role associated with the training jobs that the tuning job launches. `training_job_definitions[].static_hyper_parameters` Specifies the values of hyperparameters that do not change for the tuning job. `training_job_definitions[].stopping_condition` Specifies a limit to how long a model hyperparameter training job can run. It also specifies how long a managed spot training job has to complete. When the job reaches the time limit, SageMaker ends the training job. Use this API to cap model training costs. Show child fields `training_job_definitions[].stopping_condition.max_pending_time_in_seconds` The maximum length of time, in seconds, that a training or compilation job can be pending before it is stopped. `training_job_definitions[].stopping_condition.max_runtime_in_seconds` The maximum length of time, in seconds, that a training or compilation job can run before it is stopped. For compilation jobs, if the job does not complete during this time, a `TimeOut` error is generated. We recommend starting with 900 seconds and increasing as necessary based on your model. For all other jobs, if the job does not complete during this time, SageMaker ends the job. When `RetryStrategy` is specified in the job request, `MaxRuntimeInSeconds` specifies the maximum time for all of the attempts in total, not each individual attempt. The default value is 1 day. The maximum value is 28 days. The maximum time that a `TrainingJob` can run in total, including any time spent publishing metrics or archiving and uploading models after it has been stopped, is 30 days. `training_job_definitions[].stopping_condition.max_wait_time_in_seconds` The maximum length of time, in seconds, that a managed Spot training job has to complete. It is the amount of time spent waiting for Spot capacity plus the amount of time the job can run. It must be equal to or greater than `MaxRuntimeInSeconds`. If the job does not complete during this time, SageMaker ends the job. When `RetryStrategy` is specified in the job request, `MaxWaitTimeInSeconds` specifies the maximum time for all of the attempts in total, not each individual attempt. `training_job_definitions[].tuning_objective` Defines the objective metric for a hyperparameter tuning job. Hyperparameter tuning uses the value of this metric to evaluate the training jobs it launches, and returns the training job that results in either the highest or lowest value for this metric, depending on the value you specify for the `Type` parameter. If you want to define a custom objective metric, see Define metrics and environment variables. Show child fields `training_job_definitions[].tuning_objective.metric_name` The name of the metric to use for the objective metric. `training_job_definitions[].tuning_objective.type` Whether to minimize or maximize the objective metric. `training_job_definitions[].vpc_config` The VpcConfig object that specifies the VPC that you want the training jobs that this hyperparameter tuning job launches to connect to. Control access to and from your training container by configuring the VPC. For more information, see Protect Training Jobs by Using an Amazon Virtual Private Cloud. Show child fields `training_job_definitions[].vpc_config.security_group_ids[]` `training_job_definitions[].vpc_config.subnets[]`
`training_job_status_counters` The TrainingJobStatusCounters object that specifies the number of training jobs, categorized by status, that this tuning job launched.	STRUCT( "completed" BIGINT, "in_progress" BIGINT, "retryable_error" BIGINT, "non_retryable_error" BIGINT, "stopped" BIGINT )
Show child fields `training_job_status_counters.completed` The number of completed training jobs launched by the hyperparameter tuning job. `training_job_status_counters.in_progress` The number of in-progress training jobs launched by a hyperparameter tuning job. `training_job_status_counters.non_retryable_error` The number of training jobs that failed and can't be retried. A failed training job can't be retried if it failed because a client error occurred. `training_job_status_counters.retryable_error` The number of training jobs that failed, but can be retried. A failed training job can be retried only if it failed because an internal service error occurred. `training_job_status_counters.stopped` The number of training jobs launched by a hyperparameter tuning job that were manually stopped.
`tuning_job_completion_details` Tuning job completion information returned as the response from a hyperparameter tuning job. This information tells if your tuning job has or has not converged. It also includes the number of training jobs that have not improved model performance as evaluated against the objective function.	STRUCT( "number_of_training_jobs_objective_not_improving" BIGINT, "convergence_detected_time" TIMESTAMP_S )
Show child fields `tuning_job_completion_details.convergence_detected_time` The time in timestamp format that AMT detected model convergence, as defined by a lack of significant improvement over time based on criteria developed over a wide range of diverse benchmarking tests. `tuning_job_completion_details.number_of_training_jobs_objective_not_improving` The number of training jobs launched by a tuning job that are not improving (1% or less) as measured by model performance evaluated against an objective function.
`warm_start_config` The configuration for starting the hyperparameter parameter tuning job using one or more previous tuning jobs as a starting point. The results of previous tuning jobs are used to inform which combinations of hyperparameters to search over in the new tuning job.	STRUCT( "parent_hyper_parameter_tuning_jobs" STRUCT( "hyper_parameter_tuning_job_name" VARCHAR )[], "warm_start_type" VARCHAR )
Show child fields `warm_start_config.parent_hyper_parameter_tuning_jobs[]` Show child fields `warm_start_config.parent_hyper_parameter_tuning_jobs[].hyper_parameter_tuning_job_name` The name of the hyperparameter tuning job to be used as a starting point for a new hyperparameter tuning job. `warm_start_config.warm_start_type` Specifies one of the following: IDENTICAL_DATA_AND_ALGORITHM The new hyperparameter tuning job uses the same input data and training image as the parent tuning jobs. You can change the hyperparameter ranges to search and the maximum number of training jobs that the hyperparameter tuning job launches. You cannot use a new version of the training algorithm, unless the changes in the new version do not affect the algorithm itself. For example, changes that improve logging or adding support for a different data format are allowed. You can also change hyperparameters from tunable to static, and from static to tunable, but the total number of static plus tunable hyperparameters must remain the same as it is in all parent jobs. The objective metric for the new tuning job must be the same as for all parent jobs. TRANSFER_LEARNING The new hyperparameter tuning job can include input data, hyperparameter ranges, maximum number of concurrent training jobs, and maximum number of training jobs that are different than those of its parent hyperparameter tuning jobs. The training image can also be a different version from the version used in the parent hyperparameter tuning job. You can also change hyperparameters from tunable to static, and from static to tunable, but the total number of static plus tunable hyperparameters must remain the same as it is in all parent jobs. The objective metric for the new tuning job must be the same as for all parent jobs.