Skip to content
AWS for Query.Farm

aws.pipes.describe_pipe

Example SQL Queries

SELECT * FROM
aws.pipes.describe_pipe
WHERE
"name" = 'VALUE';

Description

Get the information about an existing pipe. For more information about pipes, see Amazon EventBridge Pipes in the Amazon EventBridge User Guide.

Table Definition

Column NameColumn Data Type
name Required Input Column

The name of the pipe.

VARCHAR
_aws_profile Input Column

The AWS profile defines the AWS identity used. It can be defined via credentials or by assuming a IAM role.

STRUCT(
    "type" VARCHAR,
    "name" VARCHAR,
    "account_id" VARCHAR,
    "via_profile_name" VARCHAR,
    "assumed_role_arn" VARCHAR,
    "organization" STRUCT(
        "account_name" VARCHAR,
        "id" VARCHAR,
        "tags" STRUCT(
            "key" VARCHAR,
            "value" VARCHAR
        )[],
        "master_account" STRUCT(
            "id" VARCHAR,
            "email" VARCHAR
        ),
        "parents" STRUCT(
            "type" VARCHAR,
            "id" VARCHAR,
            "name" VARCHAR,
            "tags" STRUCT(
                "key" VARCHAR,
                "value" VARCHAR
            )[]
        )[]
    )
)
Show child fields
_aws_profile.account_id

The AWS account id

_aws_profile.assumed_role_arn

The ARN of the assumed role

_aws_profile.name

The unique name of the profile.

_aws_profile.organization

Information about this profile's membership in the AWS organization.

Show child fields
_aws_profile.organization.account_name

The name of account speciifed by the organization

_aws_profile.organization.id

The organization id

_aws_profile.organization.master_account
Show child fields
_aws_profile.organization.master_account.email

The organization master account email address

_aws_profile.organization.master_account.id

The organization master account id

_aws_profile.organization.parents[]
Show child fields
_aws_profile.organization.parents[].id

The id of the parent

_aws_profile.organization.parents[].name

The name of the parent

_aws_profile.organization.parents[].tags[]
Show child fields
_aws_profile.organization.parents[].tags[].key
_aws_profile.organization.parents[].tags[].value
_aws_profile.organization.parents[].type

The type of parent can be an organization unit or a root

_aws_profile.organization.tags[]
Show child fields
_aws_profile.organization.tags[].key
_aws_profile.organization.tags[].value
_aws_profile.type

The type of profile, either 'credentials' or 'assumed_role'

_aws_profile.via_profile_name

This IAM role for this profile is assumed by first utilizing another profile with this name to obtain credentials.

_aws_region Input Column

The AWS region to use.

VARCHAR
arn

The ARN of the pipe.

VARCHAR
creation_time

The time the pipe was created.

TIMESTAMP_S
current_state

The state the pipe is in.

VARCHAR
description

A description of the pipe.

VARCHAR
desired_state

The state the pipe should be in.

VARCHAR
enrichment

The ARN of the enrichment resource.

VARCHAR
enrichment_parameters

The parameters required to set up enrichment on your pipe.

STRUCT(
    "input_template" VARCHAR,
    "http_parameters" STRUCT(
        "path_parameter_values" VARCHAR[],
        "header_parameters" MAP(VARCHAR, VARCHAR),
        "query_string_parameters" MAP(VARCHAR, VARCHAR)
    )
)
Show child fields
enrichment_parameters.http_parameters

Contains the HTTP parameters to use when the target is a API Gateway REST endpoint or EventBridge ApiDestination.

If you specify an API Gateway REST API or EventBridge ApiDestination as a target, you can use this parameter to specify headers, path parameters, and query string keys/values as part of your target invoking request. If you're using ApiDestinations, the corresponding Connection can also have these values configured. In case of any conflicting keys, values from the Connection take precedence.

Show child fields
enrichment_parameters.http_parameters.header_parameters

The headers that need to be sent as part of request invoking the API Gateway REST API or EventBridge ApiDestination.

enrichment_parameters.http_parameters.path_parameter_values[]
enrichment_parameters.http_parameters.query_string_parameters

The query string keys/values that need to be sent as part of request invoking the API Gateway REST API or EventBridge ApiDestination.

enrichment_parameters.input_template

Valid JSON text passed to the enrichment. In this case, nothing from the event itself is passed to the enrichment. For more information, see The JavaScript Object Notation (JSON) Data Interchange Format.

To remove an input template, specify an empty string.

last_modified_time

When the pipe was last updated, in ISO-8601 format (YYYY-MM-DDThh:mm:ss.sTZD).

TIMESTAMP_S
log_configuration

The logging configuration settings for the pipe.

STRUCT(
    "s3_log_destination" STRUCT(
        "bucket_name" VARCHAR,
        "prefix" VARCHAR,
        "bucket_owner" VARCHAR,
        "output_format" VARCHAR
    ),
    "firehose_log_destination" STRUCT(
        "delivery_stream_arn" VARCHAR
    ),
    "cloudwatch_logs_log_destination" STRUCT(
        "log_group_arn" VARCHAR
    ),
    "level" VARCHAR,
    "include_execution_data" VARCHAR[]
)
Show child fields
log_configuration.cloudwatch_logs_log_destination

The Amazon CloudWatch Logs logging configuration settings for the pipe.

Show child fields
log_configuration.cloudwatch_logs_log_destination.log_group_arn

The Amazon Web Services Resource Name (ARN) for the CloudWatch log group to which EventBridge sends the log records.

log_configuration.firehose_log_destination

The Amazon Data Firehose logging configuration settings for the pipe.

Show child fields
log_configuration.firehose_log_destination.delivery_stream_arn

The Amazon Resource Name (ARN) of the Firehose delivery stream to which EventBridge delivers the pipe log records.

log_configuration.include_execution_data[]
log_configuration.level

The level of logging detail to include. This applies to all log destinations for the pipe.

log_configuration.s3_log_destination

The Amazon S3 logging configuration settings for the pipe.

Show child fields
log_configuration.s3_log_destination.bucket_name

The name of the Amazon S3 bucket to which EventBridge delivers the log records for the pipe.

log_configuration.s3_log_destination.bucket_owner

The Amazon Web Services account that owns the Amazon S3 bucket to which EventBridge delivers the log records for the pipe.

log_configuration.s3_log_destination.output_format

The format EventBridge uses for the log records.

log_configuration.s3_log_destination.prefix

The prefix text with which to begin Amazon S3 log object names.

For more information, see Organizing objects using prefixes in the Amazon Simple Storage Service User Guide.

role_arn

The ARN of the role that allows the pipe to send data to the target.

VARCHAR
source

The ARN of the source resource.

VARCHAR
source_parameters

The parameters required to set up a source for your pipe.

STRUCT(
    "filter_criteria" STRUCT(
        "filters" STRUCT(
            "pattern" VARCHAR
        )[]
    ),
    "kinesis_stream_parameters" STRUCT(
        "batch_size" BIGINT,
        "dead_letter_config" STRUCT(
            "arn" VARCHAR
        ),
        "on_partial_batch_item_failure" VARCHAR,
        "maximum_batching_window_in_seconds" BIGINT,
        "maximum_record_age_in_seconds" BIGINT,
        "maximum_retry_attempts" BIGINT,
        "parallelization_factor" BIGINT,
        "starting_position" VARCHAR,
        "starting_position_timestamp" TIMESTAMP_S
    ),
    "dynamo_db_stream_parameters" STRUCT(
        "batch_size" BIGINT,
        "dead_letter_config" STRUCT(
            "arn" VARCHAR
        ),
        "on_partial_batch_item_failure" VARCHAR,
        "maximum_batching_window_in_seconds" BIGINT,
        "maximum_record_age_in_seconds" BIGINT,
        "maximum_retry_attempts" BIGINT,
        "parallelization_factor" BIGINT,
        "starting_position" VARCHAR
    ),
    "sqs_queue_parameters" STRUCT(
        "batch_size" BIGINT,
        "maximum_batching_window_in_seconds" BIGINT
    ),
    "active_mq_broker_parameters" STRUCT(
        "credentials" STRUCT(
            "basic_auth" VARCHAR
        ),
        "queue_name" VARCHAR,
        "batch_size" BIGINT,
        "maximum_batching_window_in_seconds" BIGINT
    ),
    "rabbit_mq_broker_parameters" STRUCT(
        "credentials" STRUCT(
            "basic_auth" VARCHAR
        ),
        "queue_name" VARCHAR,
        "virtual_host" VARCHAR,
        "batch_size" BIGINT,
        "maximum_batching_window_in_seconds" BIGINT
    ),
    "managed_streaming_kafka_parameters" STRUCT(
        "topic_name" VARCHAR,
        "starting_position" VARCHAR,
        "batch_size" BIGINT,
        "maximum_batching_window_in_seconds" BIGINT,
        "consumer_group_id" VARCHAR,
        "credentials" STRUCT(
            "sasl_scram512_auth" VARCHAR,
            "client_certificate_tls_auth" VARCHAR
        )
    ),
    "self_managed_kafka_parameters" STRUCT(
        "topic_name" VARCHAR,
        "starting_position" VARCHAR,
        "additional_bootstrap_servers" VARCHAR[],
        "batch_size" BIGINT,
        "maximum_batching_window_in_seconds" BIGINT,
        "consumer_group_id" VARCHAR,
        "credentials" STRUCT(
            "basic_auth" VARCHAR,
            "sasl_scram512_auth" VARCHAR,
            "sasl_scram256_auth" VARCHAR,
            "client_certificate_tls_auth" VARCHAR
        ),
        "server_root_ca_certificate" VARCHAR,
        "vpc" STRUCT(
            "subnets" VARCHAR[],
            "security_group" VARCHAR[]
        )
    )
)
Show child fields
source_parameters.active_mq_broker_parameters

The parameters for using an Active MQ broker as a source.

Show child fields
source_parameters.active_mq_broker_parameters.batch_size

The maximum number of records to include in each batch.

source_parameters.active_mq_broker_parameters.credentials

The credentials needed to access the resource.

Show child fields
source_parameters.active_mq_broker_parameters.credentials.basic_auth

The ARN of the Secrets Manager secret.

source_parameters.active_mq_broker_parameters.maximum_batching_window_in_seconds

The maximum length of a time to wait for events.

source_parameters.active_mq_broker_parameters.queue_name

The name of the destination queue to consume.

source_parameters.dynamo_db_stream_parameters

The parameters for using a DynamoDB stream as a source.

Show child fields
source_parameters.dynamo_db_stream_parameters.batch_size

The maximum number of records to include in each batch.

source_parameters.dynamo_db_stream_parameters.dead_letter_config

Define the target queue to send dead-letter queue events to.

Show child fields
source_parameters.dynamo_db_stream_parameters.dead_letter_config.arn

The ARN of the specified target for the dead-letter queue.

For Amazon Kinesis stream and Amazon DynamoDB stream sources, specify either an Amazon SNS topic or Amazon SQS queue ARN.

source_parameters.dynamo_db_stream_parameters.maximum_batching_window_in_seconds

The maximum length of a time to wait for events.

source_parameters.dynamo_db_stream_parameters.maximum_record_age_in_seconds

(Streams only) Discard records older than the specified age. The default value is -1, which sets the maximum age to infinite. When the value is set to infinite, EventBridge never discards old records.

source_parameters.dynamo_db_stream_parameters.maximum_retry_attempts

(Streams only) Discard records after the specified number of retries. The default value is -1, which sets the maximum number of retries to infinite. When MaximumRetryAttempts is infinite, EventBridge retries failed records until the record expires in the event source.

source_parameters.dynamo_db_stream_parameters.on_partial_batch_item_failure

(Streams only) Define how to handle item process failures. AUTOMATIC_BISECT halves each batch and retry each half until all the records are processed or there is one failed message left in the batch.

source_parameters.dynamo_db_stream_parameters.parallelization_factor

(Streams only) The number of batches to process concurrently from each shard. The default value is 1.

source_parameters.dynamo_db_stream_parameters.starting_position

(Streams only) The position in a stream from which to start reading.

source_parameters.filter_criteria

The collection of event patterns used to filter events.

To remove a filter, specify a FilterCriteria object with an empty array of Filter objects.

For more information, see Events and Event Patterns in the Amazon EventBridge User Guide.

Show child fields
source_parameters.filter_criteria.filters[]
Show child fields
source_parameters.filter_criteria.filters[].pattern

The event pattern.

source_parameters.kinesis_stream_parameters

The parameters for using a Kinesis stream as a source.

Show child fields
source_parameters.kinesis_stream_parameters.batch_size

The maximum number of records to include in each batch.

source_parameters.kinesis_stream_parameters.dead_letter_config

Define the target queue to send dead-letter queue events to.

Show child fields
source_parameters.kinesis_stream_parameters.dead_letter_config.arn

The ARN of the specified target for the dead-letter queue.

For Amazon Kinesis stream and Amazon DynamoDB stream sources, specify either an Amazon SNS topic or Amazon SQS queue ARN.

source_parameters.kinesis_stream_parameters.maximum_batching_window_in_seconds

The maximum length of a time to wait for events.

source_parameters.kinesis_stream_parameters.maximum_record_age_in_seconds

(Streams only) Discard records older than the specified age. The default value is -1, which sets the maximum age to infinite. When the value is set to infinite, EventBridge never discards old records.

source_parameters.kinesis_stream_parameters.maximum_retry_attempts

(Streams only) Discard records after the specified number of retries. The default value is -1, which sets the maximum number of retries to infinite. When MaximumRetryAttempts is infinite, EventBridge retries failed records until the record expires in the event source.

source_parameters.kinesis_stream_parameters.on_partial_batch_item_failure

(Streams only) Define how to handle item process failures. AUTOMATIC_BISECT halves each batch and retry each half until all the records are processed or there is one failed message left in the batch.

source_parameters.kinesis_stream_parameters.parallelization_factor

(Streams only) The number of batches to process concurrently from each shard. The default value is 1.

source_parameters.kinesis_stream_parameters.starting_position

(Streams only) The position in a stream from which to start reading.

source_parameters.kinesis_stream_parameters.starting_position_timestamp

With StartingPosition set to AT_TIMESTAMP, the time from which to start reading, in Unix time seconds.

source_parameters.managed_streaming_kafka_parameters

The parameters for using an MSK stream as a source.

Show child fields
source_parameters.managed_streaming_kafka_parameters.batch_size

The maximum number of records to include in each batch.

source_parameters.managed_streaming_kafka_parameters.consumer_group_id

The name of the destination queue to consume.

source_parameters.managed_streaming_kafka_parameters.credentials

The credentials needed to access the resource.

Show child fields
source_parameters.managed_streaming_kafka_parameters.credentials.client_certificate_tls_auth

The ARN of the Secrets Manager secret.

source_parameters.managed_streaming_kafka_parameters.credentials.sasl_scram512_auth

The ARN of the Secrets Manager secret.

source_parameters.managed_streaming_kafka_parameters.maximum_batching_window_in_seconds

The maximum length of a time to wait for events.

source_parameters.managed_streaming_kafka_parameters.starting_position

(Streams only) The position in a stream from which to start reading.

source_parameters.managed_streaming_kafka_parameters.topic_name

The name of the topic that the pipe will read from.

source_parameters.rabbit_mq_broker_parameters

The parameters for using a Rabbit MQ broker as a source.

Show child fields
source_parameters.rabbit_mq_broker_parameters.batch_size

The maximum number of records to include in each batch.

source_parameters.rabbit_mq_broker_parameters.credentials

The credentials needed to access the resource.

Show child fields
source_parameters.rabbit_mq_broker_parameters.credentials.basic_auth

The ARN of the Secrets Manager secret.

source_parameters.rabbit_mq_broker_parameters.maximum_batching_window_in_seconds

The maximum length of a time to wait for events.

source_parameters.rabbit_mq_broker_parameters.queue_name

The name of the destination queue to consume.

source_parameters.rabbit_mq_broker_parameters.virtual_host

The name of the virtual host associated with the source broker.

source_parameters.self_managed_kafka_parameters

The parameters for using a self-managed Apache Kafka stream as a source.

A self managed cluster refers to any Apache Kafka cluster not hosted by Amazon Web Services. This includes both clusters you manage yourself, as well as those hosted by a third-party provider, such as Confluent Cloud, CloudKarafka, or Redpanda. For more information, see Apache Kafka streams as a source in the Amazon EventBridge User Guide.

Show child fields
source_parameters.self_managed_kafka_parameters.additional_bootstrap_servers[]
source_parameters.self_managed_kafka_parameters.batch_size

The maximum number of records to include in each batch.

source_parameters.self_managed_kafka_parameters.consumer_group_id

The name of the destination queue to consume.

source_parameters.self_managed_kafka_parameters.credentials

The credentials needed to access the resource.

Show child fields
source_parameters.self_managed_kafka_parameters.credentials.basic_auth

The ARN of the Secrets Manager secret.

source_parameters.self_managed_kafka_parameters.credentials.client_certificate_tls_auth

The ARN of the Secrets Manager secret.

source_parameters.self_managed_kafka_parameters.credentials.sasl_scram256_auth

The ARN of the Secrets Manager secret.

source_parameters.self_managed_kafka_parameters.credentials.sasl_scram512_auth

The ARN of the Secrets Manager secret.

source_parameters.self_managed_kafka_parameters.maximum_batching_window_in_seconds

The maximum length of a time to wait for events.

source_parameters.self_managed_kafka_parameters.server_root_ca_certificate

The ARN of the Secrets Manager secret used for certification.

source_parameters.self_managed_kafka_parameters.starting_position

(Streams only) The position in a stream from which to start reading.

source_parameters.self_managed_kafka_parameters.topic_name

The name of the topic that the pipe will read from.

source_parameters.self_managed_kafka_parameters.vpc

This structure specifies the VPC subnets and security groups for the stream, and whether a public IP address is to be used.

Show child fields
source_parameters.self_managed_kafka_parameters.vpc.security_group[]
source_parameters.self_managed_kafka_parameters.vpc.subnets[]
source_parameters.sqs_queue_parameters

The parameters for using a Amazon SQS stream as a source.

Show child fields
source_parameters.sqs_queue_parameters.batch_size

The maximum number of records to include in each batch.

source_parameters.sqs_queue_parameters.maximum_batching_window_in_seconds

The maximum length of a time to wait for events.

state_reason

The reason the pipe is in its current state.

VARCHAR
tags

The list of key-value pairs to associate with the pipe.

MAP(VARCHAR, VARCHAR)
target

The ARN of the target resource.

VARCHAR
target_parameters

The parameters required to set up a target for your pipe.

For more information about pipe target parameters, including how to use dynamic path parameters, see Target parameters in the Amazon EventBridge User Guide.

STRUCT(
    "input_template" VARCHAR,
    "lambda_function_parameters" STRUCT(
        "invocation_type" VARCHAR
    ),
    "step_function_state_machine_parameters" STRUCT(
        "invocation_type" VARCHAR
    ),
    "kinesis_stream_parameters" STRUCT(
        "partition_key" VARCHAR
    ),
    "ecs_task_parameters" STRUCT(
        "task_definition_arn" VARCHAR,
        "task_count" BIGINT,
        "launch_type" VARCHAR,
        "network_configuration" STRUCT(
            "awsvpc_configuration" STRUCT(
                "subnets" VARCHAR[],
                "security_groups" VARCHAR[],
                "assign_public_ip" VARCHAR
            )
        ),
        "platform_version" VARCHAR,
        "group" VARCHAR,
        "capacity_provider_strategy" STRUCT(
            "capacity_provider" VARCHAR,
            "weight" BIGINT,
            "base" BIGINT
        )[],
        "enable_ecs_managed_tags" BOOLEAN,
        "enable_execute_command" BOOLEAN,
        "placement_constraints" STRUCT(
            "type" VARCHAR,
            "expression" VARCHAR
        )[],
        "placement_strategy" STRUCT(
            "type" VARCHAR,
            "field" VARCHAR
        )[],
        "propagate_tags" VARCHAR,
        "reference_id" VARCHAR,
        "overrides" STRUCT(
            "container_overrides" STRUCT(
                "command" VARCHAR[],
                "cpu" BIGINT,
                "environment" STRUCT(
                    "name" VARCHAR,
                    "value" VARCHAR
                )[],
                "environment_files" STRUCT(
                    "type" VARCHAR,
                    "value" VARCHAR
                )[],
                "memory" BIGINT,
                "memory_reservation" BIGINT,
                "name" VARCHAR,
                "resource_requirements" STRUCT(
                    "type" VARCHAR,
                    "value" VARCHAR
                )[]
            )[],
            "cpu" VARCHAR,
            "ephemeral_storage" STRUCT(
                "size_in_gi_b" BIGINT
            ),
            "execution_role_arn" VARCHAR,
            "inference_accelerator_overrides" STRUCT(
                "device_name" VARCHAR,
                "device_type" VARCHAR
            )[],
            "memory" VARCHAR,
            "task_role_arn" VARCHAR
        ),
        "tags" STRUCT(
            "key" VARCHAR,
            "value" VARCHAR
        )[]
    ),
    "batch_job_parameters" STRUCT(
        "job_definition" VARCHAR,
        "job_name" VARCHAR,
        "array_properties" STRUCT(
            "size" BIGINT
        ),
        "retry_strategy" STRUCT(
            "attempts" BIGINT
        ),
        "container_overrides" STRUCT(
            "command" VARCHAR[],
            "environment" STRUCT(
                "name" VARCHAR,
                "value" VARCHAR
            )[],
            "instance_type" VARCHAR,
            "resource_requirements" STRUCT(
                "type" VARCHAR,
                "value" VARCHAR
            )[]
        ),
        "depends_on" STRUCT(
            "job_id" VARCHAR,
            "type" VARCHAR
        )[],
        "parameters" MAP(VARCHAR, VARCHAR)
    ),
    "sqs_queue_parameters" STRUCT(
        "message_group_id" VARCHAR,
        "message_deduplication_id" VARCHAR
    ),
    "http_parameters" STRUCT(
        "path_parameter_values" VARCHAR[],
        "header_parameters" MAP(VARCHAR, VARCHAR),
        "query_string_parameters" MAP(VARCHAR, VARCHAR)
    ),
    "redshift_data_parameters" STRUCT(
        "secret_manager_arn" VARCHAR,
        "database" VARCHAR,
        "db_user" VARCHAR,
        "statement_name" VARCHAR,
        "with_event" BOOLEAN,
        "sqls" VARCHAR[]
    ),
    "sage_maker_pipeline_parameters" STRUCT(
        "pipeline_parameter_list" STRUCT(
            "name" VARCHAR,
            "value" VARCHAR
        )[]
    ),
    "event_bridge_event_bus_parameters" STRUCT(
        "endpoint_id" VARCHAR,
        "detail_type" VARCHAR,
        "source" VARCHAR,
        "resources" VARCHAR[],
        "time" VARCHAR
    ),
    "cloud_watch_logs_parameters" STRUCT(
        "log_stream_name" VARCHAR,
        "timestamp" VARCHAR
    ),
    "timestream_parameters" STRUCT(
        "time_value" VARCHAR,
        "epoch_time_unit" VARCHAR,
        "time_field_type" VARCHAR,
        "timestamp_format" VARCHAR,
        "version_value" VARCHAR,
        "dimension_mappings" STRUCT(
            "dimension_value" VARCHAR,
            "dimension_value_type" VARCHAR,
            "dimension_name" VARCHAR
        )[],
        "single_measure_mappings" STRUCT(
            "measure_value" VARCHAR,
            "measure_value_type" VARCHAR,
            "measure_name" VARCHAR
        )[],
        "multi_measure_mappings" STRUCT(
            "multi_measure_name" VARCHAR,
            "multi_measure_attribute_mappings" STRUCT(
                "measure_value" VARCHAR,
                "measure_value_type" VARCHAR,
                "multi_measure_attribute_name" VARCHAR
            )[]
        )[]
    )
)
Show child fields
target_parameters.batch_job_parameters

The parameters for using an Batch job as a target.

Show child fields
target_parameters.batch_job_parameters.array_properties

The array properties for the submitted job, such as the size of the array. The array size can be between 2 and 10,000. If you specify array properties for a job, it becomes an array job. This parameter is used only if the target is an Batch job.

Show child fields
target_parameters.batch_job_parameters.array_properties.size

The size of the array, if this is an array batch job.

target_parameters.batch_job_parameters.container_overrides

The overrides that are sent to a container.

Show child fields
target_parameters.batch_job_parameters.container_overrides.command[]
target_parameters.batch_job_parameters.container_overrides.environment[]
Show child fields
target_parameters.batch_job_parameters.container_overrides.environment[].name

The name of the key-value pair. For environment variables, this is the name of the environment variable.

target_parameters.batch_job_parameters.container_overrides.environment[].value

The value of the key-value pair. For environment variables, this is the value of the environment variable.

target_parameters.batch_job_parameters.container_overrides.instance_type

The instance type to use for a multi-node parallel job.

This parameter isn't applicable to single-node container jobs or jobs that run on Fargate resources, and shouldn't be provided.

target_parameters.batch_job_parameters.container_overrides.resource_requirements[]
Show child fields
target_parameters.batch_job_parameters.container_overrides.resource_requirements[].type

The type of resource to assign to a container. The supported resources include GPU, MEMORY, and VCPU.

target_parameters.batch_job_parameters.container_overrides.resource_requirements[].value

The quantity of the specified resource to reserve for the container. The values vary based on the type specified.

type="GPU"

The number of physical GPUs to reserve for the container. Make sure that the number of GPUs reserved for all containers in a job doesn't exceed the number of available GPUs on the compute resource that the job is launched on.

GPUs aren't available for jobs that are running on Fargate resources.
type="MEMORY"

The memory hard limit (in MiB) present to the container. This parameter is supported for jobs that are running on EC2 resources. If your container attempts to exceed the memory specified, the container is terminated. This parameter maps to Memory in the Create a container section of the Docker Remote API and the --memory option to docker run. You must specify at least 4 MiB of memory for a job. This is required but can be specified in several places for multi-node parallel (MNP) jobs. It must be specified for each node at least once. This parameter maps to Memory in the Create a container section of the Docker Remote API and the --memory option to docker run.

If you're trying to maximize your resource utilization by providing your jobs as much memory as possible for a particular instance type, see Memory management in the Batch User Guide.

For jobs that are running on Fargate resources, then value is the hard limit (in MiB), and must match one of the supported values and the VCPU values must be one of the values supported for that memory value.

value = 512

VCPU = 0.25

value = 1024

VCPU = 0.25 or 0.5

value = 2048

VCPU = 0.25, 0.5, or 1

value = 3072

VCPU = 0.5, or 1

value = 4096

VCPU = 0.5, 1, or 2

value = 5120, 6144, or 7168

VCPU = 1 or 2

value = 8192

VCPU = 1, 2, 4, or 8

value = 9216, 10240, 11264, 12288, 13312, 14336, or 15360

VCPU = 2 or 4

value = 16384

VCPU = 2, 4, or 8

value = 17408, 18432, 19456, 21504, 22528, 23552, 25600, 26624, 27648, 29696, or 30720

VCPU = 4

value = 20480, 24576, or 28672

VCPU = 4 or 8

value = 36864, 45056, 53248, or 61440

VCPU = 8

value = 32768, 40960, 49152, or 57344

VCPU = 8 or 16

value = 65536, 73728, 81920, 90112, 98304, 106496, 114688, or 122880

VCPU = 16

type="VCPU"

The number of vCPUs reserved for the container. This parameter maps to CpuShares in the Create a container section of the Docker Remote API and the --cpu-shares option to docker run. Each vCPU is equivalent to 1,024 CPU shares. For EC2 resources, you must specify at least one vCPU. This is required but can be specified in several places; it must be specified for each node at least once.

The default for the Fargate On-Demand vCPU resource count quota is 6 vCPUs. For more information about Fargate quotas, see Fargate quotas in the Amazon Web Services General Reference.

For jobs that are running on Fargate resources, then value must match one of the supported values and the MEMORY values must be one of the values supported for that VCPU value. The supported values are 0.25, 0.5, 1, 2, 4, 8, and 16

value = 0.25

MEMORY = 512, 1024, or 2048

value = 0.5

MEMORY = 1024, 2048, 3072, or 4096

value = 1

MEMORY = 2048, 3072, 4096, 5120, 6144, 7168, or 8192

value = 2

MEMORY = 4096, 5120, 6144, 7168, 8192, 9216, 10240, 11264, 12288, 13312, 14336, 15360, or 16384

value = 4

MEMORY = 8192, 9216, 10240, 11264, 12288, 13312, 14336, 15360, 16384, 17408, 18432, 19456, 20480, 21504, 22528, 23552, 24576, 25600, 26624, 27648, 28672, 29696, or 30720

value = 8

MEMORY = 16384, 20480, 24576, 28672, 32768, 36864, 40960, 45056, 49152, 53248, 57344, or 61440

value = 16

MEMORY = 32768, 40960, 49152, 57344, 65536, 73728, 81920, 90112, 98304, 106496, 114688, or 122880

target_parameters.batch_job_parameters.depends_on[]
Show child fields
target_parameters.batch_job_parameters.depends_on[].job_id

The job ID of the Batch job that's associated with this dependency.

target_parameters.batch_job_parameters.depends_on[].type

The type of the job dependency.

target_parameters.batch_job_parameters.job_definition

The job definition used by this job. This value can be one of name, name:revision, or the Amazon Resource Name (ARN) for the job definition. If name is specified without a revision then the latest active revision is used.

target_parameters.batch_job_parameters.job_name

The name of the job. It can be up to 128 letters long. The first character must be alphanumeric, can contain uppercase and lowercase letters, numbers, hyphens (-), and underscores (_).

target_parameters.batch_job_parameters.parameters

Additional parameters passed to the job that replace parameter substitution placeholders that are set in the job definition. Parameters are specified as a key and value pair mapping. Parameters included here override any corresponding parameter defaults from the job definition.

target_parameters.batch_job_parameters.retry_strategy

The retry strategy to use for failed jobs. When a retry strategy is specified here, it overrides the retry strategy defined in the job definition.

Show child fields
target_parameters.batch_job_parameters.retry_strategy.attempts

The number of times to move a job to the RUNNABLE status. If the value of attempts is greater than one, the job is retried on failure the same number of attempts as the value.

target_parameters.cloud_watch_logs_parameters

The parameters for using an CloudWatch Logs log stream as a target.

Show child fields
target_parameters.cloud_watch_logs_parameters.log_stream_name

The name of the log stream.

target_parameters.cloud_watch_logs_parameters.timestamp

The time the event occurred, expressed as the number of milliseconds after Jan 1, 1970 00:00:00 UTC.

target_parameters.ecs_task_parameters

The parameters for using an Amazon ECS task as a target.

Show child fields
target_parameters.ecs_task_parameters.capacity_provider_strategy[]
Show child fields
target_parameters.ecs_task_parameters.capacity_provider_strategy[].base

The base value designates how many tasks, at a minimum, to run on the specified capacity provider. Only one capacity provider in a capacity provider strategy can have a base defined. If no value is specified, the default value of 0 is used.

target_parameters.ecs_task_parameters.capacity_provider_strategy[].capacity_provider

The short name of the capacity provider.

target_parameters.ecs_task_parameters.capacity_provider_strategy[].weight

The weight value designates the relative percentage of the total number of tasks launched that should use the specified capacity provider. The weight value is taken into consideration after the base value, if defined, is satisfied.

target_parameters.ecs_task_parameters.enable_ecs_managed_tags

Specifies whether to enable Amazon ECS managed tags for the task. For more information, see Tagging Your Amazon ECS Resources in the Amazon Elastic Container Service Developer Guide.

target_parameters.ecs_task_parameters.enable_execute_command

Whether or not to enable the execute command functionality for the containers in this task. If true, this enables execute command functionality on all containers in the task.

target_parameters.ecs_task_parameters.group

Specifies an Amazon ECS task group for the task. The maximum length is 255 characters.

target_parameters.ecs_task_parameters.launch_type

Specifies the launch type on which your task is running. The launch type that you specify here must match one of the launch type (compatibilities) of the target task. The FARGATE value is supported only in the Regions where Fargate with Amazon ECS is supported. For more information, see Fargate on Amazon ECS in the Amazon Elastic Container Service Developer Guide.

target_parameters.ecs_task_parameters.network_configuration

Use this structure if the Amazon ECS task uses the awsvpc network mode. This structure specifies the VPC subnets and security groups associated with the task, and whether a public IP address is to be used. This structure is required if LaunchType is FARGATE because the awsvpc mode is required for Fargate tasks.

If you specify NetworkConfiguration when the target ECS task does not use the awsvpc network mode, the task fails.

Show child fields
target_parameters.ecs_task_parameters.network_configuration.awsvpc_configuration

Use this structure to specify the VPC subnets and security groups for the task, and whether a public IP address is to be used. This structure is relevant only for ECS tasks that use the awsvpc network mode.

Show child fields
target_parameters.ecs_task_parameters.network_configuration.awsvpc_configuration.assign_public_ip

Specifies whether the task's elastic network interface receives a public IP address. You can specify ENABLED only when LaunchType in EcsParameters is set to FARGATE.

target_parameters.ecs_task_parameters.network_configuration.awsvpc_configuration.security_groups[]
target_parameters.ecs_task_parameters.network_configuration.awsvpc_configuration.subnets[]
target_parameters.ecs_task_parameters.overrides

The overrides that are associated with a task.

Show child fields
target_parameters.ecs_task_parameters.overrides.container_overrides[]
Show child fields
target_parameters.ecs_task_parameters.overrides.container_overrides[].command[]
target_parameters.ecs_task_parameters.overrides.container_overrides[].cpu

The number of cpu units reserved for the container, instead of the default value from the task definition. You must also specify a container name.

target_parameters.ecs_task_parameters.overrides.container_overrides[].environment[]
Show child fields
target_parameters.ecs_task_parameters.overrides.container_overrides[].environment[].name

The name of the key-value pair. For environment variables, this is the name of the environment variable.

target_parameters.ecs_task_parameters.overrides.container_overrides[].environment[].value

The value of the key-value pair. For environment variables, this is the value of the environment variable.

target_parameters.ecs_task_parameters.overrides.container_overrides[].environment_files[]
Show child fields
target_parameters.ecs_task_parameters.overrides.container_overrides[].environment_files[].type

The file type to use. The only supported value is s3.

target_parameters.ecs_task_parameters.overrides.container_overrides[].environment_files[].value

The Amazon Resource Name (ARN) of the Amazon S3 object containing the environment variable file.

target_parameters.ecs_task_parameters.overrides.container_overrides[].memory

The hard limit (in MiB) of memory to present to the container, instead of the default value from the task definition. If your container attempts to exceed the memory specified here, the container is killed. You must also specify a container name.

target_parameters.ecs_task_parameters.overrides.container_overrides[].memory_reservation

The soft limit (in MiB) of memory to reserve for the container, instead of the default value from the task definition. You must also specify a container name.

target_parameters.ecs_task_parameters.overrides.container_overrides[].name

The name of the container that receives the override. This parameter is required if any override is specified.

target_parameters.ecs_task_parameters.overrides.container_overrides[].resource_requirements[]
Show child fields
target_parameters.ecs_task_parameters.overrides.container_overrides[].resource_requirements[].type

The type of resource to assign to a container. The supported values are GPU or InferenceAccelerator.

target_parameters.ecs_task_parameters.overrides.container_overrides[].resource_requirements[].value

The value for the specified resource type.

If the GPU type is used, the value is the number of physical GPUs the Amazon ECS container agent reserves for the container. The number of GPUs that's reserved for all containers in a task can't exceed the number of available GPUs on the container instance that the task is launched on.

If the InferenceAccelerator type is used, the value matches the deviceName for an InferenceAccelerator specified in a task definition.

target_parameters.ecs_task_parameters.overrides.cpu

The cpu override for the task.

target_parameters.ecs_task_parameters.overrides.ephemeral_storage

The ephemeral storage setting override for the task.

This parameter is only supported for tasks hosted on Fargate that use the following platform versions:

  • Linux platform version 1.4.0 or later.

  • Windows platform version 1.0.0 or later.

Show child fields
target_parameters.ecs_task_parameters.overrides.ephemeral_storage.size_in_gi_b

The total amount, in GiB, of ephemeral storage to set for the task. The minimum supported value is 21 GiB and the maximum supported value is 200 GiB.

target_parameters.ecs_task_parameters.overrides.execution_role_arn

The Amazon Resource Name (ARN) of the task execution IAM role override for the task. For more information, see Amazon ECS task execution IAM role in the Amazon Elastic Container Service Developer Guide.

target_parameters.ecs_task_parameters.overrides.inference_accelerator_overrides[]
Show child fields
target_parameters.ecs_task_parameters.overrides.inference_accelerator_overrides[].device_name

The Elastic Inference accelerator device name to override for the task. This parameter must match a deviceName specified in the task definition.

target_parameters.ecs_task_parameters.overrides.inference_accelerator_overrides[].device_type

The Elastic Inference accelerator type to use.

target_parameters.ecs_task_parameters.overrides.memory

The memory override for the task.

target_parameters.ecs_task_parameters.overrides.task_role_arn

The Amazon Resource Name (ARN) of the IAM role that containers in this task can assume. All containers in this task are granted the permissions that are specified in this role. For more information, see IAM Role for Tasks in the Amazon Elastic Container Service Developer Guide.

target_parameters.ecs_task_parameters.placement_constraints[]
Show child fields
target_parameters.ecs_task_parameters.placement_constraints[].expression

A cluster query language expression to apply to the constraint. You cannot specify an expression if the constraint type is distinctInstance. To learn more, see Cluster Query Language in the Amazon Elastic Container Service Developer Guide.

target_parameters.ecs_task_parameters.placement_constraints[].type

The type of constraint. Use distinctInstance to ensure that each task in a particular group is running on a different container instance. Use memberOf to restrict the selection to a group of valid candidates.

target_parameters.ecs_task_parameters.placement_strategy[]
Show child fields
target_parameters.ecs_task_parameters.placement_strategy[].field

The field to apply the placement strategy against. For the spread placement strategy, valid values are instanceId (or host, which has the same effect), or any platform or custom attribute that is applied to a container instance, such as attribute:ecs.availability-zone. For the binpack placement strategy, valid values are cpu and memory. For the random placement strategy, this field is not used.

target_parameters.ecs_task_parameters.placement_strategy[].type

The type of placement strategy. The random placement strategy randomly places tasks on available candidates. The spread placement strategy spreads placement across available candidates evenly based on the field parameter. The binpack strategy places tasks on available candidates that have the least available amount of the resource that is specified with the field parameter. For example, if you binpack on memory, a task is placed on the instance with the least amount of remaining memory (but still enough to run the task).

target_parameters.ecs_task_parameters.platform_version

Specifies the platform version for the task. Specify only the numeric portion of the platform version, such as 1.1.0.

This structure is used only if LaunchType is FARGATE. For more information about valid platform versions, see Fargate Platform Versions in the Amazon Elastic Container Service Developer Guide.

target_parameters.ecs_task_parameters.propagate_tags

Specifies whether to propagate the tags from the task definition to the task. If no value is specified, the tags are not propagated. Tags can only be propagated to the task during task creation. To add tags to a task after task creation, use the TagResource API action.

target_parameters.ecs_task_parameters.reference_id

The reference ID to use for the task.

target_parameters.ecs_task_parameters.tags[]
Show child fields
target_parameters.ecs_task_parameters.tags[].key

A string you can use to assign a value. The combination of tag keys and values can help you organize and categorize your resources.

target_parameters.ecs_task_parameters.tags[].value

The value for the specified tag key.

target_parameters.ecs_task_parameters.task_count

The number of tasks to create based on TaskDefinition. The default is 1.

target_parameters.ecs_task_parameters.task_definition_arn

The ARN of the task definition to use if the event target is an Amazon ECS task.

target_parameters.event_bridge_event_bus_parameters

The parameters for using an EventBridge event bus as a target.

Show child fields
target_parameters.event_bridge_event_bus_parameters.detail_type

A free-form string, with a maximum of 128 characters, used to decide what fields to expect in the event detail.

target_parameters.event_bridge_event_bus_parameters.endpoint_id

The URL subdomain of the endpoint. For example, if the URL for Endpoint is https://abcde.veo.endpoints.event.amazonaws.com, then the EndpointId is abcde.veo.

target_parameters.event_bridge_event_bus_parameters.resources[]
target_parameters.event_bridge_event_bus_parameters.source

The source of the event.

target_parameters.event_bridge_event_bus_parameters.time

The time stamp of the event, per RFC3339. If no time stamp is provided, the time stamp of the PutEvents call is used.

target_parameters.http_parameters

These are custom parameter to be used when the target is an API Gateway REST APIs or EventBridge ApiDestinations.

Show child fields
target_parameters.http_parameters.header_parameters

The headers that need to be sent as part of request invoking the API Gateway REST API or EventBridge ApiDestination.

target_parameters.http_parameters.path_parameter_values[]
target_parameters.http_parameters.query_string_parameters

The query string keys/values that need to be sent as part of request invoking the API Gateway REST API or EventBridge ApiDestination.

target_parameters.input_template

Valid JSON text passed to the target. In this case, nothing from the event itself is passed to the target. For more information, see The JavaScript Object Notation (JSON) Data Interchange Format.

To remove an input template, specify an empty string.

target_parameters.kinesis_stream_parameters

The parameters for using a Kinesis stream as a target.

Show child fields
target_parameters.kinesis_stream_parameters.partition_key

Determines which shard in the stream the data record is assigned to. Partition keys are Unicode strings with a maximum length limit of 256 characters for each key. Amazon Kinesis Data Streams uses the partition key as input to a hash function that maps the partition key and associated data to a specific shard. Specifically, an MD5 hash function is used to map partition keys to 128-bit integer values and to map associated data records to shards. As a result of this hashing mechanism, all data records with the same partition key map to the same shard within the stream.

target_parameters.lambda_function_parameters

The parameters for using a Lambda function as a target.

Show child fields
target_parameters.lambda_function_parameters.invocation_type

Specify whether to invoke the function synchronously or asynchronously.

  • REQUEST_RESPONSE (default) - Invoke synchronously. This corresponds to the RequestResponse option in the InvocationType parameter for the Lambda Invoke API.

  • FIRE_AND_FORGET - Invoke asynchronously. This corresponds to the Event option in the InvocationType parameter for the Lambda Invoke API.

For more information, see Invocation types in the Amazon EventBridge User Guide.

target_parameters.redshift_data_parameters

These are custom parameters to be used when the target is a Amazon Redshift cluster to invoke the Amazon Redshift Data API BatchExecuteStatement.

Show child fields
target_parameters.redshift_data_parameters.database

The name of the database. Required when authenticating using temporary credentials.

target_parameters.redshift_data_parameters.db_user

The database user name. Required when authenticating using temporary credentials.

target_parameters.redshift_data_parameters.secret_manager_arn

The name or ARN of the secret that enables access to the database. Required when authenticating using Secrets Manager.

target_parameters.redshift_data_parameters.sqls[]
target_parameters.redshift_data_parameters.statement_name

The name of the SQL statement. You can name the SQL statement when you create it to identify the query.

target_parameters.redshift_data_parameters.with_event

Indicates whether to send an event back to EventBridge after the SQL statement runs.

target_parameters.sage_maker_pipeline_parameters

The parameters for using a SageMaker pipeline as a target.

Show child fields
target_parameters.sage_maker_pipeline_parameters.pipeline_parameter_list[]
Show child fields
target_parameters.sage_maker_pipeline_parameters.pipeline_parameter_list[].name

Name of parameter to start execution of a SageMaker Model Building Pipeline.

target_parameters.sage_maker_pipeline_parameters.pipeline_parameter_list[].value

Value of parameter to start execution of a SageMaker Model Building Pipeline.

target_parameters.sqs_queue_parameters

The parameters for using a Amazon SQS stream as a target.

Show child fields
target_parameters.sqs_queue_parameters.message_deduplication_id

This parameter applies only to FIFO (first-in-first-out) queues.

The token used for deduplication of sent messages.

target_parameters.sqs_queue_parameters.message_group_id

The FIFO message group ID to use as the target.

target_parameters.step_function_state_machine_parameters

The parameters for using a Step Functions state machine as a target.

Show child fields
target_parameters.step_function_state_machine_parameters.invocation_type

Specify whether to invoke the Step Functions state machine synchronously or asynchronously.

  • REQUEST_RESPONSE (default) - Invoke synchronously. For more information, see StartSyncExecution in the Step Functions API Reference.

    REQUEST_RESPONSE is not supported for STANDARD state machine workflows.

  • FIRE_AND_FORGET - Invoke asynchronously. For more information, see StartExecution in the Step Functions API Reference.

For more information, see Invocation types in the Amazon EventBridge User Guide.

target_parameters.timestream_parameters

The parameters for using a Timestream for LiveAnalytics table as a target.

Show child fields
target_parameters.timestream_parameters.dimension_mappings[]
Show child fields
target_parameters.timestream_parameters.dimension_mappings[].dimension_name

The metadata attributes of the time series. For example, the name and Availability Zone of an Amazon EC2 instance or the name of the manufacturer of a wind turbine are dimensions.

target_parameters.timestream_parameters.dimension_mappings[].dimension_value

Dynamic path to the dimension value in the source event.

target_parameters.timestream_parameters.dimension_mappings[].dimension_value_type

The data type of the dimension for the time-series data.

target_parameters.timestream_parameters.epoch_time_unit

The granularity of the time units used. Default is MILLISECONDS.

Required if TimeFieldType is specified as EPOCH.

target_parameters.timestream_parameters.multi_measure_mappings[]
Show child fields
target_parameters.timestream_parameters.multi_measure_mappings[].multi_measure_attribute_mappings[]
Show child fields
target_parameters.timestream_parameters.multi_measure_mappings[].multi_measure_attribute_mappings[].measure_value

Dynamic path to the measurement attribute in the source event.

target_parameters.timestream_parameters.multi_measure_mappings[].multi_measure_attribute_mappings[].measure_value_type

Data type of the measurement attribute in the source event.

target_parameters.timestream_parameters.multi_measure_mappings[].multi_measure_attribute_mappings[].multi_measure_attribute_name

Target measure name to be used.

target_parameters.timestream_parameters.multi_measure_mappings[].multi_measure_name

The name of the multiple measurements per record (multi-measure).

target_parameters.timestream_parameters.single_measure_mappings[]
Show child fields
target_parameters.timestream_parameters.single_measure_mappings[].measure_name

Target measure name for the measurement attribute in the Timestream table.

target_parameters.timestream_parameters.single_measure_mappings[].measure_value

Dynamic path of the source field to map to the measure in the record.

target_parameters.timestream_parameters.single_measure_mappings[].measure_value_type

Data type of the source field.

target_parameters.timestream_parameters.time_field_type

The type of time value used.

The default is EPOCH.

target_parameters.timestream_parameters.time_value

Dynamic path to the source data field that represents the time value for your data.

target_parameters.timestream_parameters.timestamp_format

How to format the timestamps. For example, YYYY-MM-DDThh:mm:ss.sssTZD.

Required if TimeFieldType is specified as TIMESTAMP_FORMAT.

target_parameters.timestream_parameters.version_value

64 bit version value or source data field that represents the version value for your data.

Write requests with a higher version number will update the existing measure values of the record and version. In cases where the measure value is the same, the version will still be updated.

Default value is 1.

Timestream for LiveAnalytics does not support updating partial measure values in a record.

Write requests for duplicate data with a higher version number will update the existing measure value and version. In cases where the measure value is the same, Version will still be updated. Default value is 1.

Version must be 1 or greater, or you will receive a ValidationException error.