Skip to content
AWS for Query.Farm

aws.emr.list_steps

Example SQL Queries

SELECT * FROM
aws.emr.list_steps
WHERE
"cluster_id" = 'VALUE';

Description

Provides a list of steps for the cluster in reverse order unless you specify stepIds with the request or filter by StepStates. You can specify a maximum of 10 stepIDs. The CLI automatically paginates results to return a list greater than 50 steps. To return more than 50 steps using the CLI, specify a Marker, which is a pagination token that indicates the next set of steps to retrieve.

Table Definition

Column NameColumn Data Type
cluster_id Required Input Column

The identifier of the cluster for which to list the steps.

VARCHAR
step_ids Input Column

The filter to limit the step list based on the identifier of the steps. You can specify a maximum of ten Step IDs. The character constraint applies to the overall length of the array.

VARCHAR[]
Show child fields
step_ids[]
step_states Input Column

The filter to limit the step list based on certain states.

VARCHAR[]
Show child fields
step_states[]
_aws_profile Input Column

The AWS profile defines the AWS identity used. It can be defined via credentials or by assuming a IAM role.

STRUCT(
    "type" VARCHAR,
    "name" VARCHAR,
    "account_id" VARCHAR,
    "via_profile_name" VARCHAR,
    "assumed_role_arn" VARCHAR,
    "organization" STRUCT(
        "account_name" VARCHAR,
        "id" VARCHAR,
        "tags" STRUCT(
            "key" VARCHAR,
            "value" VARCHAR
        )[],
        "master_account" STRUCT(
            "id" VARCHAR,
            "email" VARCHAR
        ),
        "parents" STRUCT(
            "type" VARCHAR,
            "id" VARCHAR,
            "name" VARCHAR,
            "tags" STRUCT(
                "key" VARCHAR,
                "value" VARCHAR
            )[]
        )[]
    )
)
Show child fields
_aws_profile.account_id

The AWS account id

_aws_profile.assumed_role_arn

The ARN of the assumed role

_aws_profile.name

The unique name of the profile.

_aws_profile.organization

Information about this profile's membership in the AWS organization.

Show child fields
_aws_profile.organization.account_name

The name of account speciifed by the organization

_aws_profile.organization.id

The organization id

_aws_profile.organization.master_account
Show child fields
_aws_profile.organization.master_account.email

The organization master account email address

_aws_profile.organization.master_account.id

The organization master account id

_aws_profile.organization.parents[]
Show child fields
_aws_profile.organization.parents[].id

The id of the parent

_aws_profile.organization.parents[].name

The name of the parent

_aws_profile.organization.parents[].tags[]
Show child fields
_aws_profile.organization.parents[].tags[].key
_aws_profile.organization.parents[].tags[].value
_aws_profile.organization.parents[].type

The type of parent can be an organization unit or a root

_aws_profile.organization.tags[]
Show child fields
_aws_profile.organization.tags[].key
_aws_profile.organization.tags[].value
_aws_profile.type

The type of profile, either 'credentials' or 'assumed_role'

_aws_profile.via_profile_name

This IAM role for this profile is assumed by first utilizing another profile with this name to obtain credentials.

action_on_failure

The action to take when the cluster step fails. Possible values are TERMINATE_CLUSTER, CANCEL_AND_WAIT, and CONTINUE. TERMINATE_JOB_FLOW is available for backward compatibility.

VARCHAR
config

The Hadoop job configuration of the cluster step.

STRUCT(
    "jar" VARCHAR,
    "properties" MAP(VARCHAR, VARCHAR),
    "main_class" VARCHAR,
    "args" VARCHAR[]
)
Show child fields
config.args[]
config.jar

The path to the JAR file that runs during the step.

config.main_class

The name of the main class in the specified Java file. If not specified, the JAR file should specify a main class in its manifest file.

config.properties

The list of Java properties that are set when the step runs. You can use these properties to pass key-value pairs to your main function.

id

The identifier of the cluster step.

VARCHAR
name

The name of the cluster step.

VARCHAR
status

The current execution status details of the cluster step.

STRUCT(
    "state" VARCHAR,
    "state_change_reason" STRUCT(
        "code" VARCHAR,
        "message" VARCHAR
    ),
    "failure_details" STRUCT(
        "reason" VARCHAR,
        "message" VARCHAR,
        "log_file" VARCHAR
    ),
    "timeline" STRUCT(
        "creation_date_time" TIMESTAMP_S,
        "start_date_time" TIMESTAMP_S,
        "end_date_time" TIMESTAMP_S
    )
)
Show child fields
status.failure_details

The details for the step failure including reason, message, and log file path where the root cause was identified.

Show child fields
status.failure_details.log_file

The path to the log file where the step failure root cause was originally recorded.

status.failure_details.message

The descriptive message including the error the Amazon EMR service has identified as the cause of step failure. This is text from an error log that describes the root cause of the failure.

status.failure_details.reason

The reason for the step failure. In the case where the service cannot successfully determine the root cause of the failure, it returns "Unknown Error" as a reason.

status.state

The execution state of the cluster step.

status.state_change_reason

The reason for the step execution status change.

Show child fields
status.state_change_reason.code

The programmable code for the state change reason. Note: Currently, the service provides no code for the state change.

status.state_change_reason.message

The descriptive message for the state change reason.

status.timeline

The timeline of the cluster step status over time.

Show child fields
status.timeline.creation_date_time

The date and time when the cluster step was created.

status.timeline.end_date_time

The date and time when the cluster step execution completed or failed.

status.timeline.start_date_time

The date and time when the cluster step execution started.