| Column Name | Column Data Type |
|---|
virtual_cluster_id Required Input Column
The ID of the job run's virtual cluster. | VARCHAR |
created_after Input Column
The date and time after which the job runs were submitted. | TIMESTAMP_S |
created_before Input Column
The date and time before which the job runs were submitted. | TIMESTAMP_S |
name Input Column
The name of the job run. | VARCHAR |
states Input Column
The states of the job run. | VARCHAR[] |
Show child fields- states[]
|
_aws_profile Input Column
The AWS profile defines the AWS identity used. It can be defined via credentials or by assuming a IAM role. | STRUCT(
"type" VARCHAR,
"name" VARCHAR,
"account_id" VARCHAR,
"via_profile_name" VARCHAR,
"assumed_role_arn" VARCHAR,
"organization" STRUCT(
"account_name" VARCHAR,
"id" VARCHAR,
"tags" STRUCT(
"key" VARCHAR,
"value" VARCHAR
)[],
"master_account" STRUCT(
"id" VARCHAR,
"email" VARCHAR
),
"parents" STRUCT(
"type" VARCHAR,
"id" VARCHAR,
"name" VARCHAR,
"tags" STRUCT(
"key" VARCHAR,
"value" VARCHAR
)[]
)[]
)
) |
Show child fields- _aws_profile.account_id
The AWS account id - _aws_profile.assumed_role_arn
The ARN of the assumed role - _aws_profile.name
The unique name of the profile. - _aws_profile.organization
Information about this profile's membership in the AWS organization. Show child fields- _aws_profile.organization.account_name
The name of account speciifed by the organization - _aws_profile.organization.id
The organization id - _aws_profile.organization.master_account
Show child fields- _aws_profile.organization.master_account.email
The organization master account email address - _aws_profile.organization.master_account.id
The organization master account id
- _aws_profile.organization.parents[]
Show child fields- _aws_profile.organization.parents[].id
The id of the parent - _aws_profile.organization.parents[].name
The name of the parent - _aws_profile.organization.parents[].tags[]
Show child fields- _aws_profile.organization.parents[].tags[].key
- _aws_profile.organization.parents[].tags[].value
- _aws_profile.organization.parents[].type
The type of parent can be an organization unit or a root
- _aws_profile.organization.tags[]
Show child fields- _aws_profile.organization.tags[].key
- _aws_profile.organization.tags[].value
- _aws_profile.type
The type of profile, either 'credentials' or 'assumed_role' - _aws_profile.via_profile_name
This IAM role for this profile is assumed by first utilizing another profile with this name to obtain credentials.
|
_aws_region Input Column
The AWS region to use. | VARCHAR |
arn
The ARN of job run. | VARCHAR |
client_token
The client token used to start a job run. | VARCHAR |
configuration_overrides
The configuration settings that are used to override default configuration. | STRUCT(
"application_configuration" STRUCT(
"classification" VARCHAR,
"properties" MAP(VARCHAR, VARCHAR)
)[],
"monitoring_configuration" STRUCT(
"persistent_app_ui" VARCHAR,
"cloud_watch_monitoring_configuration" STRUCT(
"log_group_name" VARCHAR,
"log_stream_name_prefix" VARCHAR
),
"s3_monitoring_configuration" STRUCT(
"log_uri" VARCHAR
),
"container_log_rotation_configuration" STRUCT(
"rotation_size" VARCHAR,
"max_files_to_keep" BIGINT
)
)
) |
Show child fields- configuration_overrides.application_configuration[]
Show child fields- configuration_overrides.application_configuration[].classification
The classification within a configuration. - configuration_overrides.application_configuration[].properties
A set of properties specified within a configuration classification.
- configuration_overrides.monitoring_configuration
The configurations for monitoring. Show child fields- configuration_overrides.monitoring_configuration.cloud_watch_monitoring_configuration
Monitoring configurations for CloudWatch. Show child fields- configuration_overrides.monitoring_configuration.cloud_watch_monitoring_configuration.log_group_name
The name of the log group for log publishing. - configuration_overrides.monitoring_configuration.cloud_watch_monitoring_configuration.log_stream_name_prefix
The specified name prefix for log streams.
- configuration_overrides.monitoring_configuration.container_log_rotation_configuration
Enable or disable container log rotation. Show child fields- configuration_overrides.monitoring_configuration.container_log_rotation_configuration.max_files_to_keep
The number of files to keep in container after rotation. - configuration_overrides.monitoring_configuration.container_log_rotation_configuration.rotation_size
The file size at which to rotate logs. Minimum of 2KB, Maximum of 2GB.
- configuration_overrides.monitoring_configuration.persistent_app_ui
Monitoring configurations for the persistent application UI. - configuration_overrides.monitoring_configuration.s3_monitoring_configuration
Amazon S3 configuration for monitoring log publishing. Show child fields- configuration_overrides.monitoring_configuration.s3_monitoring_configuration.log_uri
Amazon S3 destination URI for log publishing.
|
created_at
The date and time when the job run was created. | TIMESTAMP_S |
created_by
The user who created the job run. | VARCHAR |
execution_role_arn
The execution role ARN of the job run. | VARCHAR |
failure_reason
The reasons why the job run has failed. | VARCHAR |
finished_at
The date and time when the job run has finished. | TIMESTAMP_S |
id
The ID of the job run. | VARCHAR |
job_driver
Parameters of job driver for the job run. | STRUCT(
"spark_submit_job_driver" STRUCT(
"entry_point" VARCHAR,
"entry_point_arguments" VARCHAR[],
"spark_submit_parameters" VARCHAR
),
"spark_sql_job_driver" STRUCT(
"entry_point" VARCHAR,
"spark_sql_parameters" VARCHAR
)
) |
Show child fields- job_driver.spark_sql_job_driver
The job driver for job type. Show child fields- job_driver.spark_sql_job_driver.entry_point
The SQL file to be executed. - job_driver.spark_sql_job_driver.spark_sql_parameters
The Spark parameters to be included in the Spark SQL command.
- job_driver.spark_submit_job_driver
The job driver parameters specified for spark submit. Show child fields- job_driver.spark_submit_job_driver.entry_point
The entry point of job application. - job_driver.spark_submit_job_driver.entry_point_arguments[]
- job_driver.spark_submit_job_driver.spark_submit_parameters
The Spark submit parameters that are used for job runs.
|
release_label
The release version of Amazon EMR. | VARCHAR |
retry_policy_configuration
The configuration of the retry policy that the job runs on. | STRUCT(
"max_attempts" BIGINT
) |
Show child fields- retry_policy_configuration.max_attempts
The maximum number of attempts on the job's driver.
|
retry_policy_execution
The current status of the retry policy executed on the job. | STRUCT(
"current_attempt_count" BIGINT
) |
Show child fields- retry_policy_execution.current_attempt_count
The current number of attempts made on the driver of the job.
|
state
The state of the job run. | VARCHAR |
state_details
Additional details of the job run state. | VARCHAR |
tags
The assigned tags of the job run. | MAP(VARCHAR, VARCHAR) |