Skip to content
AWS for Query.Farm

aws.databrew.list_job_runs

Example SQL Queries

SELECT * FROM
aws.databrew.list_job_runs
WHERE
"name" = 'VALUE';

Description

Lists all of the previous runs of a particular DataBrew job.

Table Definition

Column NameColumn Data Type
name Required Input Column

The name of the job.

VARCHAR
_aws_profile Input Column

The AWS profile defines the AWS identity used. It can be defined via credentials or by assuming a IAM role.

STRUCT(
    "type" VARCHAR,
    "name" VARCHAR,
    "account_id" VARCHAR,
    "via_profile_name" VARCHAR,
    "assumed_role_arn" VARCHAR,
    "organization" STRUCT(
        "account_name" VARCHAR,
        "id" VARCHAR,
        "tags" STRUCT(
            "key" VARCHAR,
            "value" VARCHAR
        )[],
        "master_account" STRUCT(
            "id" VARCHAR,
            "email" VARCHAR
        ),
        "parents" STRUCT(
            "type" VARCHAR,
            "id" VARCHAR,
            "name" VARCHAR,
            "tags" STRUCT(
                "key" VARCHAR,
                "value" VARCHAR
            )[]
        )[]
    )
)
Show child fields
_aws_profile.account_id

The AWS account id

_aws_profile.assumed_role_arn

The ARN of the assumed role

_aws_profile.name

The unique name of the profile.

_aws_profile.organization

Information about this profile's membership in the AWS organization.

Show child fields
_aws_profile.organization.account_name

The name of account speciifed by the organization

_aws_profile.organization.id

The organization id

_aws_profile.organization.master_account
Show child fields
_aws_profile.organization.master_account.email

The organization master account email address

_aws_profile.organization.master_account.id

The organization master account id

_aws_profile.organization.parents[]
Show child fields
_aws_profile.organization.parents[].id

The id of the parent

_aws_profile.organization.parents[].name

The name of the parent

_aws_profile.organization.parents[].tags[]
Show child fields
_aws_profile.organization.parents[].tags[].key
_aws_profile.organization.parents[].tags[].value
_aws_profile.organization.parents[].type

The type of parent can be an organization unit or a root

_aws_profile.organization.tags[]
Show child fields
_aws_profile.organization.tags[].key
_aws_profile.organization.tags[].value
_aws_profile.type

The type of profile, either 'credentials' or 'assumed_role'

_aws_profile.via_profile_name

This IAM role for this profile is assumed by first utilizing another profile with this name to obtain credentials.

_aws_region Input Column

The AWS region to use.

VARCHAR
attempt

The number of times that DataBrew has attempted to run the job.

BIGINT
completed_on

The date and time when the job completed processing.

TIMESTAMP_S
data_catalog_outputs

One or more artifacts that represent the Glue Data Catalog output from running the job.

STRUCT(
    "catalog_id" VARCHAR,
    "database_name" VARCHAR,
    "table_name" VARCHAR,
    "s3_options" STRUCT(
        "location" STRUCT(
            "bucket" VARCHAR,
            "key" VARCHAR,
            "bucket_owner" VARCHAR
        )
    ),
    "database_options" STRUCT(
        "temp_directory" STRUCT(
            "bucket" VARCHAR,
            "key" VARCHAR,
            "bucket_owner" VARCHAR
        ),
        "table_name" VARCHAR
    ),
    "overwrite" BOOLEAN
)[]
Show child fields
data_catalog_outputs[]
Show child fields
data_catalog_outputs[].catalog_id

The unique identifier of the Amazon Web Services account that holds the Data Catalog that stores the data.

data_catalog_outputs[].database_name

The name of a database in the Data Catalog.

data_catalog_outputs[].database_options

Represents options that specify how and where DataBrew writes the database output generated by recipe jobs.

Show child fields
data_catalog_outputs[].database_options.table_name

A prefix for the name of a table DataBrew will create in the database.

data_catalog_outputs[].database_options.temp_directory

Represents an Amazon S3 location (bucket name and object key) where DataBrew can store intermediate results.

Show child fields
data_catalog_outputs[].database_options.temp_directory.bucket

The Amazon S3 bucket name.

data_catalog_outputs[].database_options.temp_directory.bucket_owner

The Amazon Web Services account ID of the bucket owner.

data_catalog_outputs[].database_options.temp_directory.key

The unique name of the object in the bucket.

data_catalog_outputs[].overwrite

A value that, if true, means that any data in the location specified for output is overwritten with new output. Not supported with DatabaseOptions.

data_catalog_outputs[].s3_options

Represents options that specify how and where DataBrew writes the Amazon S3 output generated by recipe jobs.

Show child fields
data_catalog_outputs[].s3_options.location

Represents an Amazon S3 location (bucket name and object key) where DataBrew can write output from a job.

Show child fields
data_catalog_outputs[].s3_options.location.bucket

The Amazon S3 bucket name.

data_catalog_outputs[].s3_options.location.bucket_owner

The Amazon Web Services account ID of the bucket owner.

data_catalog_outputs[].s3_options.location.key

The unique name of the object in the bucket.

data_catalog_outputs[].table_name

The name of a table in the Data Catalog.

database_outputs

Represents a list of JDBC database output objects which defines the output destination for a DataBrew recipe job to write into.

STRUCT(
    "glue_connection_name" VARCHAR,
    "database_options" STRUCT(
        "temp_directory" STRUCT(
            "bucket" VARCHAR,
            "key" VARCHAR,
            "bucket_owner" VARCHAR
        ),
        "table_name" VARCHAR
    ),
    "database_output_mode" VARCHAR
)[]
Show child fields
database_outputs[]
Show child fields
database_outputs[].database_options

Represents options that specify how and where DataBrew writes the database output generated by recipe jobs.

Show child fields
database_outputs[].database_options.table_name

A prefix for the name of a table DataBrew will create in the database.

database_outputs[].database_options.temp_directory

Represents an Amazon S3 location (bucket name and object key) where DataBrew can store intermediate results.

Show child fields
database_outputs[].database_options.temp_directory.bucket

The Amazon S3 bucket name.

database_outputs[].database_options.temp_directory.bucket_owner

The Amazon Web Services account ID of the bucket owner.

database_outputs[].database_options.temp_directory.key

The unique name of the object in the bucket.

database_outputs[].database_output_mode

The output mode to write into the database. Currently supported option: NEW_TABLE.

database_outputs[].glue_connection_name

The Glue connection that stores the connection information for the target database.

dataset_name

The name of the dataset for the job to process.

VARCHAR
error_message

A message indicating an error (if any) that was encountered when the job ran.

VARCHAR
execution_time

The amount of time, in seconds, during which a job run consumed resources.

BIGINT
job_name

The name of the job being processed during this run.

VARCHAR
job_sample

A sample configuration for profile jobs only, which determines the number of rows on which the profile job is run. If a JobSample value isn't provided, the default is used. The default value is CUSTOM_ROWS for the mode parameter and 20,000 for the size parameter.

STRUCT(
    "mode" VARCHAR,
    "size" BIGINT
)
Show child fields
job_sample.mode

A value that determines whether the profile job is run on the entire dataset or a specified number of rows. This value must be one of the following:

  • FULL_DATASET - The profile job is run on the entire dataset.

  • CUSTOM_ROWS - The profile job is run on the number of rows specified in the Size parameter.

job_sample.size

The Size parameter is only required when the mode is CUSTOM_ROWS. The profile job is run on the specified number of rows. The maximum value for size is Long.MAX_VALUE.

Long.MAX_VALUE = 9223372036854775807

log_group_name

The name of an Amazon CloudWatch log group, where the job writes diagnostic messages when it runs.

VARCHAR
log_subscription

The current status of Amazon CloudWatch logging for the job run.

VARCHAR
outputs

One or more output artifacts from a job run.

STRUCT(
    "compression_format" VARCHAR,
    "format" VARCHAR,
    "partition_columns" VARCHAR[],
    "location" STRUCT(
        "bucket" VARCHAR,
        "key" VARCHAR,
        "bucket_owner" VARCHAR
    ),
    "overwrite" BOOLEAN,
    "format_options" STRUCT(
        "csv" STRUCT(
            "delimiter" VARCHAR
        )
    ),
    "max_output_files" BIGINT
)[]
Show child fields
outputs[]
Show child fields
outputs[].compression_format

The compression algorithm used to compress the output text of the job.

outputs[].format

The data format of the output of the job.

outputs[].format_options

Represents options that define how DataBrew formats job output files.

Show child fields
outputs[].format_options.csv

Represents a set of options that define the structure of comma-separated value (CSV) job output.

Show child fields
outputs[].format_options.csv.delimiter

A single character that specifies the delimiter used to create CSV job output.

outputs[].location

The location in Amazon S3 where the job writes its output.

Show child fields
outputs[].location.bucket

The Amazon S3 bucket name.

outputs[].location.bucket_owner

The Amazon Web Services account ID of the bucket owner.

outputs[].location.key

The unique name of the object in the bucket.

outputs[].max_output_files

Maximum number of files to be generated by the job and written to the output folder. For output partitioned by column(s), the MaxOutputFiles value is the maximum number of files per partition.

outputs[].overwrite

A value that, if true, means that any data in the location specified for output is overwritten with new output.

outputs[].partition_columns[]
recipe_reference

The set of steps processed by the job.

STRUCT(
    "name" VARCHAR,
    "recipe_version" VARCHAR
)
Show child fields
recipe_reference.name

The name of the recipe.

recipe_reference.recipe_version

The identifier for the version for the recipe.

run_id

The unique identifier of the job run.

VARCHAR
started_by

The Amazon Resource Name (ARN) of the user who initiated the job run. 

VARCHAR
started_on

The date and time when the job run began. 

TIMESTAMP_S
state

The current state of the job run entity itself.

VARCHAR
validation_configurations

List of validation configurations that are applied to the profile job run.

STRUCT(
    "ruleset_arn" VARCHAR,
    "validation_mode" VARCHAR
)[]
Show child fields
validation_configurations[]
Show child fields
validation_configurations[].ruleset_arn

The Amazon Resource Name (ARN) for the ruleset to be validated in the profile job. The TargetArn of the selected ruleset should be the same as the Amazon Resource Name (ARN) of the dataset that is associated with the profile job.

validation_configurations[].validation_mode

Mode of data quality validation. Default mode is “CHECK_ALL” which verifies all rules defined in the selected ruleset.