Skip to content
AWS for Query.Farm

aws.glue.list_data_quality_ruleset_evaluation_runs

Example SQL Queries

SELECT * FROM
aws.glue.list_data_quality_ruleset_evaluation_runs;

Description

Lists all the runs meeting the filter criteria, where a ruleset is evaluated against a data source.

Table Definition

Column NameColumn Data Type
filter Input Column

The filter criteria.

STRUCT(
    "data_source" STRUCT(
        "glue_table" STRUCT(
            "database_name" VARCHAR,
            "table_name" VARCHAR,
            "catalog_id" VARCHAR,
            "connection_name" VARCHAR,
            "additional_options" MAP(VARCHAR, VARCHAR)
        )
    ),
    "started_before" TIMESTAMP_S,
    "started_after" TIMESTAMP_S
)
Show child fields
filter.data_source

Filter based on a data source (an Glue table) associated with the run.

Show child fields
filter.data_source.glue_table

An Glue table.

Show child fields
filter.data_source.glue_table.additional_options

Additional options for the table. Currently there are two keys supported:

  • pushDownPredicate: to filter on partitions without having to list and read all the files in your dataset.

  • catalogPartitionPredicate: to use server-side partition pruning using partition indexes in the Glue Data Catalog.

filter.data_source.glue_table.catalog_id

A unique identifier for the Glue Data Catalog.

filter.data_source.glue_table.connection_name

The name of the connection to the Glue Data Catalog.

filter.data_source.glue_table.database_name

A database name in the Glue Data Catalog.

filter.data_source.glue_table.table_name

A table name in the Glue Data Catalog.

filter.started_after

Filter results by runs that started after this time.

filter.started_before

Filter results by runs that started before this time.

max_results Input Column

The maximum number of results to return.

BIGINT
next_token Input Column

A pagination token, if more results are available.

VARCHAR
_aws_profile Input Column

The AWS profile defines the AWS identity used. It can be defined via credentials or by assuming a IAM role.

STRUCT(
    "type" VARCHAR,
    "name" VARCHAR,
    "account_id" VARCHAR,
    "via_profile_name" VARCHAR,
    "assumed_role_arn" VARCHAR,
    "organization" STRUCT(
        "account_name" VARCHAR,
        "id" VARCHAR,
        "tags" STRUCT(
            "key" VARCHAR,
            "value" VARCHAR
        )[],
        "master_account" STRUCT(
            "id" VARCHAR,
            "email" VARCHAR
        ),
        "parents" STRUCT(
            "type" VARCHAR,
            "id" VARCHAR,
            "name" VARCHAR,
            "tags" STRUCT(
                "key" VARCHAR,
                "value" VARCHAR
            )[]
        )[]
    )
)
Show child fields
_aws_profile.account_id

The AWS account id

_aws_profile.assumed_role_arn

The ARN of the assumed role

_aws_profile.name

The unique name of the profile.

_aws_profile.organization

Information about this profile's membership in the AWS organization.

Show child fields
_aws_profile.organization.account_name

The name of account speciifed by the organization

_aws_profile.organization.id

The organization id

_aws_profile.organization.master_account
Show child fields
_aws_profile.organization.master_account.email

The organization master account email address

_aws_profile.organization.master_account.id

The organization master account id

_aws_profile.organization.parents[]
Show child fields
_aws_profile.organization.parents[].id

The id of the parent

_aws_profile.organization.parents[].name

The name of the parent

_aws_profile.organization.parents[].tags[]
Show child fields
_aws_profile.organization.parents[].tags[].key
_aws_profile.organization.parents[].tags[].value
_aws_profile.organization.parents[].type

The type of parent can be an organization unit or a root

_aws_profile.organization.tags[]
Show child fields
_aws_profile.organization.tags[].key
_aws_profile.organization.tags[].value
_aws_profile.type

The type of profile, either 'credentials' or 'assumed_role'

_aws_profile.via_profile_name

This IAM role for this profile is assumed by first utilizing another profile with this name to obtain credentials.

_aws_region Input Column

The AWS region to use.

VARCHAR
runs

A list of DataQualityRulesetEvaluationRunDescription objects representing data quality ruleset runs.

STRUCT(
    "run_id" VARCHAR,
    "status" VARCHAR,
    "started_on" TIMESTAMP_S,
    "data_source" STRUCT(
        "glue_table" STRUCT(
            "database_name" VARCHAR,
            "table_name" VARCHAR,
            "catalog_id" VARCHAR,
            "connection_name" VARCHAR,
            "additional_options" MAP(VARCHAR, VARCHAR)
        )
    )
)[]
Show child fields
runs[]
Show child fields
runs[].data_source

The data source (an Glue table) associated with the run.

Show child fields
runs[].data_source.glue_table

An Glue table.

Show child fields
runs[].data_source.glue_table.additional_options

Additional options for the table. Currently there are two keys supported:

  • pushDownPredicate: to filter on partitions without having to list and read all the files in your dataset.

  • catalogPartitionPredicate: to use server-side partition pruning using partition indexes in the Glue Data Catalog.

runs[].data_source.glue_table.catalog_id

A unique identifier for the Glue Data Catalog.

runs[].data_source.glue_table.connection_name

The name of the connection to the Glue Data Catalog.

runs[].data_source.glue_table.database_name

A database name in the Glue Data Catalog.

runs[].data_source.glue_table.table_name

A table name in the Glue Data Catalog.

runs[].run_id

The unique run identifier associated with this run.

runs[].started_on

The date and time when the run started.

runs[].status

The status for this run.