Skip to content
AWS for Query.Farm

aws.glue.get_data_quality_result

Example SQL Queries

SELECT * FROM
aws.glue.get_data_quality_result
WHERE
"result_id" = 'VALUE';

Description

Retrieves the result of a data quality rule evaluation.

Table Definition

Column NameColumn Data Type
result_id Required Input Column

A unique result ID for the data quality result.

VARCHAR
_aws_profile Input Column

The AWS profile defines the AWS identity used. It can be defined via credentials or by assuming a IAM role.

STRUCT(
    "type" VARCHAR,
    "name" VARCHAR,
    "account_id" VARCHAR,
    "via_profile_name" VARCHAR,
    "assumed_role_arn" VARCHAR,
    "organization" STRUCT(
        "account_name" VARCHAR,
        "id" VARCHAR,
        "tags" STRUCT(
            "key" VARCHAR,
            "value" VARCHAR
        )[],
        "master_account" STRUCT(
            "id" VARCHAR,
            "email" VARCHAR
        ),
        "parents" STRUCT(
            "type" VARCHAR,
            "id" VARCHAR,
            "name" VARCHAR,
            "tags" STRUCT(
                "key" VARCHAR,
                "value" VARCHAR
            )[]
        )[]
    )
)
Show child fields
_aws_profile.account_id

The AWS account id

_aws_profile.assumed_role_arn

The ARN of the assumed role

_aws_profile.name

The unique name of the profile.

_aws_profile.organization

Information about this profile's membership in the AWS organization.

Show child fields
_aws_profile.organization.account_name

The name of account speciifed by the organization

_aws_profile.organization.id

The organization id

_aws_profile.organization.master_account
Show child fields
_aws_profile.organization.master_account.email

The organization master account email address

_aws_profile.organization.master_account.id

The organization master account id

_aws_profile.organization.parents[]
Show child fields
_aws_profile.organization.parents[].id

The id of the parent

_aws_profile.organization.parents[].name

The name of the parent

_aws_profile.organization.parents[].tags[]
Show child fields
_aws_profile.organization.parents[].tags[].key
_aws_profile.organization.parents[].tags[].value
_aws_profile.organization.parents[].type

The type of parent can be an organization unit or a root

_aws_profile.organization.tags[]
Show child fields
_aws_profile.organization.tags[].key
_aws_profile.organization.tags[].value
_aws_profile.type

The type of profile, either 'credentials' or 'assumed_role'

_aws_profile.via_profile_name

This IAM role for this profile is assumed by first utilizing another profile with this name to obtain credentials.

_aws_region Input Column

The AWS region to use.

VARCHAR
analyzer_results

A list of DataQualityAnalyzerResult objects representing the results for each analyzer. 

STRUCT(
    "name" VARCHAR,
    "description" VARCHAR,
    "evaluation_message" VARCHAR,
    "evaluated_metrics" MAP(VARCHAR, DOUBLE)
)[]
Show child fields
analyzer_results[]
Show child fields
analyzer_results[].description

A description of the data quality analyzer.

analyzer_results[].evaluated_metrics

A map of metrics associated with the evaluation of the analyzer.

analyzer_results[].evaluation_message

An evaluation message.

analyzer_results[].name

The name of the data quality analyzer.

completed_on

The date and time when the run for this data quality result was completed.

TIMESTAMP_S
data_source

The table associated with the data quality result, if any.

STRUCT(
    "glue_table" STRUCT(
        "database_name" VARCHAR,
        "table_name" VARCHAR,
        "catalog_id" VARCHAR,
        "connection_name" VARCHAR,
        "additional_options" MAP(VARCHAR, VARCHAR)
    )
)
Show child fields
data_source.glue_table

An Glue table.

Show child fields
data_source.glue_table.additional_options

Additional options for the table. Currently there are two keys supported:

  • pushDownPredicate: to filter on partitions without having to list and read all the files in your dataset.

  • catalogPartitionPredicate: to use server-side partition pruning using partition indexes in the Glue Data Catalog.

data_source.glue_table.catalog_id

A unique identifier for the Glue Data Catalog.

data_source.glue_table.connection_name

The name of the connection to the Glue Data Catalog.

data_source.glue_table.database_name

A database name in the Glue Data Catalog.

data_source.glue_table.table_name

A table name in the Glue Data Catalog.

evaluation_context

In the context of a job in Glue Studio, each node in the canvas is typically assigned some sort of name and data quality nodes will have names. In the case of multiple nodes, the evaluationContext can differentiate the nodes.

VARCHAR
job_name

The job name associated with the data quality result, if any.

VARCHAR
job_run_id

The job run ID associated with the data quality result, if any.

VARCHAR
observations

A list of DataQualityObservation objects representing the observations generated after evaluating the rules and analyzers. 

STRUCT(
    "description" VARCHAR,
    "metric_based_observation" STRUCT(
        "metric_name" VARCHAR,
        "statistic_id" VARCHAR,
        "metric_values" STRUCT(
            "actual_value" DOUBLE,
            "expected_value" DOUBLE,
            "lower_limit" DOUBLE,
            "upper_limit" DOUBLE
        ),
        "new_rules" VARCHAR[]
    )
)[]
Show child fields
observations[]
Show child fields
observations[].description

A description of the data quality observation.

observations[].metric_based_observation

An object of type MetricBasedObservation representing the observation that is based on evaluated data quality metrics.

Show child fields
observations[].metric_based_observation.metric_name

The name of the data quality metric used for generating the observation.

observations[].metric_based_observation.metric_values

An object of type DataQualityMetricValues representing the analysis of the data quality metric value.

Show child fields
observations[].metric_based_observation.metric_values.actual_value

The actual value of the data quality metric.

observations[].metric_based_observation.metric_values.expected_value

The expected value of the data quality metric according to the analysis of historical data.

observations[].metric_based_observation.metric_values.lower_limit

The lower limit of the data quality metric value according to the analysis of historical data.

observations[].metric_based_observation.metric_values.upper_limit

The upper limit of the data quality metric value according to the analysis of historical data.

observations[].metric_based_observation.new_rules[]
observations[].metric_based_observation.statistic_id

The Statistic ID.

profile_id

The Profile ID for the data quality result.

VARCHAR
rule_results

A list of DataQualityRuleResult objects representing the results for each rule. 

STRUCT(
    "name" VARCHAR,
    "description" VARCHAR,
    "evaluation_message" VARCHAR,
    "result" VARCHAR,
    "evaluated_metrics" MAP(VARCHAR, DOUBLE),
    "evaluated_rule" VARCHAR
)[]
Show child fields
rule_results[]
Show child fields
rule_results[].description

A description of the data quality rule.

rule_results[].evaluated_metrics

A map of metrics associated with the evaluation of the rule.

rule_results[].evaluated_rule

The evaluated rule.

rule_results[].evaluation_message

An evaluation message.

rule_results[].name

The name of the data quality rule.

rule_results[].result

A pass or fail status for the rule.

ruleset_evaluation_run_id

The unique run ID associated with the ruleset evaluation.

VARCHAR
ruleset_name

The name of the ruleset associated with the data quality result.

VARCHAR
score

An aggregate data quality score. Represents the ratio of rules that passed to the total number of rules.

DOUBLE
started_on

The date and time when the run for this data quality result started.

TIMESTAMP_S