Skip to content
AWS for Query.Farm

aws.glue.get_unfiltered_partition_metadata

Example SQL Queries

SELECT * FROM
aws.glue.get_unfiltered_partition_metadata
WHERE
"catalog_id" = 'VALUE'
AND "database_name" = 'VALUE'
AND "table_name" = 'VALUE'
AND "partition_values" = 'VALUE'
AND "supported_permission_types" = 'VALUE';

Description

Retrieves partition metadata from the Data Catalog that contains unfiltered metadata.

For IAM authorization, the public IAM action associated with this API is glue:GetPartition.

Table Definition

Column NameColumn Data Type
catalog_id Required Input Column

The catalog ID where the partition resides.

VARCHAR
database_name Required Input Column

(Required) Specifies the name of a database that contains the partition.

VARCHAR
partition_values Required Input Column

(Required) A list of partition key values.

VARCHAR[]
Show child fields
partition_values[]
supported_permission_types Required Input Column

(Required) A list of supported permission types. 

VARCHAR[]
Show child fields
supported_permission_types[]
table_name Required Input Column

(Required) Specifies the name of a table that contains the partition.

VARCHAR
audit_context Input Column

A structure containing Lake Formation audit context information.

STRUCT(
    "additional_audit_context" VARCHAR,
    "requested_columns" VARCHAR[],
    "all_columns_requested" BOOLEAN
)
Show child fields
audit_context.additional_audit_context

A string containing the additional audit context information.

audit_context.all_columns_requested

All columns request for audit.

audit_context.requested_columns[]
query_session_context Input Column

A structure used as a protocol between query engines and Lake Formation or Glue. Contains both a Lake Formation generated authorization identifier and information from the request's authorization context.

STRUCT(
    "query_id" VARCHAR,
    "query_start_time" TIMESTAMP_S,
    "cluster_id" VARCHAR,
    "query_authorization_id" VARCHAR,
    "additional_context" MAP(VARCHAR, VARCHAR)
)
Show child fields
query_session_context.additional_context

An opaque string-string map passed by the query engine.

query_session_context.cluster_id

An identifier string for the consumer cluster.

query_session_context.query_authorization_id

A cryptographically generated query identifier generated by Glue or Lake Formation.

query_session_context.query_id

A unique identifier generated by the query engine for the query.

query_session_context.query_start_time

A timestamp provided by the query engine for when the query started.

region Input Column

Specified only if the base tables belong to a different Amazon Web Services Region.

VARCHAR
_aws_profile Input Column

The AWS profile defines the AWS identity used. It can be defined via credentials or by assuming a IAM role.

STRUCT(
    "type" VARCHAR,
    "name" VARCHAR,
    "account_id" VARCHAR,
    "via_profile_name" VARCHAR,
    "assumed_role_arn" VARCHAR,
    "organization" STRUCT(
        "account_name" VARCHAR,
        "id" VARCHAR,
        "tags" STRUCT(
            "key" VARCHAR,
            "value" VARCHAR
        )[],
        "master_account" STRUCT(
            "id" VARCHAR,
            "email" VARCHAR
        ),
        "parents" STRUCT(
            "type" VARCHAR,
            "id" VARCHAR,
            "name" VARCHAR,
            "tags" STRUCT(
                "key" VARCHAR,
                "value" VARCHAR
            )[]
        )[]
    )
)
Show child fields
_aws_profile.account_id

The AWS account id

_aws_profile.assumed_role_arn

The ARN of the assumed role

_aws_profile.name

The unique name of the profile.

_aws_profile.organization

Information about this profile's membership in the AWS organization.

Show child fields
_aws_profile.organization.account_name

The name of account speciifed by the organization

_aws_profile.organization.id

The organization id

_aws_profile.organization.master_account
Show child fields
_aws_profile.organization.master_account.email

The organization master account email address

_aws_profile.organization.master_account.id

The organization master account id

_aws_profile.organization.parents[]
Show child fields
_aws_profile.organization.parents[].id

The id of the parent

_aws_profile.organization.parents[].name

The name of the parent

_aws_profile.organization.parents[].tags[]
Show child fields
_aws_profile.organization.parents[].tags[].key
_aws_profile.organization.parents[].tags[].value
_aws_profile.organization.parents[].type

The type of parent can be an organization unit or a root

_aws_profile.organization.tags[]
Show child fields
_aws_profile.organization.tags[].key
_aws_profile.organization.tags[].value
_aws_profile.type

The type of profile, either 'credentials' or 'assumed_role'

_aws_profile.via_profile_name

This IAM role for this profile is assumed by first utilizing another profile with this name to obtain credentials.

_aws_region Input Column

The AWS region to use.

VARCHAR
authorized_columns

A list of column names that the user has been granted access to.

VARCHAR[]
Show child fields
authorized_columns[]
is_registered_with_lake_formation

A Boolean value that indicates whether the partition location is registered with Lake Formation.

BOOLEAN
partition

A Partition object containing the partition metadata.

STRUCT(
    "values" VARCHAR[],
    "database_name" VARCHAR,
    "table_name" VARCHAR,
    "creation_time" TIMESTAMP_S,
    "last_access_time" TIMESTAMP_S,
    "storage_descriptor" STRUCT(
        "columns" STRUCT(
            "name" VARCHAR,
            "type" VARCHAR,
            "comment" VARCHAR,
            "parameters" MAP(VARCHAR, VARCHAR)
        )[],
        "location" VARCHAR,
        "additional_locations" VARCHAR[],
        "input_format" VARCHAR,
        "output_format" VARCHAR,
        "compressed" BOOLEAN,
        "number_of_buckets" BIGINT,
        "serde_info" STRUCT(
            "name" VARCHAR,
            "serialization_library" VARCHAR,
            "parameters" MAP(VARCHAR, VARCHAR)
        ),
        "bucket_columns" VARCHAR[],
        "sort_columns" STRUCT(
            "column" VARCHAR,
            "sort_order" BIGINT
        )[],
        "parameters" MAP(VARCHAR, VARCHAR),
        "skewed_info" STRUCT(
            "skewed_column_names" VARCHAR[],
            "skewed_column_values" VARCHAR[],
            "skewed_column_value_location_maps" MAP(VARCHAR, VARCHAR)
        ),
        "stored_as_sub_directories" BOOLEAN,
        "schema_reference" STRUCT(
            "schema_id" STRUCT(
                "schema_arn" VARCHAR,
                "schema_name" VARCHAR,
                "registry_name" VARCHAR
            ),
            "schema_version_id" VARCHAR,
            "schema_version_number" BIGINT
        )
    ),
    "parameters" MAP(VARCHAR, VARCHAR),
    "last_analyzed_time" TIMESTAMP_S,
    "catalog_id" VARCHAR
)
Show child fields
partition.catalog_id

The ID of the Data Catalog in which the partition resides.

partition.creation_time

The time at which the partition was created.

partition.database_name

The name of the catalog database in which to create the partition.

partition.last_access_time

The last time at which the partition was accessed.

partition.last_analyzed_time

The last time at which column statistics were computed for this partition.

partition.parameters

These key-value pairs define partition parameters.

partition.storage_descriptor

Provides information about the physical location where the partition is stored.

Show child fields
partition.storage_descriptor.additional_locations[]
partition.storage_descriptor.bucket_columns[]
partition.storage_descriptor.columns[]
Show child fields
partition.storage_descriptor.columns[].comment

A free-form text comment.

partition.storage_descriptor.columns[].name

The name of the Column.

partition.storage_descriptor.columns[].parameters

These key-value pairs define properties associated with the column.

partition.storage_descriptor.columns[].type

The data type of the Column.

partition.storage_descriptor.compressed

True if the data in the table is compressed, or False if not.

partition.storage_descriptor.input_format

The input format: SequenceFileInputFormat (binary), or TextInputFormat, or a custom format.

partition.storage_descriptor.location

The physical location of the table. By default, this takes the form of the warehouse location, followed by the database location in the warehouse, followed by the table name.

partition.storage_descriptor.number_of_buckets

Must be specified if the table contains any dimension columns.

partition.storage_descriptor.output_format

The output format: SequenceFileOutputFormat (binary), or IgnoreKeyTextOutputFormat, or a custom format.

partition.storage_descriptor.parameters

The user-supplied properties in key-value form.

partition.storage_descriptor.schema_reference

An object that references a schema stored in the Glue Schema Registry.

When creating a table, you can pass an empty list of columns for the schema, and instead use a schema reference.

Show child fields
partition.storage_descriptor.schema_reference.schema_id

A structure that contains schema identity fields. Either this or the SchemaVersionId has to be provided.

Show child fields
partition.storage_descriptor.schema_reference.schema_id.registry_name

The name of the schema registry that contains the schema.

partition.storage_descriptor.schema_reference.schema_id.schema_arn

The Amazon Resource Name (ARN) of the schema. One of SchemaArn or SchemaName has to be provided.

partition.storage_descriptor.schema_reference.schema_id.schema_name

The name of the schema. One of SchemaArn or SchemaName has to be provided.

partition.storage_descriptor.schema_reference.schema_version_id

The unique ID assigned to a version of the schema. Either this or the SchemaId has to be provided.

partition.storage_descriptor.schema_reference.schema_version_number

The version number of the schema.

partition.storage_descriptor.serde_info

The serialization/deserialization (SerDe) information.

Show child fields
partition.storage_descriptor.serde_info.name

Name of the SerDe.

partition.storage_descriptor.serde_info.parameters

These key-value pairs define initialization parameters for the SerDe.

partition.storage_descriptor.serde_info.serialization_library

Usually the class that implements the SerDe. An example is org.apache.hadoop.hive.serde2.columnar.ColumnarSerDe.

partition.storage_descriptor.skewed_info

The information about values that appear frequently in a column (skewed values).

Show child fields
partition.storage_descriptor.skewed_info.skewed_column_names[]
partition.storage_descriptor.skewed_info.skewed_column_value_location_maps

A mapping of skewed values to the columns that contain them.

partition.storage_descriptor.skewed_info.skewed_column_values[]
partition.storage_descriptor.sort_columns[]
Show child fields
partition.storage_descriptor.sort_columns[].column

The name of the column.

partition.storage_descriptor.sort_columns[].sort_order

Indicates that the column is sorted in ascending order (== 1), or in descending order (==0).

partition.storage_descriptor.stored_as_sub_directories

True if the table data is stored in subdirectories, or False if not.

partition.table_name

The name of the database table in which to create the partition.

partition.values[]