Skip to content
AWS for Query.Farm

aws.glue.get_dataflow_graph

Example SQL Queries

SELECT * FROM
aws.glue.get_dataflow_graph;

Description

Transforms a Python script into a directed acyclic graph (DAG).

Table Definition

Column NameColumn Data Type
python_script Input Column

The Python script to transform.

VARCHAR
_aws_profile Input Column

The AWS profile defines the AWS identity used. It can be defined via credentials or by assuming a IAM role.

STRUCT(
    "type" VARCHAR,
    "name" VARCHAR,
    "account_id" VARCHAR,
    "via_profile_name" VARCHAR,
    "assumed_role_arn" VARCHAR,
    "organization" STRUCT(
        "account_name" VARCHAR,
        "id" VARCHAR,
        "tags" STRUCT(
            "key" VARCHAR,
            "value" VARCHAR
        )[],
        "master_account" STRUCT(
            "id" VARCHAR,
            "email" VARCHAR
        ),
        "parents" STRUCT(
            "type" VARCHAR,
            "id" VARCHAR,
            "name" VARCHAR,
            "tags" STRUCT(
                "key" VARCHAR,
                "value" VARCHAR
            )[]
        )[]
    )
)
Show child fields
_aws_profile.account_id

The AWS account id

_aws_profile.assumed_role_arn

The ARN of the assumed role

_aws_profile.name

The unique name of the profile.

_aws_profile.organization

Information about this profile's membership in the AWS organization.

Show child fields
_aws_profile.organization.account_name

The name of account speciifed by the organization

_aws_profile.organization.id

The organization id

_aws_profile.organization.master_account
Show child fields
_aws_profile.organization.master_account.email

The organization master account email address

_aws_profile.organization.master_account.id

The organization master account id

_aws_profile.organization.parents[]
Show child fields
_aws_profile.organization.parents[].id

The id of the parent

_aws_profile.organization.parents[].name

The name of the parent

_aws_profile.organization.parents[].tags[]
Show child fields
_aws_profile.organization.parents[].tags[].key
_aws_profile.organization.parents[].tags[].value
_aws_profile.organization.parents[].type

The type of parent can be an organization unit or a root

_aws_profile.organization.tags[]
Show child fields
_aws_profile.organization.tags[].key
_aws_profile.organization.tags[].value
_aws_profile.type

The type of profile, either 'credentials' or 'assumed_role'

_aws_profile.via_profile_name

This IAM role for this profile is assumed by first utilizing another profile with this name to obtain credentials.

_aws_region Input Column

The AWS region to use.

VARCHAR
dag_edges

A list of the edges in the resulting DAG.

STRUCT(
    "source" VARCHAR,
    "target" VARCHAR,
    "target_parameter" VARCHAR
)[]
Show child fields
dag_edges[]
Show child fields
dag_edges[].source

The ID of the node at which the edge starts.

dag_edges[].target

The ID of the node at which the edge ends.

dag_edges[].target_parameter

The target of the edge.

dag_nodes

A list of the nodes in the resulting DAG.

STRUCT(
    "id" VARCHAR,
    "node_type" VARCHAR,
    "args" STRUCT(
        "name" VARCHAR,
        "value" VARCHAR,
        "param" BOOLEAN
    )[],
    "line_number" BIGINT
)[]
Show child fields
dag_nodes[]
Show child fields
dag_nodes[].args[]
Show child fields
dag_nodes[].args[].name

The name of the argument or property.

dag_nodes[].args[].param

True if the value is used as a parameter.

dag_nodes[].args[].value

The value of the argument or property.

dag_nodes[].id

A node identifier that is unique within the node's graph.

dag_nodes[].line_number

The line number of the node.

dag_nodes[].node_type

The type of node that this is.