Skip to content
AWS for Query.Farm

aws.datapipeline.query_objects

Example SQL Queries

SELECT * FROM
aws.datapipeline.query_objects
WHERE
"pipeline_id" = 'VALUE'
AND "sphere" = 'VALUE';

Description

Queries the specified pipeline for the names of objects that match the specified set of conditions.

Table Definition

Column NameColumn Data Type
pipeline_id Required Input Column

The ID of the pipeline.

VARCHAR
sphere Required Input Column

Indicates whether the query applies to components or instances. The possible values are: COMPONENT, INSTANCE, and ATTEMPT.

VARCHAR
query Input Column

The query that defines the objects to be returned. The Query object can contain a maximum of ten selectors. The conditions in the query are limited to top-level String fields in the object. These filters can be applied to components, instances, and attempts.

STRUCT(
    "selectors" STRUCT(
        "field_name" VARCHAR,
        "operator" STRUCT(
            "type" VARCHAR,
            "values" VARCHAR[]
        )
    )[]
)
Show child fields
query.selectors[]
Show child fields
query.selectors[].field_name

The name of the field that the operator will be applied to. The field name is the "key" portion of the field definition in the pipeline definition syntax that is used by the AWS Data Pipeline API. If the field is not set on the object, the condition fails.

query.selectors[].operator

Contains a logical operation for comparing the value of a field with a specified value.

Show child fields
query.selectors[].operator.type

The logical operation to be performed: equal (EQ), equal reference (REF_EQ), less than or equal (LE), greater than or equal (GE), or between (BETWEEN). Equal reference (REF_EQ) can be used only with reference fields. The other comparison types can be used only with String fields. The comparison types you can use apply only to certain object fields, as detailed below.

The comparison operators EQ and REF_EQ act on the following fields:

  • name
  • @sphere
  • parent
  • @componentParent
  • @instanceParent
  • @status
  • @scheduledStartTime
  • @scheduledEndTime
  • @actualStartTime
  • @actualEndTime

The comparison operators GE, LE, and BETWEEN act on the following fields:

  • @scheduledStartTime
  • @scheduledEndTime
  • @actualStartTime
  • @actualEndTime

Note that fields beginning with the at sign (@) are read-only and set by the web service. When you name fields, you should choose names containing only alpha-numeric values, as symbols may be reserved by AWS Data Pipeline. User-defined fields that you add to a pipeline should prefix their name with the string "my".

query.selectors[].operator.values[]
_aws_profile Input Column

The AWS profile defines the AWS identity used. It can be defined via credentials or by assuming a IAM role.

STRUCT(
    "type" VARCHAR,
    "name" VARCHAR,
    "account_id" VARCHAR,
    "via_profile_name" VARCHAR,
    "assumed_role_arn" VARCHAR,
    "organization" STRUCT(
        "account_name" VARCHAR,
        "id" VARCHAR,
        "tags" STRUCT(
            "key" VARCHAR,
            "value" VARCHAR
        )[],
        "master_account" STRUCT(
            "id" VARCHAR,
            "email" VARCHAR
        ),
        "parents" STRUCT(
            "type" VARCHAR,
            "id" VARCHAR,
            "name" VARCHAR,
            "tags" STRUCT(
                "key" VARCHAR,
                "value" VARCHAR
            )[]
        )[]
    )
)
Show child fields
_aws_profile.account_id

The AWS account id

_aws_profile.assumed_role_arn

The ARN of the assumed role

_aws_profile.name

The unique name of the profile.

_aws_profile.organization

Information about this profile's membership in the AWS organization.

Show child fields
_aws_profile.organization.account_name

The name of account speciifed by the organization

_aws_profile.organization.id

The organization id

_aws_profile.organization.master_account
Show child fields
_aws_profile.organization.master_account.email

The organization master account email address

_aws_profile.organization.master_account.id

The organization master account id

_aws_profile.organization.parents[]
Show child fields
_aws_profile.organization.parents[].id

The id of the parent

_aws_profile.organization.parents[].name

The name of the parent

_aws_profile.organization.parents[].tags[]
Show child fields
_aws_profile.organization.parents[].tags[].key
_aws_profile.organization.parents[].tags[].value
_aws_profile.organization.parents[].type

The type of parent can be an organization unit or a root

_aws_profile.organization.tags[]
Show child fields
_aws_profile.organization.tags[].key
_aws_profile.organization.tags[].value
_aws_profile.type

The type of profile, either 'credentials' or 'assumed_role'

_aws_profile.via_profile_name

This IAM role for this profile is assumed by first utilizing another profile with this name to obtain credentials.

_aws_region Input Column

The AWS region to use.

VARCHAR
ids 
VARCHAR