Skip to content
AWS for Query.Farm

aws.lookoutequipment.describe_dataset

Example SQL Queries

SELECT * FROM
aws.lookoutequipment.describe_dataset
WHERE
"dataset_name" = 'VALUE';

Description

Provides a JSON description of the data in each time series dataset, including names, column names, and data types.

Table Definition

Column NameColumn Data Type
dataset_name Required Input Column

The name of the dataset being described. 

VARCHAR
_aws_profile Input Column

The AWS profile defines the AWS identity used. It can be defined via credentials or by assuming a IAM role.

STRUCT(
    "type" VARCHAR,
    "name" VARCHAR,
    "account_id" VARCHAR,
    "via_profile_name" VARCHAR,
    "assumed_role_arn" VARCHAR,
    "organization" STRUCT(
        "account_name" VARCHAR,
        "id" VARCHAR,
        "tags" STRUCT(
            "key" VARCHAR,
            "value" VARCHAR
        )[],
        "master_account" STRUCT(
            "id" VARCHAR,
            "email" VARCHAR
        ),
        "parents" STRUCT(
            "type" VARCHAR,
            "id" VARCHAR,
            "name" VARCHAR,
            "tags" STRUCT(
                "key" VARCHAR,
                "value" VARCHAR
            )[]
        )[]
    )
)
Show child fields
_aws_profile.account_id

The AWS account id

_aws_profile.assumed_role_arn

The ARN of the assumed role

_aws_profile.name

The unique name of the profile.

_aws_profile.organization

Information about this profile's membership in the AWS organization.

Show child fields
_aws_profile.organization.account_name

The name of account speciifed by the organization

_aws_profile.organization.id

The organization id

_aws_profile.organization.master_account
Show child fields
_aws_profile.organization.master_account.email

The organization master account email address

_aws_profile.organization.master_account.id

The organization master account id

_aws_profile.organization.parents[]
Show child fields
_aws_profile.organization.parents[].id

The id of the parent

_aws_profile.organization.parents[].name

The name of the parent

_aws_profile.organization.parents[].tags[]
Show child fields
_aws_profile.organization.parents[].tags[].key
_aws_profile.organization.parents[].tags[].value
_aws_profile.organization.parents[].type

The type of parent can be an organization unit or a root

_aws_profile.organization.tags[]
Show child fields
_aws_profile.organization.tags[].key
_aws_profile.organization.tags[].value
_aws_profile.type

The type of profile, either 'credentials' or 'assumed_role'

_aws_profile.via_profile_name

This IAM role for this profile is assumed by first utilizing another profile with this name to obtain credentials.

_aws_region Input Column

The AWS region to use.

VARCHAR
created_at

Specifies the time the dataset was created in Lookout for Equipment. 

TIMESTAMP_S
data_end_time

Indicates the latest timestamp corresponding to data that was successfully ingested during the most recent ingestion of this particular dataset. 

TIMESTAMP_S
data_quality_summary

Gives statistics associated with the given dataset for the latest successful associated ingestion job id. These statistics primarily relate to quantifying incorrect data such as MissingCompleteSensorData, MissingSensorData, UnsupportedDateFormats, InsufficientSensorData, and DuplicateTimeStamps. 

STRUCT(
    "insufficient_sensor_data" STRUCT(
        "missing_complete_sensor_data" STRUCT(
            "affected_sensor_count" BIGINT
        ),
        "sensors_with_short_date_range" STRUCT(
            "affected_sensor_count" BIGINT
        )
    ),
    "missing_sensor_data" STRUCT(
        "affected_sensor_count" BIGINT,
        "total_number_of_missing_values" BIGINT
    ),
    "invalid_sensor_data" STRUCT(
        "affected_sensor_count" BIGINT,
        "total_number_of_invalid_values" BIGINT
    ),
    "unsupported_timestamps" STRUCT(
        "total_number_of_unsupported_timestamps" BIGINT
    ),
    "duplicate_timestamps" STRUCT(
        "total_number_of_duplicate_timestamps" BIGINT
    )
)
Show child fields
data_quality_summary.duplicate_timestamps

Parameter that gives information about duplicate timestamps in the input data.

Show child fields
data_quality_summary.duplicate_timestamps.total_number_of_duplicate_timestamps

Indicates the total number of duplicate timestamps.

data_quality_summary.insufficient_sensor_data

Parameter that gives information about insufficient data for sensors in the dataset. This includes information about those sensors that have complete data missing and those with a short date range.

Show child fields
data_quality_summary.insufficient_sensor_data.missing_complete_sensor_data

Parameter that describes the total number of sensors that have data completely missing for it.

Show child fields
data_quality_summary.insufficient_sensor_data.missing_complete_sensor_data.affected_sensor_count

Indicates the number of sensors that have data missing completely.

data_quality_summary.insufficient_sensor_data.sensors_with_short_date_range

Parameter that describes the total number of sensors that have a short date range of less than 14 days of data overall.

Show child fields
data_quality_summary.insufficient_sensor_data.sensors_with_short_date_range.affected_sensor_count

Indicates the number of sensors that have less than 14 days of data.

data_quality_summary.invalid_sensor_data

Parameter that gives information about data that is invalid over all the sensors in the input data.

Show child fields
data_quality_summary.invalid_sensor_data.affected_sensor_count

Indicates the number of sensors that have at least some invalid values.

data_quality_summary.invalid_sensor_data.total_number_of_invalid_values

Indicates the total number of invalid values across all the sensors.

data_quality_summary.missing_sensor_data

Parameter that gives information about data that is missing over all the sensors in the input data.

Show child fields
data_quality_summary.missing_sensor_data.affected_sensor_count

Indicates the number of sensors that have atleast some data missing.

data_quality_summary.missing_sensor_data.total_number_of_missing_values

Indicates the total number of missing values across all the sensors.

data_quality_summary.unsupported_timestamps

Parameter that gives information about unsupported timestamps in the input data.

Show child fields
data_quality_summary.unsupported_timestamps.total_number_of_unsupported_timestamps

Indicates the total number of unsupported timestamps across the ingested data.

data_start_time

Indicates the earliest timestamp corresponding to data that was successfully ingested during the most recent ingestion of this particular dataset. 

TIMESTAMP_S
dataset_arn

The Amazon Resource Name (ARN) of the dataset being described. 

VARCHAR
ingested_files_summary

IngestedFilesSummary associated with the given dataset for the latest successful associated ingestion job id. 

STRUCT(
    "total_number_of_files" BIGINT,
    "ingested_number_of_files" BIGINT,
    "discarded_files" STRUCT(
        "bucket" VARCHAR,
        "key" VARCHAR
    )[]
)
Show child fields
ingested_files_summary.discarded_files[]
Show child fields
ingested_files_summary.discarded_files[].bucket

The name of the specific S3 bucket.

ingested_files_summary.discarded_files[].key

The Amazon Web Services Key Management Service (KMS key) key being used to encrypt the S3 object. Without this key, data in the bucket is not accessible.

ingested_files_summary.ingested_number_of_files

Indicates the number of files that were successfully ingested.

ingested_files_summary.total_number_of_files

Indicates the total number of files that were submitted for ingestion.

ingestion_input_configuration

Specifies the S3 location configuration for the data input for the data ingestion job. 

STRUCT(
    "s3_input_configuration" STRUCT(
        "bucket" VARCHAR,
        "prefix" VARCHAR,
        "key_pattern" VARCHAR
    )
)
Show child fields
ingestion_input_configuration.s3_input_configuration

The location information for the S3 bucket used for input data for the data ingestion.

Show child fields
ingestion_input_configuration.s3_input_configuration.bucket

The name of the S3 bucket used for the input data for the data ingestion.

ingestion_input_configuration.s3_input_configuration.key_pattern

The pattern for matching the Amazon S3 files that will be used for ingestion. If the schema was created previously without any KeyPattern, then the default KeyPattern {prefix}/{component_name}/* is used to download files from Amazon S3 according to the schema. This field is required when ingestion is being done for the first time.

Valid Values: {prefix}/{component_name}_* | {prefix}/{component_name}/* | {prefix}/{component_name}[DELIMITER]* (Allowed delimiters : space, dot, underscore, hyphen)

ingestion_input_configuration.s3_input_configuration.prefix

The prefix for the S3 location being used for the input data for the data ingestion.

last_updated_at

Specifies the time the dataset was last updated, if it was. 

TIMESTAMP_S
role_arn

The Amazon Resource Name (ARN) of the IAM role that you are using for this the data ingestion job. 

VARCHAR
schema

A JSON description of the data that is in each time series dataset, including names, column names, and data types. 

VARCHAR
server_side_kms_key_id

Provides the identifier of the KMS key used to encrypt dataset data by Amazon Lookout for Equipment. 

VARCHAR
source_dataset_arn

The Amazon Resource Name (ARN) of the source dataset from which the current data being described was imported from.

VARCHAR
status

Indicates the status of the dataset. 

VARCHAR