Skip to content

aws.machinelearning.describe_data_sources

Example SQL Queries

SELECT * FROM
aws.machinelearning.describe_data_sources;

Description

Returns a list of DataSource that match the search criteria in the request.

Table Definition

Column NameColumn Data Type
eq Input Column

The equal to operator. The DataSource results will have FilterVariable values that exactly match the value specified with EQ.

VARCHAR
filter_variable Input Column

Use one of the following variables to filter a list of DataSource:

  • CreatedAt - Sets the search criteria to DataSource creation dates.

  • Status - Sets the search criteria to DataSource statuses.

  • Name - Sets the search criteria to the contents of DataSource Name.

  • DataUri - Sets the search criteria to the URI of data files used to create the DataSource. The URI can identify either a file or an Amazon Simple Storage Service (Amazon S3) bucket or directory.

  • IAMUser - Sets the search criteria to the user account that invoked the DataSource creation.

VARCHAR
ge Input Column

The greater than or equal to operator. The DataSource results will have FilterVariable values that are greater than or equal to the value specified with GE.

VARCHAR
gt Input Column

The greater than operator. The DataSource results will have FilterVariable values that are greater than the value specified with GT.

VARCHAR
le Input Column

The less than or equal to operator. The DataSource results will have FilterVariable values that are less than or equal to the value specified with LE.

VARCHAR
lt Input Column

The less than operator. The DataSource results will have FilterVariable values that are less than the value specified with LT.

VARCHAR
ne Input Column

The not equal to operator. The DataSource results will have FilterVariable values not equal to the value specified with NE.

VARCHAR
prefix Input Column

A string that is found at the beginning of a variable, such as Name or Id.

For example, a DataSource could have the Name 2014-09-09-HolidayGiftMailer. To search for this DataSource, select Name for the FilterVariable and any of the following strings for the Prefix:

  • 2014-09

  • 2014-09-09

  • 2014-09-09-Holiday

VARCHAR
sort_order Input Column

A two-value parameter that determines the sequence of the resulting list of DataSource.

  • asc - Arranges the list in ascending order (A-Z, 0-9).

  • dsc - Arranges the list in descending order (Z-A, 9-0).

Results are sorted by FilterVariable.

VARCHAR
_aws_profile Input Column

The AWS profile defines the AWS identity used. It can be defined via credentials or by assuming a IAM role.

STRUCT(
"type" VARCHAR,
"name" VARCHAR,
"account_id" VARCHAR,
"via_profile_name" VARCHAR,
"assumed_role_arn" VARCHAR,
"organization" STRUCT(
"account_name" VARCHAR,
"id" VARCHAR,
"tags" STRUCT(
"key" VARCHAR,
"value" VARCHAR
)[],
"master_account" STRUCT(
"id" VARCHAR,
"email" VARCHAR
),
"parents" STRUCT(
"type" VARCHAR,
"id" VARCHAR,
"name" VARCHAR,
"tags" STRUCT(
"key" VARCHAR,
"value" VARCHAR
)[]
)[]
)
)
Show child fields
_aws_profile.account_id

The AWS account id

_aws_profile.assumed_role_arn

The ARN of the assumed role

_aws_profile.name

The unique name of the profile.

_aws_profile.organization

Information about this profile's membership in the AWS organization.

Show child fields
_aws_profile.organization.account_name

The name of account speciifed by the organization

_aws_profile.organization.id

The organization id

_aws_profile.organization.master_account
Show child fields
_aws_profile.organization.master_account.email

The organization master account email address

_aws_profile.organization.master_account.id

The organization master account id

_aws_profile.organization.parents[]
Show child fields
_aws_profile.organization.parents[].id

The id of the parent

_aws_profile.organization.parents[].name

The name of the parent

_aws_profile.organization.parents[].tags[]
Show child fields
_aws_profile.organization.parents[].tags[].key
_aws_profile.organization.parents[].tags[].value
_aws_profile.organization.parents[].type

The type of parent can be an organization unit or a root

_aws_profile.organization.tags[]
Show child fields
_aws_profile.organization.tags[].key
_aws_profile.organization.tags[].value
_aws_profile.type

The type of profile, either 'credentials' or 'assumed_role'

_aws_profile.via_profile_name

This IAM role for this profile is assumed by first utilizing another profile with this name to obtain credentials.

_aws_region Input Column

The AWS region to use.

VARCHAR
compute_statistics

The parameter is true if statistics need to be generated from the observation data.

BOOLEAN
compute_time

Long integer type that is a 64-bit signed number.

BIGINT
created_at

The time that the DataSource was created. The time is expressed in epoch time.

TIMESTAMP_S
created_by_iam_user

The AWS user account from which the DataSource was created. The account type can be either an AWS root account or an AWS Identity and Access Management (IAM) user account.

VARCHAR
data_location_s3

The location and name of the data in Amazon Simple Storage Service (Amazon S3) that is used by a DataSource.

VARCHAR
data_rearrangement

A JSON string that represents the splitting and rearrangement requirement used when this DataSource was created.

VARCHAR
data_size_in_bytes

The total number of observations contained in the data files that the DataSource references.

BIGINT
data_source_id

The ID that is assigned to the DataSource during creation.

VARCHAR
finished_at

A timestamp represented in epoch time.

TIMESTAMP_S
last_updated_at

The time of the most recent edit to the BatchPrediction. The time is expressed in epoch time.

TIMESTAMP_S
message

A description of the most recent details about creating the DataSource.

VARCHAR
name

A user-supplied name or description of the DataSource.

VARCHAR
number_of_files

The number of data files referenced by the DataSource.

BIGINT
rds_metadata

The datasource details that are specific to Amazon RDS.

STRUCT(
"database" STRUCT(
"instance_identifier" VARCHAR,
"database_name" VARCHAR
),
"database_user_name" VARCHAR,
"select_sql_query" VARCHAR,
"resource_role" VARCHAR,
"service_role" VARCHAR,
"data_pipeline_id" VARCHAR
)
Show child fields
rds_metadata.data_pipeline_id

The ID of the Data Pipeline instance that is used to carry to copy data from Amazon RDS to Amazon S3. You can use the ID to find details about the instance in the Data Pipeline console.

rds_metadata.database

The database details required to connect to an Amazon RDS.

Show child fields
rds_metadata.database.database_name

The name of a database hosted on an RDS DB instance.

rds_metadata.database.instance_identifier

The ID of an RDS DB instance.

rds_metadata.database_user_name

The username to be used by Amazon ML to connect to database on an Amazon RDS instance. The username should have sufficient permissions to execute an RDSSelectSqlQuery query.

rds_metadata.resource_role

The role (DataPipelineDefaultResourceRole) assumed by an Amazon EC2 instance to carry out the copy task from Amazon RDS to Amazon S3. For more information, see Role templates for data pipelines.

rds_metadata.select_sql_query

The SQL query that is supplied during CreateDataSourceFromRDS. Returns only if Verbose is true in GetDataSourceInput.

rds_metadata.service_role

The role (DataPipelineDefaultRole) assumed by the Data Pipeline service to monitor the progress of the copy task from Amazon RDS to Amazon S3. For more information, see Role templates for data pipelines.

redshift_metadata

Describes the DataSource details specific to Amazon Redshift.

STRUCT(
"redshift_database" STRUCT(
"database_name" VARCHAR,
"cluster_identifier" VARCHAR
),
"database_user_name" VARCHAR,
"select_sql_query" VARCHAR
)
Show child fields
redshift_metadata.database_user_name

A username to be used by Amazon Machine Learning (Amazon ML)to connect to a database on an Amazon Redshift cluster. The username should have sufficient permissions to execute the RedshiftSelectSqlQuery query. The username should be valid for an Amazon Redshift USER.

redshift_metadata.redshift_database

Describes the database details required to connect to an Amazon Redshift database.

Show child fields
redshift_metadata.redshift_database.cluster_identifier

The ID of an Amazon Redshift cluster.

redshift_metadata.redshift_database.database_name

The name of a database hosted on an Amazon Redshift cluster.

redshift_metadata.select_sql_query

The SQL query that is specified during CreateDataSourceFromRedshift. Returns only if Verbose is true in GetDataSourceInput.

role_arn

The Amazon Resource Name (ARN) of an AWS IAM Role, such as the following: arn:aws:iam::account:role/rolename.

VARCHAR
started_at

A timestamp represented in epoch time.

TIMESTAMP_S
status

The current status of the DataSource. This element can have one of the following values:

  • PENDING - Amazon Machine Learning (Amazon ML) submitted a request to create a DataSource.

  • INPROGRESS - The creation process is underway.

  • FAILED - The request to create a DataSource did not run to completion. It is not usable.

  • COMPLETED - The creation process completed successfully.

  • DELETED - The DataSource is marked as deleted. It is not usable.

VARCHAR