Skip to content
AWS for Query.Farm

aws.bedrock_agent.get_data_source

Example SQL Queries

SELECT * FROM
aws.bedrock_agent.get_data_source
WHERE
"data_source_id" = 'VALUE'
AND "knowledge_base_id" = 'VALUE';

Description

Gets information about a data source.

Table Definition

Column NameColumn Data Type
data_source_id Required Input Column

The unique identifier of the data source.

VARCHAR
knowledge_base_id Required Input Column

The unique identifier of the knowledge base that the data source was added to.

VARCHAR
_aws_profile Input Column

The AWS profile defines the AWS identity used. It can be defined via credentials or by assuming a IAM role.

STRUCT(
    "type" VARCHAR,
    "name" VARCHAR,
    "account_id" VARCHAR,
    "via_profile_name" VARCHAR,
    "assumed_role_arn" VARCHAR,
    "organization" STRUCT(
        "account_name" VARCHAR,
        "id" VARCHAR,
        "tags" STRUCT(
            "key" VARCHAR,
            "value" VARCHAR
        )[],
        "master_account" STRUCT(
            "id" VARCHAR,
            "email" VARCHAR
        ),
        "parents" STRUCT(
            "type" VARCHAR,
            "id" VARCHAR,
            "name" VARCHAR,
            "tags" STRUCT(
                "key" VARCHAR,
                "value" VARCHAR
            )[]
        )[]
    )
)
Show child fields
_aws_profile.account_id

The AWS account id

_aws_profile.assumed_role_arn

The ARN of the assumed role

_aws_profile.name

The unique name of the profile.

_aws_profile.organization

Information about this profile's membership in the AWS organization.

Show child fields
_aws_profile.organization.account_name

The name of account speciifed by the organization

_aws_profile.organization.id

The organization id

_aws_profile.organization.master_account
Show child fields
_aws_profile.organization.master_account.email

The organization master account email address

_aws_profile.organization.master_account.id

The organization master account id

_aws_profile.organization.parents[]
Show child fields
_aws_profile.organization.parents[].id

The id of the parent

_aws_profile.organization.parents[].name

The name of the parent

_aws_profile.organization.parents[].tags[]
Show child fields
_aws_profile.organization.parents[].tags[].key
_aws_profile.organization.parents[].tags[].value
_aws_profile.organization.parents[].type

The type of parent can be an organization unit or a root

_aws_profile.organization.tags[]
Show child fields
_aws_profile.organization.tags[].key
_aws_profile.organization.tags[].value
_aws_profile.type

The type of profile, either 'credentials' or 'assumed_role'

_aws_profile.via_profile_name

This IAM role for this profile is assumed by first utilizing another profile with this name to obtain credentials.

data_source

Contains details about the data source.

STRUCT(
    "created_at" TIMESTAMP_S,
    "data_deletion_policy" VARCHAR,
    "data_source_configuration" STRUCT(
        "confluence_configuration" STRUCT(
            "crawler_configuration" STRUCT(
                "filter_configuration" STRUCT(
                    "pattern_object_filter" STRUCT(
                        "filters" STRUCT(
                            "exclusion_filters" VARCHAR[],
                            "inclusion_filters" VARCHAR[],
                            "object_type" VARCHAR
                        )[]
                    ),
                    "type" VARCHAR
                )
            ),
            "source_configuration" STRUCT(
                "auth_type" VARCHAR,
                "credentials_secret_arn" VARCHAR,
                "host_type" VARCHAR,
                "host_url" VARCHAR
            )
        ),
        "s3_configuration" STRUCT(
            "bucket_arn" VARCHAR,
            "bucket_owner_account_id" VARCHAR,
            "inclusion_prefixes" VARCHAR[]
        ),
        "salesforce_configuration" STRUCT(
            "crawler_configuration" STRUCT(
                "filter_configuration" STRUCT(
                    "pattern_object_filter" STRUCT(
                        "filters" STRUCT(
                            "exclusion_filters" VARCHAR[],
                            "inclusion_filters" VARCHAR[],
                            "object_type" VARCHAR
                        )[]
                    ),
                    "type" VARCHAR
                )
            ),
            "source_configuration" STRUCT(
                "auth_type" VARCHAR,
                "credentials_secret_arn" VARCHAR,
                "host_url" VARCHAR
            )
        ),
        "share_point_configuration" STRUCT(
            "crawler_configuration" STRUCT(
                "filter_configuration" STRUCT(
                    "pattern_object_filter" STRUCT(
                        "filters" STRUCT(
                            "exclusion_filters" VARCHAR[],
                            "inclusion_filters" VARCHAR[],
                            "object_type" VARCHAR
                        )[]
                    ),
                    "type" VARCHAR
                )
            ),
            "source_configuration" STRUCT(
                "auth_type" VARCHAR,
                "credentials_secret_arn" VARCHAR,
                "domain" VARCHAR,
                "host_type" VARCHAR,
                "site_urls" VARCHAR[],
                "tenant_id" VARCHAR
            )
        ),
        "type" VARCHAR,
        "web_configuration" STRUCT(
            "crawler_configuration" STRUCT(
                "crawler_limits" STRUCT(
                    "rate_limit" BIGINT
                ),
                "exclusion_filters" VARCHAR[],
                "inclusion_filters" VARCHAR[],
                "scope" VARCHAR
            ),
            "source_configuration" STRUCT(
                "url_configuration" STRUCT(
                    "seed_urls" STRUCT(
                        "url" VARCHAR
                    )[]
                )
            )
        )
    ),
    "data_source_id" VARCHAR,
    "description" VARCHAR,
    "failure_reasons" VARCHAR[],
    "knowledge_base_id" VARCHAR,
    "name" VARCHAR,
    "server_side_encryption_configuration" STRUCT(
        "kms_key_arn" VARCHAR
    ),
    "status" VARCHAR,
    "updated_at" TIMESTAMP_S,
    "vector_ingestion_configuration" STRUCT(
        "chunking_configuration" STRUCT(
            "chunking_strategy" VARCHAR,
            "fixed_size_chunking_configuration" STRUCT(
                "max_tokens" BIGINT,
                "overlap_percentage" BIGINT
            ),
            "hierarchical_chunking_configuration" STRUCT(
                "level_configurations" STRUCT(
                    "max_tokens" BIGINT
                )[],
                "overlap_tokens" BIGINT
            ),
            "semantic_chunking_configuration" STRUCT(
                "breakpoint_percentile_threshold" BIGINT,
                "buffer_size" BIGINT,
                "max_tokens" BIGINT
            )
        ),
        "custom_transformation_configuration" STRUCT(
            "intermediate_storage" STRUCT(
                "s3_location" STRUCT(
                    "uri" VARCHAR
                )
            ),
            "transformations" STRUCT(
                "step_to_apply" VARCHAR,
                "transformation_function" STRUCT(
                    "transformation_lambda_configuration" STRUCT(
                        "lambda_arn" VARCHAR
                    )
                )
            )[]
        ),
        "parsing_configuration" STRUCT(
            "bedrock_foundation_model_configuration" STRUCT(
                "model_arn" VARCHAR,
                "parsing_prompt" STRUCT(
                    "parsing_prompt_text" VARCHAR
                )
            ),
            "parsing_strategy" VARCHAR
        )
    )
)
Show child fields
data_source.created_at

The time at which the data source was created.

data_source.data_deletion_policy

The data deletion policy for the data source.

data_source.data_source_configuration

The connection configuration for the data source.

Show child fields
data_source.data_source_configuration.confluence_configuration

The configuration information to connect to Confluence as your data source.

Confluence data source connector is in preview release and is subject to change.

Show child fields
data_source.data_source_configuration.confluence_configuration.crawler_configuration

The configuration of the Confluence content. For example, configuring specific types of Confluence content.

Show child fields
data_source.data_source_configuration.confluence_configuration.crawler_configuration.filter_configuration

The configuration of filtering the Confluence content. For example, configuring regular expression patterns to include or exclude certain content.

Show child fields
data_source.data_source_configuration.confluence_configuration.crawler_configuration.filter_configuration.pattern_object_filter

The configuration of filtering certain objects or content types of the data source.

Show child fields
data_source.data_source_configuration.confluence_configuration.crawler_configuration.filter_configuration.pattern_object_filter.filters[]
Show child fields
data_source.data_source_configuration.confluence_configuration.crawler_configuration.filter_configuration.pattern_object_filter.filters[].exclusion_filters[]
data_source.data_source_configuration.confluence_configuration.crawler_configuration.filter_configuration.pattern_object_filter.filters[].inclusion_filters[]
data_source.data_source_configuration.confluence_configuration.crawler_configuration.filter_configuration.pattern_object_filter.filters[].object_type

The supported object type or content type of the data source.

data_source.data_source_configuration.confluence_configuration.crawler_configuration.filter_configuration.type

The type of filtering that you want to apply to certain objects or content of the data source. For example, the PATTERN type is regular expression patterns you can apply to filter your content.

data_source.data_source_configuration.confluence_configuration.source_configuration

The endpoint information to connect to your Confluence data source.

Show child fields
data_source.data_source_configuration.confluence_configuration.source_configuration.auth_type

The supported authentication type to authenticate and connect to your Confluence instance.

data_source.data_source_configuration.confluence_configuration.source_configuration.credentials_secret_arn

The Amazon Resource Name of an Secrets Manager secret that stores your authentication credentials for your Confluence instance URL. For more information on the key-value pairs that must be included in your secret, depending on your authentication type, see Confluence connection configuration.

data_source.data_source_configuration.confluence_configuration.source_configuration.host_type

The supported host type, whether online/cloud or server/on-premises.

data_source.data_source_configuration.confluence_configuration.source_configuration.host_url

The Confluence host URL or instance URL.

data_source.data_source_configuration.s3_configuration

The configuration information to connect to Amazon S3 as your data source.

Show child fields
data_source.data_source_configuration.s3_configuration.bucket_arn

The Amazon Resource Name (ARN) of the S3 bucket that contains your data.

data_source.data_source_configuration.s3_configuration.bucket_owner_account_id

The account ID for the owner of the S3 bucket.

data_source.data_source_configuration.s3_configuration.inclusion_prefixes[]
data_source.data_source_configuration.salesforce_configuration

The configuration information to connect to Salesforce as your data source.

Salesforce data source connector is in preview release and is subject to change.

Show child fields
data_source.data_source_configuration.salesforce_configuration.crawler_configuration

The configuration of the Salesforce content. For example, configuring specific types of Salesforce content.

Show child fields
data_source.data_source_configuration.salesforce_configuration.crawler_configuration.filter_configuration

The configuration of filtering the Salesforce content. For example, configuring regular expression patterns to include or exclude certain content.

Show child fields
data_source.data_source_configuration.salesforce_configuration.crawler_configuration.filter_configuration.pattern_object_filter

The configuration of filtering certain objects or content types of the data source.

Show child fields
data_source.data_source_configuration.salesforce_configuration.crawler_configuration.filter_configuration.pattern_object_filter.filters[]
Show child fields
data_source.data_source_configuration.salesforce_configuration.crawler_configuration.filter_configuration.pattern_object_filter.filters[].exclusion_filters[]
data_source.data_source_configuration.salesforce_configuration.crawler_configuration.filter_configuration.pattern_object_filter.filters[].inclusion_filters[]
data_source.data_source_configuration.salesforce_configuration.crawler_configuration.filter_configuration.pattern_object_filter.filters[].object_type

The supported object type or content type of the data source.

data_source.data_source_configuration.salesforce_configuration.crawler_configuration.filter_configuration.type

The type of filtering that you want to apply to certain objects or content of the data source. For example, the PATTERN type is regular expression patterns you can apply to filter your content.

data_source.data_source_configuration.salesforce_configuration.source_configuration

The endpoint information to connect to your Salesforce data source.

Show child fields
data_source.data_source_configuration.salesforce_configuration.source_configuration.auth_type

The supported authentication type to authenticate and connect to your Salesforce instance.

data_source.data_source_configuration.salesforce_configuration.source_configuration.credentials_secret_arn

The Amazon Resource Name of an Secrets Manager secret that stores your authentication credentials for your Salesforce instance URL. For more information on the key-value pairs that must be included in your secret, depending on your authentication type, see Salesforce connection configuration.

data_source.data_source_configuration.salesforce_configuration.source_configuration.host_url

The Salesforce host URL or instance URL.

data_source.data_source_configuration.share_point_configuration

The configuration information to connect to SharePoint as your data source.

SharePoint data source connector is in preview release and is subject to change.

Show child fields
data_source.data_source_configuration.share_point_configuration.crawler_configuration

The configuration of the SharePoint content. For example, configuring specific types of SharePoint content.

Show child fields
data_source.data_source_configuration.share_point_configuration.crawler_configuration.filter_configuration

The configuration of filtering the SharePoint content. For example, configuring regular expression patterns to include or exclude certain content.

Show child fields
data_source.data_source_configuration.share_point_configuration.crawler_configuration.filter_configuration.pattern_object_filter

The configuration of filtering certain objects or content types of the data source.

Show child fields
data_source.data_source_configuration.share_point_configuration.crawler_configuration.filter_configuration.pattern_object_filter.filters[]
Show child fields
data_source.data_source_configuration.share_point_configuration.crawler_configuration.filter_configuration.pattern_object_filter.filters[].exclusion_filters[]
data_source.data_source_configuration.share_point_configuration.crawler_configuration.filter_configuration.pattern_object_filter.filters[].inclusion_filters[]
data_source.data_source_configuration.share_point_configuration.crawler_configuration.filter_configuration.pattern_object_filter.filters[].object_type

The supported object type or content type of the data source.

data_source.data_source_configuration.share_point_configuration.crawler_configuration.filter_configuration.type

The type of filtering that you want to apply to certain objects or content of the data source. For example, the PATTERN type is regular expression patterns you can apply to filter your content.

data_source.data_source_configuration.share_point_configuration.source_configuration

The endpoint information to connect to your SharePoint data source.

Show child fields
data_source.data_source_configuration.share_point_configuration.source_configuration.auth_type

The supported authentication type to authenticate and connect to your SharePoint site/sites.

data_source.data_source_configuration.share_point_configuration.source_configuration.credentials_secret_arn

The Amazon Resource Name of an Secrets Manager secret that stores your authentication credentials for your SharePoint site/sites. For more information on the key-value pairs that must be included in your secret, depending on your authentication type, see SharePoint connection configuration.

data_source.data_source_configuration.share_point_configuration.source_configuration.domain

The domain of your SharePoint instance or site URL/URLs.

data_source.data_source_configuration.share_point_configuration.source_configuration.host_type

The supported host type, whether online/cloud or server/on-premises.

data_source.data_source_configuration.share_point_configuration.source_configuration.site_urls[]
data_source.data_source_configuration.share_point_configuration.source_configuration.tenant_id

The identifier of your Microsoft 365 tenant.

data_source.data_source_configuration.type

The type of data source.

data_source.data_source_configuration.web_configuration

The configuration of web URLs to crawl for your data source. You should be authorized to crawl the URLs.

Crawling web URLs as your data source is in preview release and is subject to change.

Show child fields
data_source.data_source_configuration.web_configuration.crawler_configuration

The Web Crawler configuration details for the web data source.

Show child fields
data_source.data_source_configuration.web_configuration.crawler_configuration.crawler_limits

The configuration of crawl limits for the web URLs.

Show child fields
data_source.data_source_configuration.web_configuration.crawler_configuration.crawler_limits.rate_limit

The max rate at which pages are crawled, up to 300 per minute per host.

data_source.data_source_configuration.web_configuration.crawler_configuration.exclusion_filters[]
data_source.data_source_configuration.web_configuration.crawler_configuration.inclusion_filters[]
data_source.data_source_configuration.web_configuration.crawler_configuration.scope

The scope of what is crawled for your URLs.

You can choose to crawl only web pages that belong to the same host or primary domain. For example, only web pages that contain the seed URL "https://docs.aws.amazon.com/bedrock/latest/userguide/" and no other domains. You can choose to include sub domains in addition to the host or primary domain. For example, web pages that contain "aws.amazon.com" can also include sub domain "docs.aws.amazon.com".

data_source.data_source_configuration.web_configuration.source_configuration

The source configuration details for the web data source.

Show child fields
data_source.data_source_configuration.web_configuration.source_configuration.url_configuration

The configuration of the URL/URLs.

Show child fields
data_source.data_source_configuration.web_configuration.source_configuration.url_configuration.seed_urls[]
Show child fields
data_source.data_source_configuration.web_configuration.source_configuration.url_configuration.seed_urls[].url

A seed or starting point URL.

data_source.data_source_id

The unique identifier of the data source.

data_source.description

The description of the data source.

data_source.failure_reasons[]
data_source.knowledge_base_id

The unique identifier of the knowledge base to which the data source belongs.

data_source.name

The name of the data source.

data_source.server_side_encryption_configuration

Contains details about the configuration of the server-side encryption.

Show child fields
data_source.server_side_encryption_configuration.kms_key_arn

The Amazon Resource Name (ARN) of the KMS key used to encrypt the resource.

data_source.status

The status of the data source. The following statuses are possible:

  • Available – The data source has been created and is ready for ingestion into the knowledge base.

  • Deleting – The data source is being deleted.

data_source.updated_at

The time at which the data source was last updated.

data_source.vector_ingestion_configuration

Contains details about how to ingest the documents in the data source.

Show child fields
data_source.vector_ingestion_configuration.chunking_configuration

Details about how to chunk the documents in the data source. A chunk refers to an excerpt from a data source that is returned when the knowledge base that it belongs to is queried.

Show child fields
data_source.vector_ingestion_configuration.chunking_configuration.chunking_strategy

Knowledge base can split your source data into chunks. A chunk refers to an excerpt from a data source that is returned when the knowledge base that it belongs to is queried. You have the following options for chunking your data. If you opt for NONE, then you may want to pre-process your files by splitting them up such that each file corresponds to a chunk.

  • FIXED_SIZE – Amazon Bedrock splits your source data into chunks of the approximate size that you set in the fixedSizeChunkingConfiguration.

  • HIERARCHICAL – Split documents into layers of chunks where the first layer contains large chunks, and the second layer contains smaller chunks derived from the first layer.

  • SEMANTIC – Split documents into chunks based on groups of similar content derived with natural language processing.

  • NONE – Amazon Bedrock treats each file as one chunk. If you choose this option, you may want to pre-process your documents by splitting them into separate files.

data_source.vector_ingestion_configuration.chunking_configuration.fixed_size_chunking_configuration

Configurations for when you choose fixed-size chunking. If you set the chunkingStrategy as NONE, exclude this field.

Show child fields
data_source.vector_ingestion_configuration.chunking_configuration.fixed_size_chunking_configuration.max_tokens

The maximum number of tokens to include in a chunk.

data_source.vector_ingestion_configuration.chunking_configuration.fixed_size_chunking_configuration.overlap_percentage

The percentage of overlap between adjacent chunks of a data source.

data_source.vector_ingestion_configuration.chunking_configuration.hierarchical_chunking_configuration

Settings for hierarchical document chunking for a data source. Hierarchical chunking splits documents into layers of chunks where the first layer contains large chunks, and the second layer contains smaller chunks derived from the first layer.

Show child fields
data_source.vector_ingestion_configuration.chunking_configuration.hierarchical_chunking_configuration.level_configurations[]
Show child fields
data_source.vector_ingestion_configuration.chunking_configuration.hierarchical_chunking_configuration.level_configurations[].max_tokens

The maximum number of tokens that a chunk can contain in this layer.

data_source.vector_ingestion_configuration.chunking_configuration.hierarchical_chunking_configuration.overlap_tokens

The number of tokens to repeat across chunks in the same layer.

data_source.vector_ingestion_configuration.chunking_configuration.semantic_chunking_configuration

Settings for semantic document chunking for a data source. Semantic chunking splits a document into into smaller documents based on groups of similar content derived from the text with natural language processing.

Show child fields
data_source.vector_ingestion_configuration.chunking_configuration.semantic_chunking_configuration.breakpoint_percentile_threshold

The dissimilarity threshold for splitting chunks.

data_source.vector_ingestion_configuration.chunking_configuration.semantic_chunking_configuration.buffer_size

The buffer size.

data_source.vector_ingestion_configuration.chunking_configuration.semantic_chunking_configuration.max_tokens

The maximum number of tokens that a chunk can contain.

data_source.vector_ingestion_configuration.custom_transformation_configuration

A custom document transformer for parsed data source documents.

Show child fields
data_source.vector_ingestion_configuration.custom_transformation_configuration.intermediate_storage

An S3 bucket path for input and output objects.

Show child fields
data_source.vector_ingestion_configuration.custom_transformation_configuration.intermediate_storage.s3_location

An S3 bucket path.

Show child fields
data_source.vector_ingestion_configuration.custom_transformation_configuration.intermediate_storage.s3_location.uri

The location's URI. For example, s3://my-bucket/chunk-processor/.

data_source.vector_ingestion_configuration.custom_transformation_configuration.transformations[]
Show child fields
data_source.vector_ingestion_configuration.custom_transformation_configuration.transformations[].step_to_apply

When the service applies the transformation.

data_source.vector_ingestion_configuration.custom_transformation_configuration.transformations[].transformation_function

A Lambda function that processes documents.

Show child fields
data_source.vector_ingestion_configuration.custom_transformation_configuration.transformations[].transformation_function.transformation_lambda_configuration

The Lambda function.

Show child fields
data_source.vector_ingestion_configuration.custom_transformation_configuration.transformations[].transformation_function.transformation_lambda_configuration.lambda_arn

The function's ARN identifier.

data_source.vector_ingestion_configuration.parsing_configuration

A custom parser for data source documents.

Show child fields
data_source.vector_ingestion_configuration.parsing_configuration.bedrock_foundation_model_configuration

Settings for a foundation model used to parse documents for a data source.

Show child fields
data_source.vector_ingestion_configuration.parsing_configuration.bedrock_foundation_model_configuration.model_arn

The model's ARN.

data_source.vector_ingestion_configuration.parsing_configuration.bedrock_foundation_model_configuration.parsing_prompt

Instructions for interpreting the contents of a document.

Show child fields
data_source.vector_ingestion_configuration.parsing_configuration.bedrock_foundation_model_configuration.parsing_prompt.parsing_prompt_text

Instructions for interpreting the contents of a document.

data_source.vector_ingestion_configuration.parsing_configuration.parsing_strategy

The parsing strategy for the data source.