Skip to content
AWS for Query.Farm

aws.dynamodbstreams.get_records

Example SQL Queries

SELECT * FROM
aws.dynamodbstreams.get_records
WHERE
"shard_iterator" = 'VALUE';

Description

Retrieves the stream records from a given shard.

Specify a shard iterator using the ShardIterator parameter. The shard iterator specifies the position in the shard from which you want to start reading stream records sequentially. If there are no stream records available in the portion of the shard that the iterator points to, GetRecords returns an empty list. Note that it might take multiple calls to get to a portion of the shard that contains stream records.

GetRecords can retrieve a maximum of 1 MB of data or 1000 stream records, whichever comes first.

Table Definition

Column NameColumn Data Type
shard_iterator Required Input Column

A shard iterator that was retrieved from a previous GetShardIterator operation. This iterator can be used to access the stream records in this shard.

VARCHAR
limit Input Column

The maximum number of records to return from the shard. The upper limit is 1000.

BIGINT
_aws_profile Input Column

The AWS profile defines the AWS identity used. It can be defined via credentials or by assuming a IAM role.

STRUCT(
    "type" VARCHAR,
    "name" VARCHAR,
    "account_id" VARCHAR,
    "via_profile_name" VARCHAR,
    "assumed_role_arn" VARCHAR,
    "organization" STRUCT(
        "account_name" VARCHAR,
        "id" VARCHAR,
        "tags" STRUCT(
            "key" VARCHAR,
            "value" VARCHAR
        )[],
        "master_account" STRUCT(
            "id" VARCHAR,
            "email" VARCHAR
        ),
        "parents" STRUCT(
            "type" VARCHAR,
            "id" VARCHAR,
            "name" VARCHAR,
            "tags" STRUCT(
                "key" VARCHAR,
                "value" VARCHAR
            )[]
        )[]
    )
)
Show child fields
_aws_profile.account_id

The AWS account id

_aws_profile.assumed_role_arn

The ARN of the assumed role

_aws_profile.name

The unique name of the profile.

_aws_profile.organization

Information about this profile's membership in the AWS organization.

Show child fields
_aws_profile.organization.account_name

The name of account speciifed by the organization

_aws_profile.organization.id

The organization id

_aws_profile.organization.master_account
Show child fields
_aws_profile.organization.master_account.email

The organization master account email address

_aws_profile.organization.master_account.id

The organization master account id

_aws_profile.organization.parents[]
Show child fields
_aws_profile.organization.parents[].id

The id of the parent

_aws_profile.organization.parents[].name

The name of the parent

_aws_profile.organization.parents[].tags[]
Show child fields
_aws_profile.organization.parents[].tags[].key
_aws_profile.organization.parents[].tags[].value
_aws_profile.organization.parents[].type

The type of parent can be an organization unit or a root

_aws_profile.organization.tags[]
Show child fields
_aws_profile.organization.tags[].key
_aws_profile.organization.tags[].value
_aws_profile.type

The type of profile, either 'credentials' or 'assumed_role'

_aws_profile.via_profile_name

This IAM role for this profile is assumed by first utilizing another profile with this name to obtain credentials.

next_shard_iterator

The next position in the shard from which to start sequentially reading stream records. If set to null, the shard has been closed and the requested iterator will not return any more data.

VARCHAR
records

The stream records from the shard, which were retrieved using the shard iterator.

STRUCT(
    "event_id" VARCHAR,
    "event_name" VARCHAR,
    "event_version" VARCHAR,
    "event_source" VARCHAR,
    "aws_region" VARCHAR,
    "dynamodb" STRUCT(
        "approximate_creation_date_time" TIMESTAMP_S,
        "keys" MAP(VARCHAR, STRUCT(
                "s" VARCHAR,
                "n" VARCHAR,
                "b" BLOB,
                "ss" VARCHAR[],
                "ns" VARCHAR[],
                "bs" BLOB[],
                "null" BOOLEAN,
                "bool" BOOLEAN
            )),
        "new_image" MAP(VARCHAR, STRUCT(
                "s" VARCHAR,
                "n" VARCHAR,
                "b" BLOB,
                "ss" VARCHAR[],
                "ns" VARCHAR[],
                "bs" BLOB[],
                "null" BOOLEAN,
                "bool" BOOLEAN
            )),
        "old_image" MAP(VARCHAR, STRUCT(
                "s" VARCHAR,
                "n" VARCHAR,
                "b" BLOB,
                "ss" VARCHAR[],
                "ns" VARCHAR[],
                "bs" BLOB[],
                "null" BOOLEAN,
                "bool" BOOLEAN
            )),
        "sequence_number" VARCHAR,
        "size_bytes" BIGINT,
        "stream_view_type" VARCHAR
    ),
    "user_identity" STRUCT(
        "principal_id" VARCHAR,
        "type" VARCHAR
    )
)[]
Show child fields
records[]
Show child fields
records[].aws_region

The region in which the GetRecords request was received.

records[].dynamodb

The main body of the stream record, containing all of the DynamoDB-specific fields.

Show child fields
records[].dynamodb.approximate_creation_date_time

The approximate date and time when the stream record was created, in UNIX epoch time format and rounded down to the closest second.

records[].dynamodb.keys

The primary key attribute(s) for the DynamoDB item that was modified.

records[].dynamodb.new_image

The item in the DynamoDB table as it appeared after it was modified.

records[].dynamodb.old_image

The item in the DynamoDB table as it appeared before it was modified.

records[].dynamodb.sequence_number

The sequence number of the stream record.

records[].dynamodb.size_bytes

The size of the stream record, in bytes.

records[].dynamodb.stream_view_type

The type of data from the modified DynamoDB item that was captured in this stream record:

  • KEYS_ONLY - only the key attributes of the modified item.

  • NEW_IMAGE - the entire item, as it appeared after it was modified.

  • OLD_IMAGE - the entire item, as it appeared before it was modified.

  • NEW_AND_OLD_IMAGES - both the new and the old item images of the item.

records[].event_id

A globally unique identifier for the event that was recorded in this stream record.

records[].event_name

The type of data modification that was performed on the DynamoDB table:

  • INSERT - a new item was added to the table.

  • MODIFY - one or more of an existing item's attributes were modified.

  • REMOVE - the item was deleted from the table

records[].event_source

The Amazon Web Services service from which the stream record originated. For DynamoDB Streams, this is aws:dynamodb.

records[].event_version

The version number of the stream record format. This number is updated whenever the structure of Record is modified.

Client applications must not assume that eventVersion will remain at a particular value, as this number is subject to change at any time. In general, eventVersion will only increase as the low-level DynamoDB Streams API evolves.

records[].user_identity

Items that are deleted by the Time to Live process after expiration have the following fields:

  • Records[].userIdentity.type

    "Service"

  • Records[].userIdentity.principalId

    "dynamodb.amazonaws.com"

Show child fields
records[].user_identity.principal_id

A unique identifier for the entity that made the call. For Time To Live, the principalId is "dynamodb.amazonaws.com".

records[].user_identity.type

The type of the identity. For Time To Live, the type is "Service".