Skip to content
AWS for Query.Farm

aws.rekognition.get_label_detection

Example SQL Queries

SELECT * FROM
aws.rekognition.get_label_detection
WHERE
"job_id" = 'VALUE';

Description

Gets the label detection results of a Amazon Rekognition Video analysis started by StartLabelDetection.

The label detection operation is started by a call to StartLabelDetection which returns a job identifier (JobId). When the label detection operation finishes, Amazon Rekognition publishes a completion status to the Amazon Simple Notification Service topic registered in the initial call to StartlabelDetection.

To get the results of the label detection operation, first check that the status value published to the Amazon SNS topic is SUCCEEDED. If so, call GetLabelDetection and pass the job identifier (JobId) from the initial call to StartLabelDetection.

GetLabelDetection returns an array of detected labels (Labels) sorted by the time the labels were detected. You can also sort by the label name by specifying NAME for the SortBy input parameter. If there is no NAME specified, the default sort is by timestamp.

You can select how results are aggregated by using the AggregateBy input parameter. The default aggregation method is TIMESTAMPS. You can also aggregate by SEGMENTS, which aggregates all instances of labels detected in a given segment.

The returned Labels array may include the following attributes:

  • Name - The name of the detected label.

  • Confidence - The level of confidence in the label assigned to a detected object.

  • Parents - The ancestor labels for a detected label. GetLabelDetection returns a hierarchical taxonomy of detected labels. For example, a detected car might be assigned the label car. The label car has two parent labels: Vehicle (its parent) and Transportation (its grandparent). The response includes the all ancestors for a label, where every ancestor is a unique label. In the previous example, Car, Vehicle, and Transportation are returned as unique labels in the response.

  • Aliases - Possible Aliases for the label.

  • Categories - The label categories that the detected label belongs to.

  • BoundingBox — Bounding boxes are described for all instances of detected common object labels, returned in an array of Instance objects. An Instance object contains a BoundingBox object, describing the location of the label on the input image. It also includes the confidence for the accuracy of the detected bounding box.

  • Timestamp - Time, in milliseconds from the start of the video, that the label was detected. For aggregation by SEGMENTS, the StartTimestampMillis, EndTimestampMillis, and DurationMillis structures are what define a segment. Although the “Timestamp” structure is still returned with each label, its value is set to be the same as StartTimestampMillis.

Timestamp and Bounding box information are returned for detected Instances, only if aggregation is done by TIMESTAMPS. If aggregating by SEGMENTS, information about detected instances isn’t returned.

The version of the label model used for the detection is also returned.

Note DominantColors isn't returned for Instances, although it is shown as part of the response in the sample seen below.

Use MaxResults parameter to limit the number of labels returned. If there are more results than specified in MaxResults, the value of NextToken in the operation response contains a pagination token for getting the next set of results. To get the next page of results, call GetlabelDetection and populate the NextToken request parameter with the token value returned from the previous call to GetLabelDetection.

If you are retrieving results while using the Amazon Simple Notification Service, note that you will receive an "ERROR" notification if the job encounters an issue.

Table Definition

Column NameColumn Data Type
job_id Required Input Column

Job identifier for the label detection operation for which you want to obtain results. The job identifer is returned by an initial call to StartLabelDetection.

VARCHAR
aggregate_by Input Column

Defines how to aggregate the returned results. Results can be aggregated by timestamps or segments.

VARCHAR
max_results Input Column

Maximum number of results to return per paginated call. The largest value you can specify is 1000. If you specify a value greater than 1000, a maximum of 1000 results is returned. The default value is 1000.

BIGINT
next_token Input Column

If the response is truncated, Amazon Rekognition Video returns this token that you can use in the subsequent request to retrieve the next set of labels.

VARCHAR
sort_by Input Column

Sort to use for elements in the Labels array. Use TIMESTAMP to sort array elements by the time labels are detected. Use NAME to alphabetically group elements for a label together. Within each label group, the array element are sorted by detection confidence. The default sort is by TIMESTAMP.

VARCHAR
_aws_profile Input Column

The AWS profile defines the AWS identity used. It can be defined via credentials or by assuming a IAM role.

STRUCT(
    "type" VARCHAR,
    "name" VARCHAR,
    "account_id" VARCHAR,
    "via_profile_name" VARCHAR,
    "assumed_role_arn" VARCHAR,
    "organization" STRUCT(
        "account_name" VARCHAR,
        "id" VARCHAR,
        "tags" STRUCT(
            "key" VARCHAR,
            "value" VARCHAR
        )[],
        "master_account" STRUCT(
            "id" VARCHAR,
            "email" VARCHAR
        ),
        "parents" STRUCT(
            "type" VARCHAR,
            "id" VARCHAR,
            "name" VARCHAR,
            "tags" STRUCT(
                "key" VARCHAR,
                "value" VARCHAR
            )[]
        )[]
    )
)
Show child fields
_aws_profile.account_id

The AWS account id

_aws_profile.assumed_role_arn

The ARN of the assumed role

_aws_profile.name

The unique name of the profile.

_aws_profile.organization

Information about this profile's membership in the AWS organization.

Show child fields
_aws_profile.organization.account_name

The name of account speciifed by the organization

_aws_profile.organization.id

The organization id

_aws_profile.organization.master_account
Show child fields
_aws_profile.organization.master_account.email

The organization master account email address

_aws_profile.organization.master_account.id

The organization master account id

_aws_profile.organization.parents[]
Show child fields
_aws_profile.organization.parents[].id

The id of the parent

_aws_profile.organization.parents[].name

The name of the parent

_aws_profile.organization.parents[].tags[]
Show child fields
_aws_profile.organization.parents[].tags[].key
_aws_profile.organization.parents[].tags[].value
_aws_profile.organization.parents[].type

The type of parent can be an organization unit or a root

_aws_profile.organization.tags[]
Show child fields
_aws_profile.organization.tags[].key
_aws_profile.organization.tags[].value
_aws_profile.type

The type of profile, either 'credentials' or 'assumed_role'

_aws_profile.via_profile_name

This IAM role for this profile is assumed by first utilizing another profile with this name to obtain credentials.

_aws_region Input Column

The AWS region to use.

VARCHAR
get_request_metadata

Information about the paramters used when getting a response. Includes information on aggregation and sorting methods.

STRUCT(
    "sort_by" VARCHAR,
    "aggregate_by" VARCHAR
)
Show child fields
get_request_metadata.aggregate_by

The aggregation method chosen for a GetLabelDetection request.

get_request_metadata.sort_by

The sorting method chosen for a GetLabelDetection request.

job_status

The current status of the label detection job.

VARCHAR
job_tag

A job identifier specified in the call to StartLabelDetection and returned in the job completion notification sent to your Amazon Simple Notification Service topic.

VARCHAR
label_model_version

Version number of the label detection model that was used to detect labels.

VARCHAR
labels

An array of labels detected in the video. Each element contains the detected label and the time, in milliseconds from the start of the video, that the label was detected. 

STRUCT(
    "timestamp" BIGINT,
    "label" STRUCT(
        "name" VARCHAR,
        "confidence" DOUBLE,
        "instances" STRUCT(
            "bounding_box" STRUCT(
                "width" DOUBLE,
                "height" DOUBLE,
                "left" DOUBLE,
                "top" DOUBLE
            ),
            "confidence" DOUBLE,
            "dominant_colors" STRUCT(
                "red" BIGINT,
                "blue" BIGINT,
                "green" BIGINT,
                "hex_code" VARCHAR,
                "css_color" VARCHAR,
                "simplified_color" VARCHAR,
                "pixel_percent" DOUBLE
            )[]
        )[],
        "parents" STRUCT(
            "name" VARCHAR
        )[],
        "aliases" STRUCT(
            "name" VARCHAR
        )[],
        "categories" STRUCT(
            "name" VARCHAR
        )[]
    ),
    "start_timestamp_millis" BIGINT,
    "end_timestamp_millis" BIGINT,
    "duration_millis" BIGINT
)[]
Show child fields
labels[]
Show child fields
labels[].duration_millis

The time duration of a segment in milliseconds, I.e. time elapsed from StartTimestampMillis to EndTimestampMillis.

labels[].end_timestamp_millis

The time in milliseconds defining the end of the timeline segment containing a continuously detected label.

labels[].label

Details about the detected label.

Show child fields
labels[].label.aliases[]
Show child fields
labels[].label.aliases[].name

The name of an alias for a given label.

labels[].label.categories[]
Show child fields
labels[].label.categories[].name

The name of a category that applies to a given label.

labels[].label.confidence

Level of confidence.

labels[].label.instances[]
Show child fields
labels[].label.instances[].bounding_box

The position of the label instance on the image.

Show child fields
labels[].label.instances[].bounding_box.height

Height of the bounding box as a ratio of the overall image height.

labels[].label.instances[].bounding_box.left

Left coordinate of the bounding box as a ratio of overall image width.

labels[].label.instances[].bounding_box.top

Top coordinate of the bounding box as a ratio of overall image height.

labels[].label.instances[].bounding_box.width

Width of the bounding box as a ratio of the overall image width.

labels[].label.instances[].confidence

The confidence that Amazon Rekognition has in the accuracy of the bounding box.

labels[].label.instances[].dominant_colors[]
Show child fields
labels[].label.instances[].dominant_colors[].blue

The Blue RGB value for a dominant color.

labels[].label.instances[].dominant_colors[].css_color

The CSS color name of a dominant color.

labels[].label.instances[].dominant_colors[].green

The Green RGB value for a dominant color.

labels[].label.instances[].dominant_colors[].hex_code

The Hex code equivalent of the RGB values for a dominant color.

labels[].label.instances[].dominant_colors[].pixel_percent

The percentage of image pixels that have a given dominant color.

labels[].label.instances[].dominant_colors[].red

The Red RGB value for a dominant color.

labels[].label.instances[].dominant_colors[].simplified_color

One of 12 simplified color names applied to a dominant color.

labels[].label.name

The name (label) of the object or scene.

labels[].label.parents[]
Show child fields
labels[].label.parents[].name

The name of the parent label.

labels[].start_timestamp_millis

The time in milliseconds defining the start of the timeline segment containing a continuously detected label.

labels[].timestamp

Time, in milliseconds from the start of the video, that the label was detected. Note that Timestamp is not guaranteed to be accurate to the individual frame where the label first appears.

status_message

If the job fails, StatusMessage provides a descriptive error message.

VARCHAR
video

Video file stored in an Amazon S3 bucket. Amazon Rekognition video start operations such as StartLabelDetection use Video to specify a video for analysis. The supported file formats are .mp4, .mov and .avi.

STRUCT(
    "s3_object" STRUCT(
        "bucket" VARCHAR,
        "name" VARCHAR,
        "version" VARCHAR
    )
)
Show child fields
video.s3_object

The Amazon S3 bucket name and file name for the video.

Show child fields
video.s3_object.bucket

Name of the S3 bucket.

video.s3_object.name

S3 object key name.

video.s3_object.version

If the bucket is versioning enabled, you can specify the object version.

video_metadata

Information about a video that Amazon Rekognition Video analyzed. Videometadata is returned in every page of paginated responses from a Amazon Rekognition video operation.

STRUCT(
    "codec" VARCHAR,
    "duration_millis" BIGINT,
    "format" VARCHAR,
    "frame_rate" DOUBLE,
    "frame_height" BIGINT,
    "frame_width" BIGINT,
    "color_range" VARCHAR
)
Show child fields
video_metadata.codec

Type of compression used in the analyzed video.

video_metadata.color_range

A description of the range of luminance values in a video, either LIMITED (16 to 235) or FULL (0 to 255).

video_metadata.duration_millis

Length of the video in milliseconds.

video_metadata.format

Format of the analyzed video. Possible values are MP4, MOV and AVI.

video_metadata.frame_height

Vertical pixel dimension of the video.

video_metadata.frame_rate

Number of frames per second in the video.

video_metadata.frame_width

Horizontal pixel dimension of the video.