boto3_kinesis

Connection module for Amazon Kinesis using boto3.

Renamed from boto_kinesis to boto3_kinesis and rewritten to use the boto3 kinesis client APIs directly via saltext.boto3.utils.boto3mod. The legacy boto2 code path (object-style access, retry loops) has been removed.

depends:

• boto3 >= 1.28.0

• botocore >= 1.31.0

configuration:

This module accepts explicit kinesis credentials but can also utilize IAM roles assigned to the instance through Instance Profiles. Dynamic credentials are then automatically obtained from AWS API and no further configuration is necessary. More Information available at:

http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/iam-roles-for-amazon-ec2.html

If IAM roles are not used you need to specify them either in the minion’s config file or as a profile. For example, to specify them in the minion’s config file:

kinesis.keyid: GKTADJGHEIQSXMKKRBJ08H
kinesis.key: askdjghsdfjkghWupUjasdflkdfklgjsdfjajkghs

A region may also be specified in the configuration:

kinesis.region: us-east-1

It’s also possible to specify key, keyid and region via a profile, either as a passed in dict, or as a string to pull from pillars or minion config:

myprofile:
    keyid: GKTADJGHEIQSXMKKRBJ08H
    key: askdjghsdfjkghWupUjasdflkdfklgjsdfjajkghs
    region: us-east-1

Added in version 1.0.0.

saltext.boto3.modules.boto3_kinesis.__virtual__()

Only load if boto3 is available.

saltext.boto3.modules.boto3_kinesis.get_stream_when_active(stream_name, region=None, key=None, keyid=None, profile=None)

Get complete stream info from AWS, returning only when the stream is in the ACTIVE state. Continues to retry when stream is updating or creating. If the stream is deleted during retries, the loop will catch the error and break.

CLI Example:

salt myminion boto3_kinesis.get_stream_when_active my_stream region=us-east-1

saltext.boto3.modules.boto3_kinesis.exists(stream_name, region=None, key=None, keyid=None, profile=None)

Check if the stream exists. Returns False and the error if it does not.

CLI Example:

salt myminion boto3_kinesis.exists my_stream region=us-east-1

saltext.boto3.modules.boto3_kinesis.create_stream(stream_name, num_shards, region=None, key=None, keyid=None, profile=None)

Create a stream with name stream_name and initial number of shards num_shards.

CLI Example:

salt myminion boto3_kinesis.create_stream my_stream N region=us-east-1

saltext.boto3.modules.boto3_kinesis.delete_stream(stream_name, region=None, key=None, keyid=None, profile=None)

Delete the stream with name stream_name. This cannot be undone! All data will be lost!!

CLI Example:

salt myminion boto3_kinesis.delete_stream my_stream region=us-east-1

saltext.boto3.modules.boto3_kinesis.increase_stream_retention_period(stream_name, retention_hours, region=None, key=None, keyid=None, profile=None)

Increase stream retention period to retention_hours

CLI Example:

salt myminion boto3_kinesis.increase_stream_retention_period my_stream N region=us-east-1

saltext.boto3.modules.boto3_kinesis.decrease_stream_retention_period(stream_name, retention_hours, region=None, key=None, keyid=None, profile=None)

Decrease stream retention period to retention_hours

CLI Example:

salt myminion boto3_kinesis.decrease_stream_retention_period my_stream N region=us-east-1

saltext.boto3.modules.boto3_kinesis.enable_enhanced_monitoring(stream_name, metrics, region=None, key=None, keyid=None, profile=None)

Enable enhanced monitoring for the specified shard-level metrics on stream stream_name

CLI Example:

salt myminion boto3_kinesis.enable_enhanced_monitoring my_stream ["metrics", "to", "enable"] region=us-east-1

saltext.boto3.modules.boto3_kinesis.disable_enhanced_monitoring(stream_name, metrics, region=None, key=None, keyid=None, profile=None)

Disable enhanced monitoring for the specified shard-level metrics on stream stream_name

CLI Example:

salt myminion boto3_kinesis.disable_enhanced_monitoring my_stream ["metrics", "to", "disable"] region=us-east-1

saltext.boto3.modules.boto3_kinesis.get_info_for_reshard(stream_details)

Collect some data: number of open shards, key range, etc. Modifies stream_details to add a sorted list of OpenShards. Returns (min_hash_key, max_hash_key, stream_details)

CLI Example:

salt myminion boto3_kinesis.get_info_for_reshard existing_stream_details

saltext.boto3.modules.boto3_kinesis.long_int(hash_key)

The hash key is a 128-bit int, sent as a string. It’s necessary to convert to int/long for comparison operations. This helper method handles python 2/3 incompatibility

CLI Example:

salt myminion boto3_kinesis.long_int some_MD5_hash_as_string

Returns:

long object if python 2.X, int object if python 3.X

saltext.boto3.modules.boto3_kinesis.reshard(stream_name, desired_size, force=False, region=None, key=None, keyid=None, profile=None)

Reshard a kinesis stream. Each call to this function will wait until the stream is ACTIVE, then make a single split or merge operation. This function decides where to split or merge with the assumption that the ultimate goal is a balanced partition space.

For safety, user must past in force=True; otherwise, the function will dry run.

CLI Example:

salt myminion boto3_kinesis.reshard my_stream N True region=us-east-1

Returns:

True if a split or merge was found/performed, False if nothing is needed

saltext.boto3.modules.boto3_kinesis.list_streams(region=None, key=None, keyid=None, profile=None)

Return a list of all streams visible to the current account

CLI Example:

salt myminion boto3_kinesis.list_streams