boto3_datapipeline

Connection module for Amazon Data Pipeline using boto3.

Renamed from boto_datapipeline to boto3_datapipeline and rewritten to use the boto3 datapipeline client APIs directly via saltext.boto3.utils.boto3mod. The legacy boto2 code path (object-style access, retry loops) has been removed.

depends:

• boto3 >= 1.28.0

• botocore >= 1.31.0

configuration:

This module accepts explicit Data Pipeline credentials but can also utilize IAM roles assigned to the instance through Instance Profiles. Dynamic credentials are then automatically obtained from AWS API and no further configuration is necessary. More Information available at:

http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/iam-roles-for-amazon-ec2.html

If IAM roles are not used you need to specify them either in the minion’s config file or as a profile. For example, to specify them in the minion’s config file:

datapipeline.keyid: GKTADJGHEIQSXMKKRBJ08H
datapipeline.key: askdjghsdfjkghWupUjasdflkdfklgjsdfjajkghs

A region may also be specified in the configuration:

datapipeline.region: us-east-1

It’s also possible to specify key, keyid and region via a profile, either as a passed in dict, or as a string to pull from pillars or minion config:

myprofile:
    keyid: GKTADJGHEIQSXMKKRBJ08H
    key: askdjghsdfjkghWupUjasdflkdfklgjsdfjajkghs
    region: us-east-1

Added in version 1.0.0.

saltext.boto3.modules.boto3_datapipeline.__virtual__()

Only load if boto3 is available.

saltext.boto3.modules.boto3_datapipeline.activate_pipeline(pipeline_id, region=None, key=None, keyid=None, profile=None)

Start processing pipeline tasks. This function is idempotent.

CLI Example:

salt myminion boto3_datapipeline.activate_pipeline my_pipeline_id

saltext.boto3.modules.boto3_datapipeline.create_pipeline(name, unique_id, description='', region=None, key=None, keyid=None, profile=None)

Create a new, empty pipeline. This function is idempotent.

CLI Example:

salt myminion boto3_datapipeline.create_pipeline my_name my_unique_id

saltext.boto3.modules.boto3_datapipeline.delete_pipeline(pipeline_id, region=None, key=None, keyid=None, profile=None)

Delete a pipeline, its pipeline definition, and its run history. This function is idempotent.

CLI Example:

salt myminion boto3_datapipeline.delete_pipeline my_pipeline_id

saltext.boto3.modules.boto3_datapipeline.describe_pipelines(pipeline_ids, region=None, key=None, keyid=None, profile=None)

Retrieve metadata about one or more pipelines.

CLI Example:

salt myminion boto3_datapipeline.describe_pipelines ['my_pipeline_id']

saltext.boto3.modules.boto3_datapipeline.get_pipeline_definition(pipeline_id, version='latest', region=None, key=None, keyid=None, profile=None)

Get the definition of the specified pipeline.

CLI Example:

salt myminion boto3_datapipeline.get_pipeline_definition my_pipeline_id

saltext.boto3.modules.boto3_datapipeline.list_pipelines(region=None, key=None, keyid=None, profile=None)

Get a list of pipeline ids and names for all pipelines.

CLI Example:

salt myminion boto3_datapipeline.list_pipelines profile=myprofile

saltext.boto3.modules.boto3_datapipeline.pipeline_id_from_name(name, region=None, key=None, keyid=None, profile=None)

Get the pipeline id, if it exists, for the given name.

CLI Example:

salt myminion boto3_datapipeline.pipeline_id_from_name my_pipeline_name

saltext.boto3.modules.boto3_datapipeline.put_pipeline_definition(pipeline_id, pipeline_objects, parameter_objects=None, parameter_values=None, region=None, key=None, keyid=None, profile=None)

Add tasks, schedules, and preconditions to the specified pipeline. This function is idempotent and will replace an existing definition.

CLI Example:

salt myminion boto3_datapipeline.put_pipeline_definition my_pipeline_id my_pipeline_objects