elasticsearch

Return data to an elasticsearch server for indexing.

Copied from original returner and modified to support Elasticsearch 7 and below

Original maintainers: Jurnell Cockhren <jurnell.cockhren@sophicware.com>, Arnold Bechtoldt <mail@arnoldbechtoldt.com>

maintainer:

Cesar Sanchez <cesan3@gmail.com>

To enable this returner the elasticsearch python client must be installed on the desired minions (all or some subset).

Please see documentation of elasticsearch execution module for a valid connection configuration.

Warning

The index that you wish to store documents will be created by Elasticsearch automatically if doesn’t exist yet. It is highly recommended to create predefined index templates with appropriate mapping(s) that will be used by Elasticsearch upon index creation. Otherwise you will have problems as described in #20826.

To use the returner per salt call:

salt '*' test.ping --return elasticsearch

In order to have the returner apply to all minions:

ext_job_cache: elasticsearch
Minion configuration:
debug_returner_payload’: False

Output the payload being posted to the log file in debug mode

doc_type: ‘default’

Document type to use for normal return messages

functions_blacklist

Optional list of functions that should not be returned to elasticsearch

index_date: False

Use a dated index (e.g. <index>-2016.11.29)

master_event_index: ‘salt-master-event-cache’

Index to use when returning master events

master_event_doc_type: ‘default’

Document type to use for master events

master_job_cache_index: ‘salt-master-job-cache’

Index to use for master job cache

master_job_cache_doc_type: ‘default’

Document type to use for master job cache

number_of_shards: 1

Number of shards to use for the indexes

number_of_replicas: 0

Number of replicas to use for the indexes

NOTE: The following options are valid for ‘state.apply’, ‘state.sls’ and ‘state.highstate’ functions only.

states_count: False

Count the number of states which succeeded or failed and return it in top-level item called ‘counts’. States reporting None (i.e. changes would be made but it ran in test mode) are counted as successes.

states_order_output: False

Prefix the state UID (e.g. file_|-yum_configured_|-/etc/yum.conf_|-managed) with a zero-padded version of the ‘__run_num__’ value to allow for easier sorting. Also store the state function (i.e. file.managed) into a new key ‘_func’. Change the index to be ‘<index>-ordered’ (e.g. salt-state_apply-ordered).

states_single_index: False

Store results for state.apply, state.sls and state.highstate in the salt-state_apply index (or -ordered/-<date>) indexes if enabled

failover: False

Probe each configured host and use the first one that responds when establishing the connection. Implemented by the elasticsearch execution module’s connection setup.

dev: False

Prefix the job-cache and job indexes with ‘dev-’ to keep testing returns separated from production indexes.

retry_on_conflicts: 5

Number of times the document update adding a minion id to the master job cache doc is retried on a version conflict.

elasticsearch:
    hosts:
      - "10.10.10.10:9200"
      - "10.10.10.11:9200"
      - "10.10.10.12:9200"
    index_date: True
    number_of_shards: 5
    number_of_replicas: 1
    debug_returner_payload: True
    states_count: True
    states_order_output: True
    states_single_index: True
    functions_blacklist:
      - test.ping
      - saltutil.find_job

Note

The master job cache stores documents in <index>-v2 indexes. On first use the returner rotates the <index> alias from the legacy -v1 index to the new -v2 index. Returns recorded before the upgrade stay in the -v1 index and are no longer visible to jobs.lookup_jid; only post-upgrade returns appear in the new cache.

class saltext.elasticsearch.returners.elasticsearch6_mod.UTC[source]
utcoffset(_dt)[source]

datetime -> timedelta showing offset from UTC, negative values indicating West of UTC

tzname(_dt)[source]

datetime -> string name of time zone.

dst(_dt)[source]

datetime -> DST offset as timedelta positive east of UTC.

saltext.elasticsearch.returners.elasticsearch6_mod.returner(ret)[source]

Process the return from Salt

saltext.elasticsearch.returners.elasticsearch6_mod.event_return(events)[source]

Return events to Elasticsearch

Requires that the event_return configuration be set in master config.

saltext.elasticsearch.returners.elasticsearch6_mod.prep_jid(nocache=False, passed_jid=None)[source]

Do any work necessary to prepare a JID, including sending a custom id

saltext.elasticsearch.returners.elasticsearch6_mod.save_load(jid, load, minions=None)[source]

Save the load to the specified jid id

Added in version 2015.8.1.

saltext.elasticsearch.returners.elasticsearch6_mod.get_load(jid)[source]

Return the load data that marks a specified jid

Added in version 2015.8.1.

saltext.elasticsearch.returners.elasticsearch6_mod.get_jid(jid)[source]

Return the load data that marks a specified jid

Added in version 2015.8.1.

saltext.elasticsearch.returners.elasticsearch6_mod.get_jids()[source]

Return the load data that marks a specified jid

Added in version 2015.8.1.

Return data to an elasticsearch server for indexing.

Copied from original returner and modified to support elasticsearch 8.x

Original maintainers: Jurnell Cockhren <jurnell.cockhren@sophicware.com>, Arnold Bechtoldt <mail@arnoldbechtoldt.com>

maintainer:

Cesar Sanchez <cesan3@gmail.com>

To enable this returner the elasticsearch python client must be installed on the desired minions (all or some subset).

Please see documentation of elasticsearch execution module for a valid connection configuration.

Warning

The index that you wish to store documents will be created by Elasticsearch automatically if doesn’t exist yet. It is highly recommended to create predefined index templates with appropriate mapping(s) that will be used by Elasticsearch upon index creation. Otherwise you will have problems as described in #20826.

To use the returner per salt call:

salt '*' test.ping --return elasticsearch

In order to have the returner apply to all minions:

ext_job_cache: elasticsearch
Minion configuration:
debug_returner_payload’: False

Output the payload being posted to the log file in debug mode

doc_type: ‘default’

Document type to use for normal return messages

functions_blacklist

Optional list of functions that should not be returned to elasticsearch

index_date: False

Use a dated index (e.g. <index>-2016.11.29)

master_event_index: ‘salt-master-event-cache’

Index to use when returning master events

master_event_doc_type: ‘default’

Document type to use for master events

master_job_cache_index: ‘salt-master-job-cache’

Index to use for master job cache

master_job_cache_doc_type: ‘default’

Document type to use for master job cache

number_of_shards: 1

Number of shards to use for the indexes

number_of_replicas: 0

Number of replicas to use for the indexes

NOTE: The following options are valid for ‘state.apply’, ‘state.sls’ and ‘state.highstate’ functions only.

states_count: False

Count the number of states which succeeded or failed and return it in top-level item called ‘counts’. States reporting None (i.e. changes would be made but it ran in test mode) are counted as successes.

states_order_output: False

Prefix the state UID (e.g. file_|-yum_configured_|-/etc/yum.conf_|-managed) with a zero-padded version of the ‘__run_num__’ value to allow for easier sorting. Also store the state function (i.e. file.managed) into a new key ‘_func’. Change the index to be ‘<index>-ordered’ (e.g. salt-state_apply-ordered).

states_single_index: False

Store results for state.apply, state.sls and state.highstate in the salt-state_apply index (or -ordered/-<date>) indexes if enabled

failover: False

Probe each configured host and use the first one that responds when establishing the connection. Implemented by the elasticsearch execution module’s connection setup.

dev: False

Prefix the job-cache and job indexes with ‘dev-’ to keep testing returns separated from production indexes.

retry_on_conflicts: 5

Number of times the document update adding a minion id to the master job cache doc is retried on a version conflict.

elasticsearch:
    hosts:
      - "10.10.10.10:9200"
      - "10.10.10.11:9200"
      - "10.10.10.12:9200"
    index_date: True
    number_of_shards: 5
    number_of_replicas: 1
    debug_returner_payload: True
    states_count: True
    states_order_output: True
    states_single_index: True
    functions_blacklist:
      - test.ping
      - saltutil.find_job

Note

The master job cache stores documents in <index>-v2 indexes. On first use the returner rotates the <index> alias from the legacy -v1 index to the new -v2 index. Returns recorded before the upgrade stay in the -v1 index and are no longer visible to jobs.lookup_jid; only post-upgrade returns appear in the new cache.

class saltext.elasticsearch.returners.elasticsearch8_mod.UTC[source]
utcoffset(_dt)[source]

datetime -> timedelta showing offset from UTC, negative values indicating West of UTC

tzname(_dt)[source]

datetime -> string name of time zone.

dst(_dt)[source]

datetime -> DST offset as timedelta positive east of UTC.

saltext.elasticsearch.returners.elasticsearch8_mod.returner(ret)[source]

Process the return from Salt

saltext.elasticsearch.returners.elasticsearch8_mod.event_return(events)[source]

Return events to Elasticsearch

Requires that the event_return configuration be set in master config.

saltext.elasticsearch.returners.elasticsearch8_mod.prep_jid(nocache=False, passed_jid=None)[source]

Do any work necessary to prepare a JID, including sending a custom id

saltext.elasticsearch.returners.elasticsearch8_mod.save_load(jid, load, minions=None)[source]

Save the load to the specified jid id

Added in version 2015.8.1.

saltext.elasticsearch.returners.elasticsearch8_mod.get_load(jid)[source]

Return the load data that marks a specified jid

Added in version 2015.8.1.

saltext.elasticsearch.returners.elasticsearch8_mod.get_jid(jid)[source]

Return the load data that marks a specified jid

Added in version 2015.8.1.

saltext.elasticsearch.returners.elasticsearch8_mod.get_jids()[source]

Return the load data that marks a specified jid

Added in version 2015.8.1.

Return data to an elasticsearch server for indexing.

Copied from original returner and modified to support elasticsearch 9.x

Original maintainers: Jurnell Cockhren <jurnell.cockhren@sophicware.com>, Arnold Bechtoldt <mail@arnoldbechtoldt.com>

maintainer:

Cesar Sanchez <cesan3@gmail.com>

To enable this returner the elasticsearch python client must be installed on the desired minions (all or some subset).

Please see documentation of elasticsearch execution module for a valid connection configuration.

Warning

The index that you wish to store documents will be created by Elasticsearch automatically if doesn’t exist yet. It is highly recommended to create predefined index templates with appropriate mapping(s) that will be used by Elasticsearch upon index creation. Otherwise you will have problems as described in #20826.

To use the returner per salt call:

salt '*' test.ping --return elasticsearch

In order to have the returner apply to all minions:

ext_job_cache: elasticsearch
Minion configuration:
debug_returner_payload’: False

Output the payload being posted to the log file in debug mode

doc_type: ‘default’

Document type to use for normal return messages

functions_blacklist

Optional list of functions that should not be returned to elasticsearch

index_date: False

Use a dated index (e.g. <index>-2016.11.29)

master_event_index: ‘salt-master-event-cache’

Index to use when returning master events

master_event_doc_type: ‘default’

Document type to use for master events

master_job_cache_index: ‘salt-master-job-cache’

Index to use for master job cache

master_job_cache_doc_type: ‘default’

Document type to use for master job cache

number_of_shards: 1

Number of shards to use for the indexes

number_of_replicas: 0

Number of replicas to use for the indexes

NOTE: The following options are valid for ‘state.apply’, ‘state.sls’ and ‘state.highstate’ functions only.

states_count: False

Count the number of states which succeeded or failed and return it in top-level item called ‘counts’. States reporting None (i.e. changes would be made but it ran in test mode) are counted as successes.

states_order_output: False

Prefix the state UID (e.g. file_|-yum_configured_|-/etc/yum.conf_|-managed) with a zero-padded version of the ‘__run_num__’ value to allow for easier sorting. Also store the state function (i.e. file.managed) into a new key ‘_func’. Change the index to be ‘<index>-ordered’ (e.g. salt-state_apply-ordered).

states_single_index: False

Store results for state.apply, state.sls and state.highstate in the salt-state_apply index (or -ordered/-<date>) indexes if enabled

failover: False

Probe each configured host and use the first one that responds when establishing the connection. Implemented by the elasticsearch execution module’s connection setup.

dev: False

Prefix the job-cache and job indexes with ‘dev-’ to keep testing returns separated from production indexes.

retry_on_conflicts: 5

Number of times the document update adding a minion id to the master job cache doc is retried on a version conflict.

elasticsearch:
    hosts:
      - "10.10.10.10:9200"
      - "10.10.10.11:9200"
      - "10.10.10.12:9200"
    index_date: True
    number_of_shards: 5
    number_of_replicas: 1
    debug_returner_payload: True
    states_count: True
    states_order_output: True
    states_single_index: True
    functions_blacklist:
      - test.ping
      - saltutil.find_job

Note

The master job cache stores documents in <index>-v2 indexes. On first use the returner rotates the <index> alias from the legacy -v1 index to the new -v2 index. Returns recorded before the upgrade stay in the -v1 index and are no longer visible to jobs.lookup_jid; only post-upgrade returns appear in the new cache.

class saltext.elasticsearch.returners.elasticsearch9_mod.UTC[source]
utcoffset(_dt)[source]

datetime -> timedelta showing offset from UTC, negative values indicating West of UTC

tzname(_dt)[source]

datetime -> string name of time zone.

dst(_dt)[source]

datetime -> DST offset as timedelta positive east of UTC.

saltext.elasticsearch.returners.elasticsearch9_mod.returner(ret)[source]

Process the return from Salt

saltext.elasticsearch.returners.elasticsearch9_mod.event_return(events)[source]

Return events to Elasticsearch

Requires that the event_return configuration be set in master config.

saltext.elasticsearch.returners.elasticsearch9_mod.prep_jid(nocache=False, passed_jid=None)[source]

Do any work necessary to prepare a JID, including sending a custom id

saltext.elasticsearch.returners.elasticsearch9_mod.save_load(jid, load, minions=None)[source]

Save the load to the specified jid id

Added in version 2015.8.1.

saltext.elasticsearch.returners.elasticsearch9_mod.get_load(jid)[source]

Return the load data that marks a specified jid

Added in version 2015.8.1.

saltext.elasticsearch.returners.elasticsearch9_mod.get_jid(jid)[source]

Return the load data that marks a specified jid

Added in version 2015.8.1.

saltext.elasticsearch.returners.elasticsearch9_mod.get_jids()[source]

Return the load data that marks a specified jid

Added in version 2015.8.1.