Plugins

Polemarch can be extended with different type of plugins. This manual describes how to create, setup and start using your first Execution or Inventory plugin and provides full API reference to learn more about them.

Execution plugins

Execution plugins system allows you to execute any external command from Polemarch. To create an execution plugin you need to

  • create a python class describing the argument processing and API talking logic;

  • configure this plugin in your settings.ini.

Warning

Since v3.0.0 path to execution plugins has been changed from polemarch.plugins to polemarch.plugins.execution.

Execution plugins: quick start

All built-in execution backends, such as ansible playbook or module execution, use plugins (starting from 2.2.0).

To get started let’s create a simple plugin allows you to run echo command.

from rest_framework.fields import BooleanField, empty
from vstutils.api.fields import VSTCharField
from polemarch.plugins.execution.base import BasePlugin


# Any plugin must be inherited from BasePlugin
class TestEcho(BasePlugin):
    # Define fields which will be used to execute this plugin and create template with
    serializer_fields = {
        # You can use fields either from rest_framework or vstutils
        'string': VSTCharField(),
        'n': BooleanField(default=empty, required=False, label='No trailing newlines'),
        'e': BooleanField(default=empty, required=False, label='Interpret backslash escapes'),
    }
    # Value of this field will be shown on history detail page as Mode
    arg_shown_on_history_as_mode = 'string'

    # This is our binary from which any command starts
    @property
    def base_command(self):
        return ['echo']

    # This method is called by get_args method of BasePlugin for each argument received from API. As we defined
    # 'string', 'n', and 'e' arguments in serializer_fields, we can get their keys and values here.
    def _process_arg(self, key, value):
        # As 'string' argument in this case is a positional argument, let's return just it's value, so the final
        # command will be something like ['echo', 'string to output', ...]
        if key == 'string':
            return value
        # As this guys are just boolean flags, let's return them as '-n' or '-e' accordingly
        if key in ('n', 'e') and value:
            return f'-{key}'
        # Note, that if we received for example `n` argument with False value, we are returning None,
        # means that it won't be included to execution command. But of course, you may override this behavior
        # in get_args method.

Supposing that described plugin is located at polemarch.plugins.execution.custom.Echo, let’s connect it to Polemarch. In your /etc/polemarch/settings.ini add following section:

[execution.plugin.echo]
backend = polemarch.plugins.execution.custom.Echo

Also you may want to provide additional options directly to plugin:

[execution.plugin.echo.options]
some_option = 'some_option'

In this example some_option will be available in any plugin’s instance method as self.config['some_option'], as all options are initialized in the __init__ method.

So it’s all done! After restarting your polemarch server and resetting schema, you can check #/project/<your_project_id>/execute_echo/. Here you should be able to execute echo plugin as any built-in one. Also you should be able to create a template with it at #/project/<your_project_id>/execution_templates/new/.

If you tried executing echo plugin with flags, you may see that this flags are being outputted too. This is because they goes after string argument. To fix this issue, we may do something like this:

...

class TestEcho(BasePlugin):
    ...

    def get_args(self, raw_args):
        # We know that 'string' is required so no default for .pop() is needed
        string_value = raw_args.pop('string')
        args = super().get_args(raw_args)
        # Push our string to the end of command
        args += [string_value]
        return args

    @property
    def base_command(self):
        return ['echo']

    def _process_arg(self, key, value):
        if key in ('n', 'e') and value:
            return f'-{key}'

Now if you are passing flags to execution, they should work the same except not being outputted.

Note

If your execution plugin may work with inventories, you should specify which inventory plugins are compatible with your execution plugin. By default it’s assumed that execution plugin can’t work with inventories. For more information about inventory plugins please see Inventory plugins.

To learn more about what plugins are provide, please check API reference.

Inventory plugins

Inventory plugins system allows you to define how inventory stores, manages and displays its state. To create an inventory plugin you need to

  • create a python class which manages inventory state and API talking logic;

  • configure this plugin in your settings.ini.

Inventory plugins: quick start

To get started let’s create a simple plugin which works like a simplified version of built-in AnsibleString with some additional changes.

from vstutils.api import fields as vstfields
from polemarch.plugins.inventory.base import BasePlugin

class CustomAnsibleString(BasePlugin):
    # Our plugin will support import and we will implement corresponding method
    supports_import = True

    # These fields will be used for working with inventory state in API
    serializer_fields = {
        # You can use fields either from rest_framework or vstutils
        'body': vstfields.TextareaField(allow_blank=True, default=''),
    }
    # Default values for our fields which will be used to initialize inventory state
    defaults = {
        'body': 'localhost ansible_connection=local',
    }
    # These fields will be used for import action in API
    serializer_import_fields = {
        'body': vstfields.FileInStringField(),
    }

    def render_inventory(self, execution_dir):
        # Getting inventory state. This is possible because by default state_managed attribute of plugin class
        # is True
        state_data = self.instance.inventory_state.data
        filename = str(uuid1())
        # Any created files must be in execution_dir
        filepath = Path(execution_dir) / filename
        filepath.write_text(state_data['body'])
        # We doesn't need any additional files so the second argument is empty list
        return filepath, []

    def get_raw_inventory(self, inventory_string):
        # This string will be shown on history page
        return f'File contents:\n{inventory_string}'

    @classmethod
    def import_inventory(cls, instance, data):
        # Here we got data which structure corresponds to serializer_import_fields
        # and created inventory instance. It's recommended to always use update_inventory_state method
        # rather than accessing inventory state directly.
        instance.update_inventory_state(data=data)
        return instance

Supposing that described plugin is located at polemarch.plugins.inventory.custom.CustomInventoryString, let’s connect it to Polemarch. In your /etc/polemarch/settings.ini add following section:

[inventory.plugin.custom_inventory_string]
backend = polemarch.plugins.inventory.custom.CustomInventoryString

Also you may want to provide additional options directly to plugin:

[inventory.plugin.custom_inventory_string.options]
some_option = 'some_option'

In this example some_option will be available in any plugin’s instance method as self.options['some_option'], as all options are initialized in the __init__ method.

To start working with the created plugin we also need to allow some execution plugin work with this one. Let’s say that ANSIBLE_MODULE plugin can play with our new CUSTOM_ANSIBLE_STRING:

[execution.plugin.ansible_module.options]
; Make sure you not disabled other built-in inventory plugins
compatible_inventory_plugins = polemarch_db,ansible_file,ansible_string,custom_inventory_string

Done! Now you can create inventory with your plugin at #/project/<your_project_id>/inventory/new/, execute ANSIBLE_MODULE plugin at #/project/<your_project_id>/execute_ansible_module/ selecting created inventory.

API reference

class polemarch.plugins.execution.base.BasePlugin(options=None, output_handler=None)

Base execution plugin class from which any other plugin should inherit. The plugin itself is an entity which, on the one hand provides appropriate fields for the API, and on the other hand, processes arguments received from it to build execution command.

For each configured plugin an endpoint will be generated allows you to choose arguments and execute it. Also, this plugin will be available to create template with.

Parameters
  • options (dict) – settings.ini options mapping for this plugin.

  • output_handler (typing.Optional[typing.Callable]) – executor’s function which handles logging and outputting to history. Used by verbose_output method.

_get_serializer_fields(exclude_fields=())

Returns field name and field instance mapping used to generate fields for serializer.

Parameters

exclude_fields (tuple) – field names that should not be presented in serializer.

Return type

typing.Mapping[str, rest_framework.fields.Field]

_get_serializer_metaclass(exclude_fields=())

Returns serializer metaclass used to generate fields in serializer.

Parameters

exclude_fields (tuple) – field names that should not be presented in serializer.

Return type

typing.Type[typing.Type[vstutils.api.serializers.BaseSerializer]]

_process_arg(key, value)

Returns single argument with value for get_args method. Should return None if argument must not be included to the execution command.

Parameters
  • key (str) – argument key (e.g. verbose).

  • value (typing.Any) – argument value (e.g. 2).

Return type

typing.Optional[str]

arg_shown_on_history_as_inventory: typing.Optional[str] = None

Name of argument presented in generated serializer which will be shown on list history page as Inventory.

arg_shown_on_history_as_mode: typing.Optional[str] = None

Name of argument presented in generated serializer which will be shown on detail history page as Mode. For example, if you are executing some module with additional arguments and fields contains module field, than you can set ‘module’ here, and it’s value will be shown after execution. If not set, Mode in the history will show [<plugin name> plugin] string.

base_command: typing.List[str]

Base command (usually binary) from which execution command starts, e.g. ['echo']. You may also override this attribute as a property if more complex logic needs to be performed.

error_codes: typing.Mapping[int, str] = {}

This mapping will be looked up to choose an appropriate error message for history output if execution finished with errors. If no code found, then just “ERROR” string outputs.

get_args(raw_args)

Returns list of processed arguments which will be substituted into execution command.

Parameters

raw_args (dict) – argument name-value mapping which should be processed.

Return type

typing.List[str]

get_env_vars(project_data)

Returns env variables which will be used in execution, project’s env variables by default.

Parameters

project_data – proxy of the project instance, allows you to access it’s readonly properties, such as config vars, env_vars etc.

Return type

typing.Mapping[str, str]

get_execution_data(execution_dir, raw_args, project_data)

Returns tuple of execution command and env variables. This method will be called directly by executor before execution starts.

Parameters
  • execution_dir (pathlib.Path) – path to execution directory in which project copy located. All additional files that should be generated (e.g. inventory file) must be placed here.

  • raw_args (dict) – argument name-value mapping which should be processed.

  • project_data – proxy of the project instance, allows you to access it’s readonly properties, such as config vars, env_vars etc.

Return type

typing.Tuple[typing.List[str], dict]

get_pre_commands(raw_args)

This method will be called before execution. Returns list of commands which are needed to be executed before main execution command.

Parameters

raw_args (dict) – dictionary with arguments received from API.

Return type

typing.List[typing.List[str]]

get_raw_inventory()

Returns raw inventory string used to show it on history page.

Return type

str

get_serializer_class(exclude_fields=())

Returns serializer class which will be used to generate fields for arguments. Uses metaclass returned by _get_serializer_metaclass method.

Parameters

exclude_fields (tuple) – field names that should not be presented in serializer.

Return type

typing.Type[vstutils.api.serializers.BaseSerializer]

get_verbose_level(raw_args)

Returns verbose level used for history output and logging. Should be taken from execution arguments (usually from verbose argument). This method will be called directly by executor.

Parameters

raw_args (dict) – argument name-value mapping which should be processed.

Return type

int

property name: str

Returns name of plugin, class name by default. Primarily used to generate an appropriate model name for OpenAPI schema.

post_execute_hook(cmd, raw_args)

This method will be called after execution.

Parameters
  • cmd (typing.List[str]) – list of arguments which were used for execution.

  • raw_args (dict) – dictionary with arguments received from API.

Return type

None

prepare_execution_dir(dir)

Gets execution directory with copied project. All files needed for execution (e.g. generated inventory file) should be here.

Parameters

dir (pathlib.Path) – path to execution directory in which project copy located. All additional files that should be generated (e.g. inventory file) must be placed here.

Return type

None

serializer_fields: typing.Mapping[str, rest_framework.fields.Field] = {}

Fields mapping used to generate serializer. By default returned by _get_serializer_fields method.

verbose_output(message, level=3)

Logs value with logger and outputs it to history.

Parameters
  • message (str) – message to output.

  • level (int) – verbosity level from which message should be outputted.

Return type

None

class polemarch.plugins.inventory.base.BasePlugin(options)

Base inventory plugin class from which any other plugin should inherit. The plugin itself is an entity which, on the one hand provides appropriate fields for the API, and on the other hand, manages inventory state.

Parameters

optionssettings.ini options for this plugin.

_get_serializer_fields()

Returns field name and field instance mapping used to generate fields for serializer used for working with state in API.

Return type

typing.Mapping[str, rest_framework.fields.Field]

_get_serializer_import_fields()

Returns field name and field instance mapping used to generate fields for serializer used for import action.

Return type

typing.Mapping[str, rest_framework.fields.Field]

_get_serializer_import_metaclass()

Returns serializer metaclass used to generate fields in serializer for import action.

Return type

typing.Type[typing.Type[vstutils.api.serializers.BaseSerializer]]

_get_serializer_metaclass()

Returns serializer metaclass used to generate fields in serializer for working with state in API.

Return type

typing.Type[typing.Type[vstutils.api.serializers.BaseSerializer]]

defaults: typing.Mapping[str, typing.Any] = {}

Field name and its default value mapping used to initialize inventory state. Only works if plugin is state_managed.

classmethod get_raw_inventory(inventory_string)

Returns raw inventory string used to show it on history page.

Parameters

inventory_string (str) – inventory string which is needed to be processed.

Return type

str

get_serializer_class()

Returns serializer class which will be used for working with state in API. Uses metaclass returned by _get_serializer_metaclass method.

Return type

typing.Type[vstutils.api.serializers.BaseSerializer]

get_serializer_import_class()

Returns serializer class which will be used for import action. Uses metaclass returned by _get_serializer_metaclass method.

Return type

typing.Type[vstutils.api.serializers.BaseSerializer]

classmethod import_inventory(instance, data)

Method which implements importing inventory from external source. Must be implemented if supports_import is True.

Parameters
  • data (dict) – data received from user in API as a result of import action.

  • instance (polemarch.main.models.hosts.Inventory) – created Inventory instance.

render_inventory(instance, execution_dir)

Renders inventory into text file and puts it into execution_dir directory. Additional files may be returned by second argument as list (or empty list, if no any).

Parameters
  • instance (polemarch.main.models.hosts.Inventory) – Inventory instance.

  • execution_dir (pathlib.Path) – path

Return type

typing.Tuple[pathlib.Path, list]

serializer_fields: typing.Mapping[str, rest_framework.fields.Field] = {}

Fields mapping used to generate serializer for working with state in API. By default returned by _get_serializer_fields method.

serializer_import_fields: typing.Mapping[str, rest_framework.fields.Field] = {}

Fields mapping used to generate serializer for import action. By default returned by _get_serializer_import_fields method.

state_managed: bool = True

Flag indicating whether Inventory stores all its data in json field or in separate models.

supports_import: bool = False

Flag indicating whether Inventory supports import from external source or not. If True import_inventory method must be implemented.