Configuration manual¶

Introduction¶

Default installation is suitable for most simple and common cases, but Polemarch is highly configurable system. If you need something more advanced (scalability, dedicated DB, custom cache, logging or directories) you can always configure Polemarch deeply by tweaking /etc/polemarch/settings.ini.

This manual does not have purpose to describe all possible configuration options in details because settings.ini has commentary for every option which makes clear its purpose and possible values. But here is a brief overview of the most important settings to make clear for you big picture: what you can actually customize in Polemarch.

We advice you to read Polemarch clustering overview if you want to setup cluster of Polemarch nodes to maintain reliability or speedup things. It will give you understanding of services, which are included into Polemarch and how to distribute them between the nodes to reach your goal.

Polemarch clustering overview¶

Polemarch actually consists of two services: web-server and worker. Polemarch uses worker for long-running tasks (such as ansible-playbook runs, repo synchronizations and so on). Those services are designed as systemd services you can control using regular distro-tools for service manipulation. You can run more than one server with those services. In default configuration those services uses local file system to keep data and exchange with each other, but for multiple nodes they must be configured to use shared client-server database, cache server and network filesystem (for multiple workers). All those settings are described in appropriate sections of this documentation. It is up to you to make sure that configuration identical on every node to prevent discordant behaviour of nodes. If you have multiple web-servers, don’t forget to setup HAProxy or something similar for balancing load between them.

Lets assume, that you want to create 2 servers with web-part of Polemarch to maintain reliability of your admin-panel and 4 servers with workers to prevent denial for service because of overloading. Then briefly (it is mostly example than general howto) you must do such steps:

Install Polemarch from PyPI at every server with worker and web-server.
```
pip install polemarch
```

Setup DB-server. MariaDB for example. For all nodes edit Database settings and provide credential for you database. Like this:

[database]
engine = django.db.backends.mysql
name = mydatabase
user = root
password = mypassword
host = 127.0.0.1
port = 3306

Setup cache-server. Redis, for example. Specify his credentials in Locks settings and Cache settings for all nodes. Like this:

[cache]
backend = django_redis.cache.RedisCache
location = redis://redis_hostname_or_ip:6379/1

[locks]
backend = django_redis.cache.RedisCache
location = redis://redis_hostname_or_ip:6379/2

Setup some network filesystem. NFS, for example. Mount it in the same directory of all worker-intended nodes. Write this directory in Main settings. Example:
```
[main]
projects_dir = /mnt/mynfs
```
Setup some http-balancer. HAProxy, for example. Point it to web-intended nodes.
Prepare default database structure (tables and so on) in your MySQL database. Polemarch can do it for you with following command:
```
sudo -u polemarch /opt/bin/polemarchctl migrate
```

Create polemarch systemd service:

Firtsly, create a file /etc/systemd/system/polemarch.service:

[Unit]
Description=Polemarch Service HTTP Server
After=network.target remote-fs.target nss-lookup.target redis.service

[Service]
Type=forking
ExecStart=/opt/polemarch3/bin/polemarchctl webserver
ExecReload=/opt/polemarch3/bin/polemarchctl webserver reload=/var/run/polemarch/web.pid
ExecStop=/opt/polemarch3/bin/polemarchctl webserver stop=/var/run/polemarch/web.pid
PIDFile=/var/run/polemarch/web.pid
User=polemarch
Group=polemarch
KillSignal=SIGCONT
Restart=always

[Install]
 WantedBy=multi-user.target

Notice, that user and group ‘polemarch’ should exist in your system. If they don’t exist, create them.

Reload systemctl daemon:
systemctl daemon-reload
Add polemarch.service to autoload:
systemctl enable polemarch.service
Start polemarch.service:
systemctl start polemarch.service

That’s it.

Main settings¶

Section [main].

This section is for settings related to whole Polemarch (both worker and web). Here you can specify verbosity level of Polemarch during work, which can be useful for troubleshoot problems (logging level etc). Also there are settings for changing of timezone for whole app and directory where Polemarch will store ansible projects cloned from repositories.

If you want to use LDAP protocol, you should create next settings in section [main].

ldap-server = ldap://server-ip-or-host:port
ldap-default-domain = domain.name

ldap-default-domain is an optional argument, that is aimed to make user authorization easier (without input of domain name).

So in this case authorization logic will be the following:

System checks combination of login:password in database;
System checks combination of login:password in LDAP:
- if domain was mentioned, it will be set during authorization (if user enter login without user@domain.name or without DOMAIN\user );
- if authorization was successful and there is user with mentioned login in database, server creates session for him.

debug - Enable debug mode. Default: false.
allowed_hosts - Comma separated list of domains, which allowed to serve. Default: *.
ldap-server - LDAP server connection.
ldap-default-domain - Default domain for auth.
timezone - Timezone of web-application. Default: UTC.
log_level - Logging level. Default: WARNING.
enable_admin_panel - Enable or disable Django Admin panel. Defaul: false.
projects_dir - Path where projects will be stored.
hooks_dir - Path where hook scripts stored.
executor_path - Path for polemarch-ansible wrapper binary.

Database settings¶

Section [database].

Here you can change settings related to database system, which Polemarch will use. Polemarch supports all databases supported by django. List of supported out of the box: SQLite (default choice), MySQL, Oracle, or PostgreSQL. Configuration details you can look at Django database documentation. If you run Polemarch at multiple nodes (clusterization), you should use some of client-server database (SQLite not suitable) shared for all nodes.

If you use MySQL there is a list of required settings, that you should create for correct database work.

Firstly, if you use MariaDB and you have set timezone different from “UTC” you should run next command:

mysql_tzinfo_to_sql /usr/share/zoneinfo | mysql -u root -p mysql

Secondly, for correct MariaDB work you should set next options in settings.ini file:

[database.options]
connect_timeout = 10
init_command = SET sql_mode='STRICT_TRANS_TABLES', default_storage_engine=INNODB, NAMES 'utf8', CHARACTER SET 'utf8', SESSION collation_connection = 'utf8_unicode_ci'

Finally, you should add some options to MariaDB configuration:

[client]
default-character-set=utf8
init_command = SET collation_connection = @@collation_database

[mysqld]
character-set-server=utf8
collation-server=utf8_unicode_ci

Cache settings¶

Section [cache].

This section is for settings related to cache backend used by Polemarch. Polemarch supports all cache backends that Django supports. Currently is: filesystem, in-memory, memcached out of the box and many more by additional plugins. You can find details about cache configuration at Django caches documentation. In clusterization scenario we advice to share cache between nodes to speedup their work using client-server cache realizations. We recommend to use Redis in production environments.

Locks settings¶

Section [locks].

Locks is system that Polemarch uses to prevent damage from parallel actions working on something simultaneously. It is based on Django cache, so there is another bunch of same settings as Cache settings. And why there is another section for them, you may ask. Because cache backend used for locking must provide some guarantees, which does not required to usual cache: it MUST be shared for all Polemarch threads and nodes. So, in-memory backend, for example, is not suitable. In case of clusterization we strongly recommend to use Redis or Memcached as backend for that purpose. Cache and locks backend can be same, but don’t forget about requirement we said above.

Session cache settings¶

Section [session].

Polemarch store sessions in Database settings, but for better performance, we use a cache-based session backend. It is based on Django cache, so there is another bunch of same settings as Cache settings. By default, settings getted from Cache settings.

Rpc settings¶

Section [rpc].

Polemarch uses Celery for long-running tasks (such as ansible-playbook runs, repo synchronizations and so on). Celery is based on message queue concept, so between web-service and workers running under Celery bust be some kind of message broker (RabbitMQ or something). Those settings relate to this broker and Celery itself. Those kinds of settings: broker backend, number of worker-processes per node and some settings used for troubleshoot server-broker-worker interaction problems.

connection - Celery broker connection. Read more: http://docs.celeryproject.org/en/latest/userguide/configuration.html#conf-broker-settings Default: filesystem:///var/tmp.
concurrency - Count of celery worker threads. Default: 4.
heartbeat - Interval between sending heartbeat packages, which says that connection still alive. Default: 10.
enable_worker - Enable or disable worker with webserver. Default: true.
clone_retry_count - Count of retrys on project sync operation.

Worker settings¶

Section [worker].

Celery worker options for start. Useful settings:

loglevel - Celery worker logging level. Default: from main section log_level.
pidfile - Celery worker pidfile. Default: /run/polemarch_worker.pid
autoscale - Options for autoscaling. Two comma separated numbers: max,min.
beat - Enable or disable celery beat scheduler. Default: true.

Other settings can be getted from command celery worker --help.

Web settings¶

Section [web].

Here placed settings related to web-server. Those settings like: session_timeout, static_files_url or pagination limit.

session_timeout - Session life-cycle time. Default: 2w (two weeks).
rest_page_limit - Default limit of objects in API list. Default: 1000.
public_openapi - Allow to have access to OpenAPI schema from public. Default: false.

Git settings¶

Sections [git.fetch] and [git.clone].

Options for git commands. See options in git fetch --help or git clone --help.

Production web settings¶

Section [uwsgi].

Here placed settings related to web-server used by Polemarch in production (for deb and rpm packages by default). Most of them related to system paths (logging, PID-file and so on). More settings in uWSGI docs.

Installation of additional packages to Polemarch¶

If you want to install some additional package to Polemarch from rpm or dep, you should run next command:

sudo -U polemarch /opt/polemarch/bin/pip install package_name

For correct work all requirements for this package should be installed in your system. Notice, that after package reinstallation or after package update you should set all this requirements again.

If you want to install some additional package from github or gitlab, you should just install this package to your system or to your virtual environment.

Polemarch documentation