Ansible-based installation on Linux

Overview

MiaRec uses Ansible IT automation engine to deploy its software components on Linux system. This guide provides step-by-step instructions for both initial deployment as well as update of MiaRec software.

What is Ansible?

Ansible is an automation tool for provisioning, application deployment, and configuration management.

Ansible uses playbooks written in the YAML language for orchestration. For more information, see Ansible - Intro to Playbooks.

Compared with other server configuration management DevOps tools, Ansible doesn’t require agents to be installed on the managed servers. Instead, Ansible manages the IT infrastructure by using SSH protocol to communicate the managed resources. This dramatically simplifies the configuration of managed systems for two reasons—no process daemons need to run on the remote servers to communicate with a central controller and IT administrators aren’t required to manage or maintain agents on each managed node.

Ansible can communicate with multiple managed nodes at the same time. This allows to easily deploy various software components, like database, web server, recorder on multiple dedicated servers using a single command.

Comparing to manual installation commands, Ansible allows to build a completely reproducible server configuration. It is a good practice to test Ansible playbooks towards the staging environment and after verification apply the same configuration to the production environment.

Installation workflow

The following diagram shows the general workflow of an MiaRec installation using Ansible.

In the next chapters, each of these steps is described in details.

This guide refers to the following types of hosts:

  • Controller host, which runs the Ansible playbook
  • Target hosts, where Ansible installs MiaRec software components.

In simple scenarios, like "all-in-one" configuration when all MiaRec software components are deployed on a single host, the same host can be used for both Controller and Target roles, i.e. the Ansible playbook could be run to deploy MiaRec locally. The following diagram demonstrate a difference between these use cases: remote controller and local controller.

In more complex scenarios, like the deployment of MiaRec software components on multiple hosts, the Ansible playbook should be executed from a remote host. The following diagram shows how the remote controller host automatically deploys MiaRec on multiple servers.

1. Prepare controller host

When deploying MiaRec in "all-in-one" configuration on a single server, you can use the same host for both Controller and Target roles. In this case, the Ansible playbook will deploy MiaRec locally.

When deploying MiaRec on multiple servers, it is necessary to use a dedicated host for the Controller role.

Supported operating systems for the Controller host

MiaRec team officially supports the following operating system for the controller host:

  • Centos 7 64-bit
  • Centos 6 64-bit
  • Ubuntu Server 14.04 (Xenial Xerus) LTS 64-bit
  • Ubuntu Server 16.04 (Trusty Tahr) LTS 64-bit
  • Windows 10 with Bash on Ubuntu *

(*) - The Windows 10 machine could be used solely for the Controller role. If you need to install MiaRec software on Windows operating system, then check the guide Installation on Windows.

It is possible to run Ansible playbook from Mac OSX and other operating systems. The complete list of the supported OSs is available in the official Ansible documentation. The MiaRec team provides technical support for the above mentioned OSs only.

Install Ansible on Ubuntu

Install additional software packages and configure Network Time Protocol (NTP). Before you begin, we recommend upgrading your system packages and kernel.

Update package source lists:

sudo apt-get update

Upgrade the system packages and kernel:

sudo apt-get dist-upgrade

Reboot the host.

Install additional software packages if they were not installed during the operating system installation:

sudo apt-get install aptitude build-essential git ntp ntpdate openssh-server libssl-dev 

Install PIP (a tool for installing Python packages. Ansible is written in Python):

sudo apt-get install python-dev python-pip

Install Ansible using PIP:

sudo pip install ansible

Verify Ansible version:

ansible --version

The output should be something like:

$ ansible --version
ansible 2.3.1.0
  config file = 
  configured module search path = Default w/o overrides
  python version = 2.7.12 (default, Nov 19 2016, 06:48:10) [GCC 5.4.0 20160609]

Verify that ansible version is 2.2+ or higher and python version is either 2.7 or 2.6. If the python version shows 3.x then the installation of Ansible is not correct. Contact the MiaRec representative for support.

Install Ansible on Centos 7

Install additional software packages and configure Network Time Protocol (NTP). Before you begin, we recommend upgrading your system packages and kernel.

Upgrade the system packages and kernel

sudo yum upgrade

Reboot the host.

Install the Software Collections (SCL) repository. It is required for the latest version of Postgresql (11/12).

On Centos 7:

sudo yum install centos-release-scl

On RedHat Enterprise:

sudo yum-config-manager --enable rhel-server-rhscl-7-rpms

Install additional software packages if they were not installed during the operating system installation:

sudo yum install git ntp ntpdate

Download PIP installer script and run it (PIP is a tool for installing Python packages. Ansible is written in Python):

curl "https://bootstrap.pypa.io/get-pip.py" -o "get-pip.py"
sudo python get-pip.py

Install Ansible using PIP:

sudo pip install ansible

Verify Ansible version:

ansible --version

The output should be something like:

$ ansible --version
ansible 2.3.1.0
  config file = 
  configured module search path = Default w/o overrides
  python version = 2.7.12 (default, Nov 19 2016, 06:48:10) [GCC 5.4.0 20160609]

Verify that ansible version is 2.2+ or higher and python version is either 2.7 or 2.6. If the python version shows 3.x then the installation of Ansible is not correct. Contact the MiaRec representative for support.

Install the MiaRec ansible scripts

Clone the latest stable release of the MiaRec-Ansible Git repository in the /opt/ansible-miarec directory:

git clone --recursive https://github.com/miarec/ansible-miarec /opt/ansible-miarec

2. Prepare target hosts

This section describes the installation and configuration of operating systems for the target host(s).

MiaRec team officially supports the following operating system for the controller host:

  • Centos/RedHat 6 64-bit
  • Centos/RedHat 7 64-bit
  • Ubuntu Server 14.04 LTS 64-bit
  • Ubuntu Server 16.04 LTS 64-bit

2.1. Configure the operating system (Ubuntu)

Install additional software packages and configure Network Time Protocol (NTP). Before you begin, we recommend upgrading your system packages and kernel.

Update package source lists:

sudo apt-get update

Upgrade the system packages and kernel:

sudo apt-get dist-upgrade

Reboot the host to use the new kernel.

Install additional software packages if they were not installed during the operating system installation:

sudo apt-get install aptitude ntp ntpdate openssh-server acl python

Configure Network Time Protocol (NTP) in /etc/ntp.conf to synchronize with a suitable time source and restart the service:

service ntp restart

2.1. Configure the operating system (RedHat/Centos)

Install additional software packages and configure Network Time Protocol (NTP). Before you begin, we recommend upgrading your system packages and kernel.

Upgrade the system packages and kernel

sudo yum upgrade

Reboot the host to use the new kernel.

Install the Software Collections (SCL) repository. It is required for the latest version of PostgreSQL (11/12).

On Centos 7:

sudo yum install centos-release-scl 

On RedHat Enterprise:

sudo yum-config-manager --enable rhel-server-rhscl-7-rpms

Install additional software packages if they were not installed during the operating system installation:

sudo yum install ntp ntpdate openssh-server

Configure Network Time Protocol (NTP) in /etc/ntp.conf to synchronize with a suitable time source and start the service:

# On RedHat/Centos 7 (SystemD)
sudo systemctl enable ntpd.service
sudo systemctl start ntpd.service

2.2. Configured password-less access (optional, recommended)

Skip this step if the same host is used as a controller and target host.

Use the following instructions to setup password-less access from the Ansible controller to the target hosts.

3. Configure deployment

This section describes the configuration of MiaRec deployment. Such configuration should be done on the Ansible controller host.

3.1. Create inventory file (hosts)

The Ansible inventory file is an INI-formatted file that defines the hosts and groups of hosts upon which commands, modules, and tasks in playbooks operate.

Create the file /opt/ansible-miarec/hosts and add entries for every server you want to manage with Ansible (the Inventory File is highly configurable, see the Ansible documentation for more information):

vim /opt/ansible-miarec/hosts

Example 1. Local installation (all-in-one):

For local installation (when Ansible is running on the same host as MiaRec software), create the following hosts file:

[all]
; ---------------------------------
; All-in-one host
; Parameters:
;   - private_ip_address => ip address to access the host from other components (for example, web application needs to connecto to database)
; ---------------------------------

miarec ansible_connection=local private_ip_address=127.0.0.1


[all:vars]
; -------------------------------
; Version of installed packages
; -------------------------------
miarecweb_version   = x.x.x.x
miarec_version      = x.x.x.x
miarec_screen_version = x.x.x.x
postgresql_version  = 12
python_version      = 3.6.10
redis_version       = 5.0.5


; Set more secure password below
secret_db_password = secret


[recorder]
miarec

[screen]
miarec

[db]
miarec

[redis]
miarec

[web]
miarec

[celery]
miarec

[celerybeat]
miarec

Example 2. Remote installation via SSH (all-in-one):

If you are running Ansible playbook from the Controller host over SSH, create the following hosts file (replace 1.2.3.4 ip-address with the target host address):

[all]
; ---------------------------------
; All-in-one host
; Parameters:
;   - ansible_ssh_host => ip address to access the host using Ansible    
;   - ansible_root     => root account to login to server. Usually, 'root', but for Ubuntu it may be 'ubuntu'
;   - private_ip_address => ip address to access the host from other components (for example, web application needs to connecto to database)
;                           For 'all-in-one' setup, the private_ip_address should be set to '127.0.0.1' as all communication is done internally
; ---------------------------------

miarec ansible_host=1.2.3.4 ansible_port=22 ansible_user=root private_ip_address=127.0.0.1


[all:vars]
; -------------------------------
; Version of installed packages
; -------------------------------
miarecweb_version   = x.x.x.x
miarec_version      = x.x.x.x
miarec_screen_version = x.x.x.x
postgresql_version  = 12
python_version      = 3.6.10
redis_version       = 5.0.5

; Set more secure password below
secret_db_password = secret

[recorder]
miarec

[screen]
miarec

[db]
miarec

[redis]
miarec

[web]
miarec

[celery]
miarec

[celerybeat]
miarec

Example 3. Remote installation via SSH to multiple hosts (decoupled architecture):

If you deploy MiaRec components on dedicated hosts, create the following hosts file (replace ip-adresses accordingly):

[all]

; ---------------------------------
; All hosts
; Parameters:
;   - ansible_ssh_host => ip address to access the host using Ansible
;   - ansible_root     => root account to login to server. Usually, 'root', but for Ubuntu it may be 'ubuntu'
;   - private_ip_address => ip address to access the host from other components (for example, web application needs to connecto to database)
;                           For 'all-in-one' setup, the private_ip_address should be set to '127.0.0.1' as all communication is done internally
; ---------------------------------

rec1.miarec  ansible_ssh_host=192.168.88.11  private_ip_address=192.168.88.11  ansible_user=root 
rec2.miarec  ansible_ssh_host=192.168.88.12  private_ip_address=192.168.88.12  ansible_user=root 
db.miarec    ansible_ssh_host=192.168.88.15  private_ip_address=192.168.88.15  ansible_user=root 
redis.miarec ansible_ssh_host=192.168.88.16  private_ip_address=192.168.88.16  ansible_user=root 
web1.miarec  ansible_ssh_host=192.168.88.21  private_ip_address=192.168.88.21  ansible_user=root 
web2.miarec  ansible_ssh_host=192.168.88.22  private_ip_address=192.168.88.22  ansible_user=root 


[all:vars]
; -------------------------------
; Version of installed packages
; -------------------------------
miarecweb_version   = x.x.x.x
miarec_version      = x.x.x.x
miarec_screen_version = x.x.x.x 
postgresql_version  = 12
python_version      = 3.6.10
redis_version       = 5.0.5

; Set more secure password below
secret_db_password = secret

[recorder]
rec1.miarec
rec2.miarec

[screen]
rec1.miarec
rec2.miarec

[db]
db.miarec

[redis]
redis.miarec

[web]
web1.miarec
web2.miarec

[celery]
web1.miarec
web2.miarec

[celerybeat]
web1.miarec

In this example, we define two remote machines miarec1 and miarec2 and then place them into group miarec. Ansible playbook is executed against whole group.

Example 4. Remote installation to Amazon EC2 instance using SSH key (all-in-one):

The following example demonstrates how to deploy MiaRec to Amazon EC2 instances (Ubuntu) using SSH private key for connection.

[all]
; ---------------------------------
; All-in-one host
; Parameters:
;   - ansible_ssh_host => ip address to access the host using Ansible    
;   - ansible_root     => root account to login to server. Usually, 'root', but for Ubuntu it may be 'ubuntu'
;   - private_ip_address => ip address to access the host from other components (for example, web application needs to connecto to database)
;                           For 'all-in-one' setup, the private_ip_address should be set to '127.0.0.1' as all communication is done internally
; ---------------------------------

miarec ansible_host=1.2.3.4 ansible_port=22 ansible_user=ubuntu private_ip_address=127.0.0.1 ansible_ssh_private_key_file=~/.ssh/aws-key.pem 


[all:vars]
; -------------------------------
; Version of installed packages
; -------------------------------
miarecweb_version   = x.x.x.x
miarec_version      = x.x.x.x
miarec_screen_version = x.x.x.x 
postgresql_version  = 12
python_version      = 3.6.10
redis_version       = 5.0.5

; Set more secure password below
secret_db_password = secret

[recorder]
miarec

[screen]
miarec

[db]
miarec

[redis]
miarec

[web]
miarec

[celery]
miarec

[celerybeat]
miarec

3.2 Edit the version info in the inventory file

The hosts file contains the version of to be installed packages.

You need to edit at least the following parameters:

  • miarecweb_version
  • miarec_version
  • miarec_screen_version

You can find the latest MiaRec version info at Download page.

Example:

[all:vars]
; -------------------------------
; Version of installed packages
; -------------------------------
miarecweb_version   = 1.1.1.1
miarec_version      = 2.2.2.2
miarec_screen_version = 3.3.3.3
postgresql_version  = 12
python_version      = 3.6.10
redis_version       = 5.0.5

4. Run playbooks

MiaRec playbooks

MiaRec installation process is split on three playbooks:

  1. The prepare-hosts.yml Ansible foundation playbook installs the infrastructure services (PostgreSQL database, Redis, Apache web server, Python) and configures firewall (iptables). You need to run this playbook only once.

  2. The configure-firewall.yml playbook configures iptables firewall on target host(s). It is optional playbook, but for security reasons, it is recommended to run it. Alternatively, the firewall can be configured manually. You need to run this playbook when firewall rules change.

  3. The setup-miarec.yml playbook installs the MiaRec services, including web portal (miarecweb), recorder (miarec) and screen recording contoller (miarec_screen). Run this playbook for initial installation as well as for subsequent updates.

4.1. Run prepare-hosts.yml playbook to provision the server(s)

The playbook prepare-hosts.yml will install the required packages, like PostgreSQL database, Apache web server, Redis, Python, opens appropriate ports in firewall, etc. Normally you need to run this playbook only once when you prepare the system for MiaRec installation.

In case of remote installation, it is necessary to establish trust relationships between the controller and target machines. When speaking with remote machines, Ansible by default assumes you are using SSH keys. SSH keys are encouraged but password authentication can also be used where needed by supplying the option --ask-pass. You need to supply also the option --ask-sudo-pass if you are connecting to the remote server as non-root user.

When using password-less authentication (or when running Ansible locally on target host), you can simply run the following command:

cd /opt/ansible-miarec
ansible-playbook -i hosts prepare-hosts.yml

When using password authentication, you can run the following command and you will prompted to enter the password for SSH connection:

ansible-playbook -i hosts prepare-hosts.yml --ask-pass

Confirm satisfactory completion with zero items unreachable or failed:

PLAY RECAP ********************************************************************
...
miarec                :  ok=79   changed=42   unreachable=0    failed=0

4.2. Run configure-firewall.yml playbook to enable iptables on the server(s)

CAUTION! MiaRec installer uses iptables as a default firewall. It will be enabled automatically on the target system and other firewall software, if any, will be disabled. For example, on Centos 7, firewalld will be disabled. On Unbuntu 16.04, ufw will be disabled.

Alternatively, you can skip this step and configure firewall for MiaRec manually.

Run playbook:

ansible-playbook -i hosts configure-firewall.yml

4.3. Run setup-miarec.yml playbook to install or update MiaRec software

The playbook setup-miarec.yml will install the MiaRec software components (recorder, web portal, etc.). You need to run this playbook during initial installation as well as during upgrade of MiaRec to the new version.

To install/update MiaRec, run the following command:

ansible-playbook -i hosts setup-miarec.yml

Confirm satisfactory completion with zero items unreachable or failed:

PLAY RECAP ********************************************************************
...
miarec                :  ok=38   changed=25   unreachable=0    failed=0

5. Verify MiaRec operation

Use web browser to access MiaRec web portal. Navigate to Administration -> Maintenance -> System Log to check the errors.

Configure appropriate recording interface in Administration -> System -> Recording Interfaces and make a few test calls. Verify that calls are recorded.

It is recommended to reboot the target machine and verify all services are up and running after system reboot.

shutdown -r now
  • PostgreSQL database:

    service postgresql-9.5 status
    
  • Redis cache (use ping command. It should print PONG if success):

    /opt/redis/bin/redis-cli ping
    
  • Apache web server

    service httpd status
    
  • Celery task manager

    Centos 6 (init.d):

    service celeryd status
    

    Centos 7 (SystemD):

    systemctl status celeryd
    
  • Celery beat scheduler

    service celerybeat status
    
  • MiaRec recorder

    Centos 6 (Upstart):

    initctl status miarec
    

    Centos 7 (SystemD):

    systemctl status miarec
    
  • MiaRec scree recorder

    Centos 6 (Upstart):

    initctl status miarec_screen
    

    Centos 7 (SystemD):

    systemctl status miarec_screen