Storage management

Audio file encryption

File encryption overview

MiaRec provides rock-solid audio encryption functionality, ensuring all call recordings are securely stored. MiaRec encryption functionality helps companies confidently adhere to the highest corporate security standards and comply with legal regulations such as PCI-DSS, HIPAA, Dodd-Frank, and Sarbanes-Oxley.

Some key features of MiaRec audio file encryption:

  • Asymmetric encryption, where a public key is used for encrypting and a private key is used for decrypting
  • Administrator has control over who can play back (decrypt) the recordings
  • In a multi-tenant mode, each tenant has it's own unique encryption key
  • Encryption is applied to backup data, as well

MiaRec Audio File Encryption

Audio file encryption vs role-based access control

MiaRec role-based access control system provides protection of data from unauthorized access to the MiaRec web-portal. Everyone accessing the system must be an authenticated user with associated set of permissions.

Audio file encryption provides an additional layer of security over the role-based access control system in MiaRec. If encryption is enabled, then audio files are stored on a hard disk in encrypted format. This insures that even if unauthorized user gains physical access to the storage system, he/she has no ability to play back recordings because he/she doesn't have the private encryption key.

Download of encrypted recordings

When a user downloads individual call recordings through MiaRec web-portal, the file is decrypted in flight. The file is saved on the user's computer in unencrypted form.

However, when a user uses the bulk download feature and downloads multiple call recordings in ZIP archive, then the downloaded files are retrieved in encrypted form. The user cannot play back such call recordings unless he/she imports them into the MiaRec system together with private encryption key.

Encryption for backups

Use of file encryption is beneficial for backup data, as well. All recordings in backup archive can be encrypted.

Encryption in multi-tenant environment

In multi-tenant mode, each tenant has it's own encryption key. Even if an audio file from one tenant becomes available to another tenant, the latter could not play back, because the file is encrypted with a different key.

Additionally, in a multi-tenant hosted environment, MiaRec supports the following usage scenario: Tenant may provide the service provider with the public encryption key only. The tenant doesn't is not required to disclose their own private key to the service provider. This means that nobody on the service provider side - even system administrators - would be able to play back tenants' call recordings. To play back such call recordings, they should be uploaded to tenant's private network and imported into a local instance of MiaRec software.

Encryption algorithms

MiaRec encrypts every call recording with asymmetric encryption. For every recording, MiaRec generates a random AES encryption key. This symmetric encryption key is then encrypted using asymmetric encryption (one key for encryption - often referred to as the "public" key - and a different key for decryption - often referred to as the "private" key).

MiaRec uses Advanced Encryption Standard (AES) for symmetric encryption (256-bit key) and the Rivest-Shamir-Adleman (RSA) public key algorithm for asymmetric encryption (2,048-bit keys).

The details and theory behind the asymmetric encryption method is beyond the scope of this article. However, a good primer is available at https://en.wikipedia.org/wiki/Public-key_cryptography. In short, a public key is used for encrypting data and private key is used for decrypting it. The public key doesn't need to be stored securely. Anyone can access the public key, but no one can use the public key to decrypt the data that the public key encrypted. The only way users can decrypt data is with the private key.

User access to encryption keys

Administrators need to grant particular users access to encryption key(s) before they can play back (decrypt) audio files. Note, the administrator may grant access only to those encryption keys which are granted to him/her. If administrator (even if he/she has role "Root administrator") has no access to the encryption key, then he/she cannot grant access to other users for the same key.

MiaRec software never stores encryption keys in the database in plain text for security reasons. Even if an unauthorized party gains access to database files, he/she could not retrieve the private keys because they are stored in encrypted format. There is no way to gain user's private key without knowing the user's password.

Configuration check-list

Configure MiaRec audio file encryption as follows:

  1. Create new encryption key or use existing one for System or Tenant (in multi-tenant mode)

  2. Export/backup new encryption key and save it in secure place for recovery purposes

  3. Grant access to encryption key to authorized users

  4. Enable audio file encryption on System or Tenant profile.

Create new encryption key

Navigate to Administration -> Storage -> File Encryption to create new encryption key.

Note, in multi-tenant version, you need to create key for "System" account first. Then you can create tenant encryption key. On System account, you do not need to enable "Audio file encryption" unless you record calls into System tenant (which is not recommended).

MiaRec License

MiaRec License

Import encryption key

Encryption key can be imported from the existing key rather than generated from scratch.

Navigate to Administration -> Storage -> File Encryption to import the existing encryption key.

MiaRec License

MiaRec License

Export encryption key

Navigate to Administration -> Storage -> File Encryption to export the existing encryption key.

It is highly recommended to export all existing keys and store them in secure place for backup purposes. You may need such backup copies when all authorized people forgot their passwords or database is destroyed and you need to recover the audio files from archive.

MiaRec License

MiaRec License

MiaRec License

MiaRec License

Grant access to encryption key

Navigate to Administration -> Storage -> File Encryption, select the appropriate key and authorize users to access the data encrypted with the same key.

Administrators need to grant particular users access to encryption key(s) before they can play back (decrypt) audio files. Note, the administrator may grant access only to those encryption keys which are granted to him/her. If administrator (even if he/she has role "Root administrator") has no access to the encryption key, then he/she cannot grant access to other users for the same key.

MiaRec software never stores encryption keys in the database in plain text for security reasons. Even if an unauthorized party gains access to database files, he/she could not retrieve the private keys because they are stored in encrypted format. There is no way to gain user's private key without knowing the user's password.

MiaRec License

MiaRec License

Enable file encryption

Non-multi-tenant configuration

In a non-multi-tenant configuration, navigate to Administration -> Storage -> File encryption and click "Edit configuration** to enable encryption for all data.

MiaRec License

MiaRec License

Non-multi-tenant configuration

In a multi-tenant configuration, navigate to Administration -> Storage -> File encryption, select the appropriate tenant profile, then click "Edit configuration** to enable encryption for this particular tenant.

Alternatively, you can enable encryption on tenant profile under Administration -> User Management -> Tenants.

MiaRec License

Export of the encrypted files

An important aspect of any file encryption facility's design is that file data is never available in unencrypted form except to users that access the file via the encryption facility. This restriction particularly affects backup process, when data is exported to external storage.

MiaRec addresses this problem by keeping files in encrypted form during backup process. The backup utility don't have to be able to decrypt file data before backup.

It is safe to export encrypted files to backup archive. The backup archive may be imported back to the same system or to new system during recovery process. When importing data to new system, it is necessary to import old encryption key as well.

Audio settings

Navigate to Administration -> System Configuration -> Audio Settings to change audio format (stereo/mono), MP3 bitrate and other settings.

Audio settings

Backup and restore

Backup call recordings

Navigate to Administration -> Storage -> Export Recordings to create backup job. In version before March 2016, navigate to menu Administration -> Maintenance -> Backup Calls.

Backup job may be started manually or by schedule (for example, every night/week etc).

MiaRec Backup

Export to FTP server:

Export to FTP

Restore call recordings

Navigate to Administration -> Storage -> Import Recordings to create job. In version before March 2016, navigate to menu Administration -> Maintenance -> Restore Calls

Import calls

In "Edit Call Import Job" form specify the location of backup files and click on "Import now" button.

Import calls

Additional steps in case the backup files are located on network share

It is important to note, that backup files will be accessed by a program application running on MiaRec server rather than from the computer on which you open MiaRec web portal. This means that even if you can access backup files from your own computer, the same files may be unaccessible from MiaRec server.

If backup files are stored on a network share, then on Windows servers you should use correct UNC path like \server\dir, on Linux servers you should mount the network share to a local file system, for example, /mount/backup.

When using UNC path on Windows, take into account that such path will accessed by a process running as a Windows service application. By default service applications are running under credentials of LOCAL_SYSTEM user account. This is internal user account, which has no access to network. To solve this issue, you would need to change parameters of "MiaRec Celery" service and run it under credentials of some user account, which can access the backup network share.

The process of call importing will be started and the progress will be displayed on web-page.

Import calls

Location for audio files

Navigate to Administration -> System Configuration -> Storage to view/edit location of audio files and filename format.

MiaRec Storage Settings

Click on Edit Configuration to modify settings.

MiaRec Storage Settings

Audio File Name Format is a parametrized format of audio file name. This is very powerful way of configuring audio files location. Parameters are described in details in article File name format.

File name format

MiaRec supports flexible naming of audio files. It is possible to include date/time, ip-address, phone number and other call parameters into file name.

Example:

C:\Recordings\%{setup-time#%Y%m%d}\%{setup-time#%Y-%m-%d-%H%M%S}.mp3

In above example audio files are stored inside directory C:\Recordings\.

For each day a new sub-directory is created (for example, C:\Recordings\20110203\ for 3rd of February 2011). This is done with the help of parametrized string %{setup-time#%Y%m%d}, which is converted to date (read details about parametrized strings below).

The file name consists of a date and time of when a call is started, for example, 20110203104522.mp3.

If two or more calls are started at the same time, then MiaRec appends a unique number at the end of file name, for example, 20110203104522_2.mp3, 20110203104522_3.mp3 etc.

Parameters have the following format:

%{parameter-name} or
%{parameter-name#format-string}

where:

  • parameter-name is a name of call parameter (see Table 1)
  • #format-string is an optional format of call parameter (see Time formatting).

Examples:

  • %{caller-number}
  • %{setup-time#%Y%m%d}

Table 1. Supported parameters inside file path

ParameterDescription
%{call-id}Unique id of a call, which is assigned to each recorded call by MiaRec
%{parent-call-id}Id of a call, which is a parent to the current call. The meaning of this parameter depends on particular voip protocol. For example, for Avaya H.323 protocol, when call is put on hold and then retrieved from hold, the new audio file will be created. In this case %{parent-call-id} points to the very first call part.
%{protocol-call-id}

Id of a call, which is assigned by IP PBX.

This value is valid only for supported voip protocol (SIP, Skinny, H.323 and MGCP).

For example, for SIP protocol this value is retrieved from header "Call-ID" inside SIP INVITE message.

%{protocol-tracking-id}

Id of a call interaction assigned by IP PBX. Usually IP PBX assigned the same tracking id to related calls, like transferred from one agent to another. 

For Avaya Aura Communication Manager, it is UCID.

For Broadworks, it is extTrackingID.

Available since August 2018

%{call-state}

Phase (state) of the call. It is a numeric value, one of:

  • 0 - Idle
  • 1 - Initiated. The first phase of a call: the caller sent invitation to the callee
  • 2 - Accepted. The callee received invitation and confirmed this
  • 3 - Alerting. The callee started ringing
  • 4 - Connected. The call was answered
  • 5 - Disconnecting. The call was initiated for disconnecting by one of  parties
  • 6 - Disconnected. The call was completed (disconnected)
  • 7 - Hold. The call was put on Hold
  • 8 - Transferred. The call was transferred to the third party
  • 9 - Deleted. The call was deleted from disk (see Recording filters)
%{record-state}

State of the audio recording. It is a numeric value, one of:

  • 10 - Active. Call is active at the moment and recording is in progress
  • 20 - Partial recording. Recording of call was stopped because of not enough licenses
  • 30 - Finished. Call is finished. Audio was recorded in full
  • 40 - Ignored. Call is ignored by recording filters.
%{voip-protocol}

Voip protocol of the call. It is a numeric value, one of:

  • 0 – Unknown (not recognized protocol). Call is recorded from RTP packets
  • 1 - SIP
  • 2 - H.323
  • 4 - SCCP (Cisco Skinny)
  • 5 - MGCP
  • 6 - Avaya (H.323 protocol with proprietary extensions)
  • 7 - Nortel UNISTIM
  • 8 - TAPI 
  • 9 - MGCP PRI Backhaul (it is used between Cisco CCM and Voice Gateway)
  • 10 - Alcatel (proprietary protocol used by Alcatel OmniPCX)
  • 11 - Avaya RTP (passive recording w/o signaling)
  • 12 - Avaya TSAPI (TSAPI + port mirroring recording)
  • 13 - SIPREC
  • 14 - Cisco Built-in-Bridge
  • 15 - NEC SIP (SIP protocol with NEC proprietary extensions)
  • 16 - ED137
  • 17 - Cisco Built-in-Bridge passive
  • 18 - SIPREC passive
  • 19 - Avaya DMCC
%{protocol-call-direction}

Call direction reported by IP PBX, available for active recording interfaces only. It is a numeric value, one of:

  • 0 - Unknown
  • 1 - Outbound
  • 2 - Inbound

Available since August 2018

%{setup-time}
Time when call was established (when a called party received incoming call message). See Time formatting
%{alerting-time}
Time when phone started ringing on called party side. See Time formatting
%{connect-time}
Time when call was answered. See Time formatting
%{disconnect-time}
Time when call was disconnected. See Time formatting
%{duration}
Duration of voice part of a call in seconds. This is a difference beween %{connect-time} and %{disconnect-time}
%{total-duration}Total duration of a call in seconds. This is a difference beween %{setup-time} and %{disconnect-time}
%{filename}

Name of audio file without full path (for example, 20110410104600.mp3)

Caution! This value is available only to a recording engine when file is initially created. It is not available to post-processing jobs, like export, relocate, etc.

%{filename-full}

Full path to the file, including directory (for example, C:\Recordings\20110410104600.mp3)

Caution! This value is available only to a recording engine when file is initially created. It is not available to post-processing jobs, like export, relocate, etc.

%{filename-dir}

Directory path to the file, excluding drive letter (for example, \Recordings\)

Caution! This value is available only to a recording engine when file is initially created. It is not available to post-processing jobs, like export, relocate, etc.

%{caller-number} or

%{callee-number}

Phone number of caller/callee

%{caller-name} or

%{callee-name}

Name of caller/callee. This parameter is protocol-dependent. For example, for SIP protocol name is extracted from "From" and "To" sip headers

%{caller-id} or

%{callee-id}

Id of a caller/callee. This paramter is protocol-dependent. For example, for SIP protocol it is SIP URI

%{caller-ip} or

%{callee-ip}

Ip-address of caller/callee

%{caller-port} or

%{callee-port}

Port of caller/callee

%{caller-mac} or

%{callee-mac}

Mac-address of caller/callee

%{transfer-from-number}

%{transfer-from-name}

%{transfer-from-id}

Name, number and id of party, from which the call was transferred. This parameter is available only for Skinny protocol.

%{transfer-to-number}

%{transfer-to-name}

%{transfer-to-id}

Name, number and id of party, to which the call was transferred. This parameter is available only for Skinny protocol.
%{sip-header-invite}

Value of specific SIP header inside INVITE message. The name of header is specified after hash (#) symbol.

Examples:

  • %{sip-header-invite#User-Agent}
  • %{sip-header-invite#X-My-header}

Caution! This value is available only to a recording engine when file is initially created. It is not available to post-processing jobs, like export, relocate, etc.

%{BroadWorks-userID}

Broadworks User ID

%{BroadWorks-groupID}

Broadwors Group ID

%{BroadWorks-serviceProviderID}

Broadworks Service Provider ID

%{MetaSwitch-recorderParty}Metaswitch CFS User Extension
%{MetaSwitch-userName}Metaswitch CFS User Name
%{MetaSwitch-businessGroup}Metaswitch CFS Business Group Name
%{MetaSwitch-systemName}Metaswitch CFS System Name
%{agent-id}Avaya Agent ID
%{agent-name}Avaya Agent Name
%{orig-caller-number}Originally Caller Number (if different from caller-number)
%{orig-caller-name}Originally Caller Name (if different from caller-name)
%{orig-callee-number}Originally Dialed Number (if different from callee-number)
%{orig-callee-name}Originally Dialed Name (if different from callee-name)

%{user-id}

%{user-name}

ID, name of user, the call recording is assigned to. If the call is an internal (i.e. assigned to multiple users), then this value points to the first user only.

Note: this value is available in post-processing jobs only (Export/Replication/File relocation). It is not available for the initial filename creation by the recorder process (configured at menu Administration -> Storage -> File Location)

Available since May 2018.

%{group-id}

%{group-name}

ID, name of group, the call recording is assigned to. If the call is an internal (i.e. assigned to multiple users) or user belongs to multiple groups, then this value points to the first group only.

Note: this value is available in post-processing jobs only (Export/Replication/File relocation). It is not available for the initial filename creation by the recorder process (configured at menu Administration -> Storage -> File Location)

Available since May 2018.

%{tenant-id}

%{tenant-name}

ID, name of tenant, the call recording is assigned to.

Note: this value is available in post-processing jobs only (Export/Replication/File relocation). It is not available for the initial filename creation by the recorder process (configured at menu Administration -> Storage -> File Location)

Available since May 2018.

%{acd-number}

%{acd-name}

%{acd-id}

Number/name/id of ACD.

Broadworks and Avaya envorinments only.

Available since July 2018.

Example 1

C:\Recordings\%{setup-time#%Y%m%d%H%M%S}.mp3

%{setup-time#%Y%m%d%H%M%S} will be replaced with a date and time of when a call was started. For example, if a call was started on 1st of May 2007 at 10:56:34, it will be stored into directory ‘C:\Recordings’ with the filename ‘20070501105634.mp3’.

Note: If two or more calls were started at the same time, a unique decimal suffix will be added to every file name (expect the first one), like: ‘20070501105634_2.mp3’, ‘20070501105634_3.mp3’ etc.

Example 2

C:\Recordings\%{setup-time#%Y%m%d}\File.mp3

This example contains a parameterized string inside a directory path. This means that files will be stored into sub-directories with name %{setup-time#%Y%m%d} (which will be replaced by a date of a call, for example, ‘20070501’). If such directory doesn’t exist, it will be created automatically.

In this example calls will be grouped into directories by date, like:

Directories structure

For every new day a separate directory will be created (a directory is not created if no calls were recorded at that day).

Audio file names in this example will be File.mp3, File_2.mp3, File_3.mp3 and so on.

Example 3

C:\Recordings\%{caller-ip}\File.mp3


%{caller-ip} will be replaced with ip-address of a caller, for example 192.168.0.1.

Calls will be grouped into directories by caller ip-address, like:

Directories structure

Example 4

C:\Recordings\%{setup-time#%Y%m}\%{setup-time#%d}\%{caller-ip}\File.mp3

In this example multiple parameter replacements occur:

  • %{setup-time#%Y%m} will be replaced with a year and month of a call (YYYYMM). For 1st of May 2007 it will be 200705.
  • %{setup-time#%d} will be replaced with a day of a call (DD). For 1st of May 2007 it will be 01. 
  • %{caller-ip} will be replaced with an ip-address of a caller, for example 192.168.0.1.

Calls will be grouped into directories by months, then by days and then by callers' ip-addresses, like:

Directories structure

Read also:

Time formatting inside file name

All date/time parameters support a formatting attribute. Formatting attribute is specified after hash (#) symbol.

For example:

  • %{setup-time#%Y} will return year, like: 2011
  • %{setup-time#%m} will return month, like: 02
  • %{setup-time#%Y-%m} will return both year and month, like: 2011-02

Table 1. Formatting codes

CodeDescription
%aAbbreviated weekday name
%A Full weekday name
%bAbbreviated month name
%BFull month name
%dDay of month as decimal number (01 – 31)
%HHour in 24-hour format (00 – 23)
%IHour in 12-hour format (01 – 12)
%jDay of year as decimal number (001 – 366)
%mMonth as decimal number (01 – 12)
%MMinute as decimal number (00 – 59)
%pA.M./P.M. indicator for 12-hour clock
%SSecond as decimal number (00 – 59)
%UWeek of year as decimal number, with Sunday as first day of week (00 – 53)
%wWeekday as decimal number (0 – 6; Sunday is 0)
%WWeek of year as decimal number, with Monday as first day of week (00 – 53)
%yYear without century, as decimal number (00 – 99)
%YYear with century, as decimal number
%%Percent sign
%uMicroseconds as decimal number

 

Table 2. Examples of time formatting

Format stringResult
%Y-%m-%d2004-11-10
%H%M%S160201
%I%M%S040201
%d %b %Y, %A10 Nov 2004, Wednesday

Note, for all examples, we used the same date/time, which is “10th of November 2004 16:02:01”. This day is a Wednesday.

Read also:

Replication

MiaRec multi-master asynchronous replication

MiaRec solution implements data replication with the following characteristics:

  • Multi-master
  • One-way, two-way or N-way
  • Asynchronous
  • Application-level
  • GEO distributed
  • Continuous, manual or scheduled
  • Auto resume after network breakdown

This articles describes in details each of these characteristics and compares MiaRec solution with alternatives. The competitive solutions are built usually on file-storage based replication and have a number of weaknesses discussed below.

How it works

When recording of each individual call is completed, MiaRec pushes it into queue for automatic replication to other server(s) in a cluster. Such data replication may be started immediately upon call completion or scheduled to specific time of day (for example, at night).

Besides replication of call recordings, MiaRec replicates also user data in one-way or two-way directions. The updates to user data is automatically uploaded to other servers in a cluster.

Multi-master asynchronous replication

Replication architecture of MiaRec has the following characteristics:

  • Multi-master. Any of servers in a cluster can be used for recording tasks at any time. It is possible to use multiple recorders simultaneously for load balancing purposes.

  • Asynchronous replication. Data is replicated asynchronously. Data synchronization can be triggered by schedule (once per hour/day/week) or continuously upon each individual call completion. It works seamlessly in GEO-redundant architecture when datacenters are located too far from each other. In a contrast, other solution may use synchronous replication, which require low latency (less than 5ms) connection between datacenters, this is equal to maximum 100km distance between servers. With a synchronous replication, if a link between datacenters is down even for 1 second, the redundant server is removed from a cluster and manual re-synchronization is required between servers. Automatic restore of cluster is not possible by design with synchronous replication.

  • Application-level replication. MiaRec implements replication internally on application level. It has a few advantages: cost, easy management and selective replication. In a contrast, other solutions may use replication on database level or disk level (SAN). SAN replication is supported only in highly expensive enterprise SAN disk arrays. In both of these competitive solusions (database replication and SAN replication) the selective replication is not supported.

Multi-master vs master-slave replication

Multi-master replication (MiaRec) Master-slave replication (other vendors)
All servers run as master servers, thus you can record calls on any of servers at any time or even simultaneously to multiple servers.

This makes system highly flexible in a way that any operation can be processed in any server which enables better load balancing.

However, such flexibility brings the challenge of keeping servers consistent. A conflict occurs if more than one server tries to update the same object. In MiaRec we solved this issue with the following mechanisms:

  1. Careful design of database structure from scratch to address unique redundancy requirements. We do not use integer auto-incremental fields for IDs. Instead we use UUID all over the database to guarantee uniqueness through multiple servers.

  2. Replication is implemented on application level instead of database engine or disk-level. More about this later.

In master-slave replication, there is only one server in the system which is capable of recording data. All other replicating servers are called slaves and can only accept read-only requests.

In master-slave replication, the master server becomes overwhelmed and system suffers from scalability due to using a single server for write operations (call recording).

Setup of automatic fail-over mechanism can be tricky. When master server becomes unavailable, one of the slaves can be promoted as a master. When the master server is back, it usually stays in off-line mode and requires manual re-synchronization of servers to assure data consistency. Such synchronization process is quote time consuming and it is recommended to have at least 3 servers in a cluster (1 master and 2 slaves) in order to avoid single point of failure situations while master server is in off-line mode.

If such configuration is used in GEO-redundant setup, it may create too much burden to administration staff in case of frequent issues with connection between datacenters.

Asynchronous vs synchronous replication

Asynchronous replication (MiaRec) Synchronous replication (other vendors)
In asynchronous replication, an incoming request is processed and get committed on the receiving server without propagating it to other replicating servers simultaneously. Instead, committed request are deferred and sent to all other replicating servers asynchronously. Once replicating servers receive these deferred request, they process them and make themselves synchronized.

Asynchronous replication utilizes network resources intelligently, creates less traffic, and provides higher performance. Deferring multiple request and propagating them all as a big chunk of requests is much more efficient rather than to propagate each of them separately. Operation latency is reduced as opposed to synchronous replication because a server can go ahead and process a request without need to talk with other servers to commit it. It also provides better scalability since response time of a server is independent from the number of replicating servers, and generated network traffic is proportional to the number of replicating servers. Moveover, network latency introduced due to the geographical distance between replicating servers can be tolerated and hidden since requests are deferred and propagated asynchronously.

Additionally, asynchronous replication can be scheduled to execute during less busy hours, like at night or weekends.

In synchronous replication, incoming requests are propagated to and processed by all replicating servers immediately. The benefits of synchronous replication is to guarantee that data is consistent at all servers at any time.

While propagating requests and synchronizing servers, two-phase commit protocol is used. When a request comes in a sever, the same request is also forwarded immediately to all replicating servers. All servers have to process incoming request to see if it is OK to be committed, and have to inform the propagating server in this regard. If and only if all replicating servers inform that request can be committed, then second message is propagated to commit the request in all replicating servers. If any replicating server complains about the request, than abort message is propagated and all servers have to disregard the request.

Although it ensures that replicating servers are synchronized immediately when a request is committed and prevent consistencies may occur otherwise, it generates huge network traffic due to high number of sends and receives to decide to commit or abort. It increases processing latency which degrades operation performance since operation has to wait until all replicating servers have been synchronized. Scalability also suffers from increasing number of replicating metadata servers that tend to create exponentially growing network traffic and processing latency that ends up with longer response time.

Synchronous replication is not suitable for GEO-redundancy when distance between datacenters is more than 100km.

Application-level vs Storage array-based replication

Application-level replication (MiaRec) Storage array-based replication (other vendors)
MiaRec replication mechanism is based on knowledge of data. This allows it to selectively replicate only the necessary data. For example, administrator may enable continuous (as soon as possible) replication for call recording data and for the rest of data (like logs) schedule replication during off-hours (at night, for example).

Additionally, it is possible to set filtering criteria for replication. For example, replicate only call recordings of particular tenant(s) or group(s).

Having knowledge of data allows MiaRec application to resolve conflicts intelligently. For example, if the same user record is updated from multiple servers simultaneously, then administrator may decide to resolve conflicts automatically based on priorities or manually.

In a contrast to storage array-based replication (SAN), MiaRec replication mechanism supports any storage, like NAS, local, virtual environment. It doesn’t depend on hardware. It is possible to mix different storage types in the same clustomer, for example, replicate form local or NAS storage to SAN.

MiaRec application-level replication supports multi-master architecture, which is not possible with a storage array-based replication. As a result, utilization of hardware is much better due to using second storage in load balancing configuration. MiaRec supports also replication to multiple servers simultaneously.

Application-level replication is tolerant to temporary problems with a link between replicating servers. In case of problems with a link between datacenters, MiaRec replication process is postponed and automatically resumed when link is restored. No data loss occurs due to in this case.

Storage array-based replication is expensive. Usually it is available only in enterprise SAN disk arrays.

It doesn’t have knowledge of data that is stored on disk. As a result, it is not possible to configure selective replication. You need to replicate an entire SAN or nothing.

Storage array-based replication works only for a pair of SAN arrays of exactly the same vendor/model and size. It is not possible to mix SANs from different vendors or even different models of the same vendor.

SAN replication usually supports both asynchronous and synchronous replication, but the latter is not suitable in GEO-redundant environment because it works only for a distance up to 100km between datacenters.

When using SAN replication in asynchronous mode, it suffers from ineffectiveness of investments. One of SAN-arrays in a pair is used in passive mode most of the time until disaster occurs.

In case of DR, a switch from primary SAN to the secondary usually occurs automatically, but a reverse operation requires the manual intervention of human.

In case of problems with a link between datacenters, data on primary and secondary SAN arrays becomes inconsistent and requires manual re-synchronization, which is very time consuming.

Use cases for replication

MiaRec supports advanced replication mechanism between two or more MiaRec servers.

Such replication may be configured one-way or two-way. MiaRec server may play role of target (recipient) or source (sender) or both roles at the same time.

The following scenarios are supported:

Replication to backup storage

Replication to backup storage

Replication to centralized storage

Replication to centralized storage

Redundant recorder with BroadWorks SIPREC

Broadworks SIPREC redundant recorder

Redundant recorder with Cisco Built-in-Bridge

Cisco Built-in-Bridge redundant recorder

Upload call recordings from service provider to customer network

Upload of recordings to customers

Configuring target server (recipient)

Step 1. Create Storage Target for the received recordings

Navigate to Administration -> Storage -> Storage Targets, click Add button to create new storage target for the received files from a remote server.

Supported storage target types:

  • Local File System (the same server, where the web portal component is running on)
  • Network Share (SMB)
  • FTP/FTPS Server
  • SFTP Server
  • Amazon S3 bucket

In this example, we create a Local File System storage target, i.e. the received files will be stored on the local server, where the MiaRec web portal is running on.

Add storage target

When Local File System storage is used and the web portal is running on Linux, then you need to change ownership to the folder on disk. Execute the following command (change the file path as necessary):

On Centos:

chown -R apache:apache /var/miarec/replicated-recordings

On Ubuntu:

chown -R www-data:www-data /var/miarec/replicated-recordings

Caution! Do not use the same folder for storing the locally recorded files as well as replicated files as it will cause permission issues. The locally recorded files are stored by default at /var/miarec/recordings.

Step 2. Create the incoming replication token.

Navigate to Administration -> Storage -> Replication to configure incoming replication token

Click on Add token button to create a secure token for incoming replication. This secure token will be used by the sender server to upload data to the target server.

Fill out the following parameters:

  • Replication token. A replication token, auto-generated. Optionally, it can be modified.
  • Remote ip address (recommended). The IP-address or IP network mask of the sender server. This parameter can be set to "0.0.0.0/0" to accept replication data from any IP-address.
  • Replicate data. Data that is replicated
  • Update existing data. A conflict resolution strategy when the same record is updated on both servers.
  • Storage target. A location of the received data (audio files)
  • Directory (optional). A sub-directory within the Storage Target path
  • Filename format. A format for filenames and, optionally, directories. The replication process can inject various call metadata attributes into file/directory names. For example, it can create directory for each day in format YYYYMMDD and then include caller-number and called-number into file name. More details about file name format Tenant (optional). When specified, the replicated data will be imported into the specified tenant account.

The same target server may receive data from multiple source servers. You will need to create a token for each source server.

Replication token

Hit Save button.

Configuring replication server (sender)

Navigate to Administration -> Storage -> Replication -> Outgoing Replication on the source (sender) replication server to create an outgoing replication job.

Click New Job button to create the replication job. If necessary, you may create multiple replication jobs to upload the same recordings to multiple target servers simultaneously.

Fill out the required configuration parameters:

  • Access scope (visible in multi-tenant version only). Specifies what tenants are replicated to the target server.
  • Target server url. The URL (domain or IP-address) or the target server web portal.
  • SSL verify. If enabled and a domain name is used for the Target server url, then the sender automatically verifies the target server's SSL certificate (recommended).
  • Replication token. A secure replication token created on the target server. See the previous step
  • Parallel upload. A number of parallel upload workers sending data simultaneously. Depending on network latency, an increase of the parallel workers may improve a replication speed due to better bandwidth utilization.
  • Upload chunk size. A maximum file chunk per one upload request. Depending on network bandwidth/latency, an increase of this attribute may improve a replication speed.
  • Replication mode. Full or incremental replication mode.

    • Full replication mode will upload all call recordings to target server everytime the job is started. It will gracefully skip upload process if the target server contains such recordings already.

    • Incremental replication mode remembers which records have been uploaded already to the target server and do not process them on next start. Such mode is useful when job is scheduled for periodic replication (every hour/day etc). It will work a lot faster than the full replication mode because it will skip automatically the previously uploaded recordings.

  • Replicate data. Type of data to be replicated (audio files, call metadata, users configuration).

  • Remove after replication. The recordings can be deleted automatically after successful replication.

Replication job

Each replication job supports filtering criteria to limit what call recordings are uploaded to the target server. For example, you may configure replication for specific group of users only.

Replication job

Replication job may be started manually or automatically by schedule. Schedule may be configured by time (for example every hour/day/week) or automatic continuous replication. With continuous replication call recordings are uploaded to the target server immediately upon call completion.

Replication job

Optionally, the replication process may assign/unassign a category once the recording is replicated. This capability can be used to create a chain of post-processing, like relocate files first, then replicate, then transcribe, etc.

Replication job

Status of replication job is available on job page. For incremental replication mode MiaRec stores statistics of replicated calls per day.

Retention policy


Navigate to Administration -> Storage -> Retention Policies to add one or more retention policies.

You can create more than one retention policies. For example, one group of users will have retention period 3 years, while other groups will have retention period 7 years.

Click on "New Job" to create a retention policy job.

Inside the job settings you can specify the filtering criteria, for example, delete recording that are older than 180 days, limit to a particular group of users, etc.

Retention job may be started manually or automatically by schedule.

Retention policy

Results of a retention job execution are displayed on the job page.

Retention policy