Upgrading Configuration Conflicts

Upgrading Configuration Conflicts#

On upgrade, the configuration files are changed. If you have previously edited those files, the updates may not apply cleanly.

Configuration differences are particularly prevalent in multi-node Squirro clusters. Follow this guide to resolve those conflicts.

Introduction#

Squirro uses RPM packages to install on Linux systems. One functionality of these packages is, that they install configuration files. These files are usually located in the /etc directory and can be edited to change the configuration of the installed services.

In Squirro’s case, the following directories contain configuration files relevant to the running of its services:

  • /etc/squirro - configuration for the Squirro microservices

  • /etc/nginx/conf.d - web server configuration to serve the Squirro application and microservices

  • /etc/elasticsearch - Elasticsearch configuration

  • /etc/sysconfig - Squirro environment settings (filename: squirro)

Identifying Conflicts#

If you have edited any of the files in these directories and upgrade Squirro, this may result in .rpmnew files being written.

You can find those files with the following command:

find /etc -name '*.rpmnew'

This command will find all the configuration conflicts, not just the ones for Squirro services. Use the folder list above as a reference if you only want to identify the Squirro configuration conflicts.

Resolve File#

For each file that has been identified in the previous section, follow this process to resolve any conflicts.

In the example we assume that the find command returned the file /etc/squirro/test.ini.rpmnew.

  1. Change into the right directory: cd /etc/squirro

  2. Compare the rpmnew file to the original: diff -u test.ini test.ini.rpmnew

  3. Now there are two ways to resolve this conflict:

    • If you want to discard all your changes, you can simply move the rpmnew file over the old file: mv test.ini.rpmnew test.ini

    • To manually copy some of the configuration changes from the rpmnew file to your configuration file, edit the file: nano test.ini or vi test.ini

Warning

See the following section regarding the need to remove rpmnew files after resolving conflicts with INI files.

Deleting RPMNEW File After Fixing INI Conflicts#

If you are resolving conflicts with INI files, for example common.ini, you must remove the .rpmnew file after you have resolved the conflicts.

Usage of Diff#

To compare the files, the Unix utility diff can be used. The recommended usage is a follows:

diff -u test.ini test.ini.rpmnew

This outputs the changed lines prefixed with + for lines that have been added in the rpmnew file compared to the old configuration and prefixed with - for removed lines. Additionally some lines before and after each change are included for better context.

When resolving these conflicts, proceed as follows:

  • Ignore any of your changes - even though they are flagged as one addition and one removal. For example if you changed the database password, diff will flag an addition of the default database string and removal of your custom database string.

  • Add any new configuration options that are not present in your config file.

  • Uncomment or remove any removed configuration options. You can comment out lines by prefixing them with the # character.

Multi-Node Squirro Clusters#

This section highlights configuration settings specific to multi-node Squirro clusters. If you run Squirro on a single system, you can safely skip this section. On multi-node clusters, Squirro services do not always use the local MySql or Redis servers, but to one of the servers that acts as the “”master:.

As a result, all settings called either db_endpoint_discovery (for MySQL connections) or endpoint_discovery (for Redis) need to be set to true.

In contrast, single-node Squirro installations and new installations (or upgrades introducing new Squirro services or existing Squirro services whose new version uses a new MySQL or Redis database) introduce out-of-the-box settings of false for both endpoint_discovery and db_endpoint_discovery.

Here is an example based on the Squirro Topic-Api service corresponding to the Squirro 2.4.2 release:

Single-node

Multi-node

/etc/squirro/pipeline.ini

[handler_file]
application = pipeline

[pipeline]
item_debug_output_directory = /var/lib/squirro/pipeline/debug

[fetch]
fetch_directory = /var/cache/squirro/pipeline/pages

[redis_bloom]
endpoint_discovery = false
password = ...

[redis_in_flight]
endpoint_discovery = false
password = ...

[redis_pipeline_retries]
endpoint_discovery = false
password = ...

[queues_local]
endpoint_discovery = false
redis_password = ...

/etc/squirro/pipeline.ini

[handler_file]
application = pipeline

[pipeline]
item_debug_output_directory = /var/lib/squirro/pipeline/debug

[fetch]
fetch_directory = /var/cache/squirro/pipeline/pages

[redis_bloom]
endpoint_discovery = true
password = ...

[redis_in_flight]
endpoint_discovery = true
password = ...

[redis_pipeline_retries]
endpoint_discovery = true
password = ...

[queues_local]
endpoint_discovery = true
redis_password = ...

Guidance for resolving differences in multi-node clusters are:

  1. Ensure that all properties endpoint_discovery and db_endpoint_discovery are set to true

  2. If a /etc/squirro/*.ini.rpmnew file has an endpoint_discovery or db_endpoint_discovery setting (set to false), apply that setting as well as a password to the same section (adding it if it does not exist yet).

    For example, the upgrade between Squirro versions 2.4.1 and 2.4.2 adds the redis_pipeline_retries section along withendpoint_discovery = false and the password which is only right for single-node clusters. For multi-node clusters you’d need to copy the redis_pipeline_retries section in /etc/squirro/pipeline.ini and modify endpoint_discovery to true as has been done in the Multi-Node example above.

Contents