Upgrading Configuration Conflicts
Upgrading Configuration Conflicts#
On upgrade, the configuration files are changed. If you have previously edited those files, the updates may not apply cleanly.
Configuration differences are particularly prevalent in multi-node Squirro clusters. Follow this guide to resolve those conflicts.
Squirro uses RPM packages to install on Linux systems. One functionality of these packages is, that they install configuration files. These files are usually located in the
/etc directory and can be edited to change the configuration of the installed services.
In Squirro’s case, the following directories contain configuration files relevant to the running of its services:
/etc/squirro- configuration for the Squirro microservices
/etc/nginx/conf.d- web server configuration to serve the Squirro application and microservices
/etc/elasticsearch- Elasticsearch configuration
/etc/sysconfig- Squirro environment settings (filename:
If you have edited any of the files in these directories and upgrade Squirro, this may result in
.rpmnew files being written.
You can find those files with the following command:
find /etc -name '*.rpmnew'
This command will find all the configuration conflicts, not just the ones for Squirro services. Use the folder list above as a reference if you only want to identify the Squirro configuration conflicts.
For each file that has been identified in the previous section, follow this process to resolve any conflicts.
In the example we assume that the
find command returned the file
Change into the right directory:
rpmnewfile to the original:
diff -u test.ini
Now there are two ways to resolve this conflict:
If you want to discard all your changes, you can simply move the
rpmnewfile over the old file:
mv test.ini.rpmnew test.ini
To manually copy some of the configuration changes from the
rpmnewfile to your configuration file, edit the file:
Usage of Diff#
To compare the files, the Unix utility
diff can be used. The recommended usage is a follows:
diff -u test.ini test.ini.rpmnew
This outputs the changed lines prefixed with
+ for lines that have been added in the
rpmnew file compared to the old configuration and prefixed with
- for removed lines. Additionally some lines before and after each change are included for better context.
When resolving these conflicts, proceed as follows:
Ignore any of your changes - even though they are flagged as one addition and one removal. For example if you changed the database password, diff will flag an addition of the default database string and removal of your custom database string.
Add any new configuration options that are not present in your config file.
Uncomment or remove any removed configuration options. You can comment out lines by prefixing them with the
Multi-Node Squirro Clusters#
This section highlights configuration settings specific to multi-node Squirro clusters. If you run Squirro on a single system, you can safely skip this section. On multi-node clusters, Squirro services do not always use the local MySql or Redis servers, but to one of the servers that acts as the “”master:.
As a result, all settings called either
db_endpoint_discovery (for MySQL connections) or
endpoint_discovery (for Redis) need to be set to
In contrast, single-node Squirro installations and new installations (or upgrades introducing new Squirro services or existing Squirro services whose new version uses a new MySQL or Redis database) introduce out-of-the-box settings of false for both
Here is an example based on the Squirro Topic-Api service corresponding to the Squirro 2.4.2 release:
[handler_file] application = pipeline [pipeline] item_debug_output_directory = /var/lib/squirro/pipeline/debug [fetch] fetch_directory = /var/cache/squirro/pipeline/pages [redis_bloom] endpoint_discovery = false password = ... [redis_in_flight] endpoint_discovery = false password = ... [redis_pipeline_retries] endpoint_discovery = false password = ... [queues_local] endpoint_discovery = false redis_password = ...
[handler_file] application = pipeline [pipeline] item_debug_output_directory = /var/lib/squirro/pipeline/debug [fetch] fetch_directory = /var/cache/squirro/pipeline/pages [redis_bloom] endpoint_discovery = true password = ... [redis_in_flight] endpoint_discovery = true password = ... [redis_pipeline_retries] endpoint_discovery = true password = ... [queues_local] endpoint_discovery = true redis_password = ...
Guidance for resolving differences in multi-node clusters are:
Ensure that all properties
db_endpoint_discoveryare set to
/etc/squirro/*.ini.rpmnewfile has an
db_endpoint_discoverysetting (set to
false), apply that setting as well as a password to the same section (adding it if it does not exist yet).
For example, the upgrade between Squirro versions 2.4.1 and 2.4.2 adds the
withendpoint_discovery = falseand the password which is only right for single-node clusters. For multi-node clusters you’d need to copy the
trueas has been done in the Multi-Node example above.