Upgrading Squirro#

The upgrade instructions can vary from release to release and on the version of Red Hat / CentOS.

Please consult the release notes for any given Squirro release for details on which instructions to follow.

For upgrade troubleshooting, see Upgrading Configuration Conflicts.

For upgrade instructions for earlier versions, see Upgrading Earlier Versions.

Upgrading Using Ansible#

If you have manually installed Squirro via Linux and want to use Ansible to upgrade, you will need to carefully test the upgrade process on a test system before upgrading your production system. It is recommended that you speak with Squirro Support for assistance or upgrade via Linux by referring to the section later on this page.

If you have installed Squirro using Ansible, you have three options to upgrade:

  • Upgrade manually by pinning the desired release channel and version within your playbook.

  • Upgrade automatically to the latest patch of a specific LTS by pinning that LTS version within your playbook.

  • Upgrade automatically to the latest version. (This is the default Squirro configuration when installing Ansible.)

Caution

During an upgrade, the Squiro services will be restarted. This will cause a short downtime. As a result, it’s recommended that upgrades to production systems be run inside a maintenance window and be tested in a lower environment.

Manual Upgrade#

To pin the release channel and version of your Squirro installation, edit the playbook-quickstart.yml file and set the following variables:

squirro_channel:
squirro_version:

For channel, you will most likely want to enter stable to stay on the latest stable version.

For version, you can enter a specific version number, such as 3.5.3.

Tip: When you are upgrading manually, it is a good idea to check the mirror for the latest version of Ansible before upgrading.

Pinning to an LTS Release#

You can also select a LTS version to stay upgraded, such as 3.6-lts. This will upgrade to the latest LTS version including patch releases, but will not upgrade to the next LTS version when it is released.

This will not upgrade to new biweekly releases of the next LTS version. For example, if you pin to 3.6-lts, you will not upgrade to 3.7.0 when it is released.

However, you will need to manually upgrade to the next LTS version when it is released.

Reference: Learn more about the Squirro Release Process.

Automatic Upgrade#

The default Ansible configuration based on the Installing Squirro Using Ansible is to automatically upgrade to the latest version, with squirro_version set to latest.

To verify your configuration is set to upgrade to the latest biweekly release, check the playbook-quickstart.yml file and ensure the following variables are set:

squirro_channel: stable
squirro_version: latest

Automatic upgrades to biweekly releases are not recommended for production systems. It is recommended that you pin to a specific version or LTS release and upgrade manually.

Offline Upgrades#

You can perform offline upgrades using Ansible in a similar way to offline installations, by downloading the appropriate tar.gz file from the Squirro mirror site, as described in Offline Installation.

Manual Upgrades for Squirro 3.5.3 and Later#

These upgrade instructions explain how to manually update an existing Linux-based Squirro installation (i.e. without using Ansible) to the current version.

Important

Squirro recommends using Ansible to install and upgrade Squirro. The instructions found within this section should be understood as a secondary option. If you encounter any issues manually upgrading, contact Squirro Support.

Caution

Only follow these instructions if a specific Squirro release notes document references these as the correct upgrade instructions.

Reference: If you are looking to upgrade a version earlier than 3.5.3, see Upgrading Earlier Versions.

Before Upgrading#

Maintenance Window#

A Squirro upgrade requires downtime. This is typically quite short, but will still affect user experience for a moment. As a result, upgrades to production systems should only be run in a maintenance window.

Storage vs. Cluster Nodes#

Squirro distinguishes between storage and cluster nodes. These can be installed on the same server or split up. Both can also be horizontally scaled and have multiple instances. For full details see How Squirro Scales.

Independent of the setup, in the instructions below always execute all the storage node updates first, and only then do the cluster node updates. This applies independent of whether cluster and storage node are on the same system, whether they are split into one server each, or if they have multiple instances each.

Offline Repository#

If you have used an offline installation of Linux, or have an internal mirror of the Squirro yum repository, make sure it is updated prior to starting the Squirro update.

Upgrading#

Storage Nodes#

Warning

Starting from Release 3.7.4, Squirro storage nodes use Elasticsearch 8.x. If you are upgrading Squirro from older versions, you will also need to upgrade Elasticsearch to 8.x. For detailed instructions on migrating Elasticsearch, see Elasticsearch Migration.

Warning

If Elasticsearch security is enabled on any version previous to the most recent LTS version of Squirro (3.7.6), your upgrade will fail. Elasticsearch security must be disabled before upgrading the storage node in those situations.

Run these instructions on every single server that has storage nodes installed before moving on to the cluster nodes.

# Clear all cached metadata
yum clean all

# Update Java JDK
yum update java-1.8.0-openjdk

# Update Elasticsearch
yum update elasticsearch

# Update storage node
yum update squirro-storage-node-users
yum update squirro-storage-node

Cluster Nodes#

Please note that user-provided custom assets which include a requirements.txt file (e.g., a custom data loader plugin) might need to re-uploaded after the upgrade if the packages they depend on (if any) do not work on Python 3.8.

# Clear all cached metadata
yum clean all

# Update Java JDK
yum update java-1.8.0-openjdk

# Update cluster node
yum update squirro-cluster-node-users
yum update squirro-cluster-node

# Cleanup
yum autoremove
mv /opt/squirro/virtualenv3 /opt/squirro/virtualenv3.old
# /opt/squirro/virtualenv3.old is not needed for Squirro platform operation

# Make sure that all the Squirro packages have been updated
yum update "squirro-*"

At this point you will have to check if any *.rpmnew files have been created in /etc/squirro and /etc/elasticsearch. See Upgrading Configuration Conflicts for details on how to handle those conflicts.

For example, the /etc/squirro/common.ini would need resolution if you are upgrading from a version that does not have the new pdfconversion service.

Finally, restart the services:

systemctl restart mariadb
systemctl reload nginx
squirro_restart

And validate if everything is coming up (this may have to be run a few times until everything stabilizes):

squirro_status

The yum autoremove command can cause the Squirro systemd services not to start during a reboot. We recommend that you issue the following commands to fix this issue and reboot the system to ensure it works. This only is an issue with upgrades from 3.4 and earlier.

systemctl enable sqconfigurationd.service
systemctl enable sqcontentd.service
systemctl enable sqdatasourced.service
systemctl enable sqdigestmailerd.service
systemctl enable sqemailsenderd.service
systemctl enable sqfilteringd.service
systemctl enable sqfrontendd.service
systemctl enable sqingesterd.service
systemctl enable sqmachinelearningd.service
systemctl enable sqpdfconversiond.service
systemctl enable sqplumberd.service
systemctl enable sqproviderd.service
systemctl enable sqrelatedstoryd.service
systemctl enable sqschedulerd.service
systemctl enable sqthumblerd.service
systemctl enable sqtopicd.service
systemctl enable sqtopicproxyd.service
systemctl enable sqtrendsd.service
systemctl enable squserd.service
systemctl enable squserproxyd.service
systemctl enable sqwebshotd.service