<!-- Source: https://docs.squirro.com/en/latest/getting/install/linux.html -->
# Installing Squirro on Linux

[Deprecated](../release/squirro-release-process.md#getting-squirro-process)

> **Note for administrators**
>
> End of support for installing Squirro on Linux manually. This page remains available for reference only and is no longer maintained. Use Ansible for all installations and upgrades. See the [Install and Manage Squirro with Ansible](ansible/index.md#ansible) page.

## Architecture Overview

A Squirro cluster can contain anywhere from just one to dozens of servers. Adding additional servers to a Squirro cluster always serves two different needs: performance and capacity. As these needs do not necessarily scale in the same way, Squirro differentiates two different types of servers:

- **Storage Nodes:** stores the data that is indexed into Squirro. Add more storage nodes to scale capacity.
- **Cluster Nodes:** answers requests by users and handles the processing of items that come in. Add more cluster nodes to improve performance.

Storage and cluster nodes can be installed on the same server, and for a single-node setup that is the recommended approach. But if you intend to scale to more than just one server, it is recommended to only install one of the two node types on a server. That makes better use of the available performance and allows for more flexible scaling.

While this document primarily focuses on MySQL as the database backend, Squirro also supports PostgreSQL. For PostgreSQL setup instructions, see the [Configure PostgreSQL as the Database Backend](postgresql.md#install-postgresql) page.

## Prerequisites

This page assumes that a few prerequisites have been met. This includes a system with internet connectivity and some relevant ports opened in the firewall.

If some of these conditions can not be met, visit the [Squirro Support website](https://go.squirro.com/support) and submit a technical support request.

### Linux Setup

The installation of the base system is not covered on this page. A fully functional Red Hat Enterprise Linux (RHEL) or Rocky Linux installation is assumed to be set up. The supported versions of these Linux distributions are documented on the [System Requirements](system-requirements.md#install-system-requirements) page.

### Networking

The Linux system needs internet access to download the Squirro packages. Additionally, the system should be accessible from your internal network, so that the Squirro interface can be accessed by administrators and users.

Information on how to set up networking is provided by Red Hat in their [Official Networking Guide](https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Deployment_Guide/ch-Network_Interfaces.html).

### Firewall

Incoming traffic to Squirro servers must be open on several TCP ports. For the complete list of required ports, see the [System Requirements](system-requirements.md#install-system-requirements) page.

### Users

Squirro provides packages that set up all the Linux users required on a system. Those packages are used in the instructions below.

However, in some environments, users must not be created by packages. For these cases, users have to be manually created up-front. See [Linux Users for Squirro](#install-linux-users) for a detailed list of users that need to be set up.

## YUM Repositories

Squirro packages are provided with a YUM repository. YUM is a utility, provided with the RHEL Linux system, that downloads and installs packages from central repositories. To configure a new repository, it needs to be set up using a file in the folder `/etc/yum.repos.d.`

### Getting the Right Version of Squirro

For production deployments, Squirro recommends using the latest LTS release. Use the current LTS version identifier in the `baseurl` of the YUM repository configuration to pin to that release family. That ensures a `yum update` does not automatically move the server to a newer major release. To find the current LTS version, see the [Release Notes](../release/index.md#getting-release-notes) page.

Moving to a newer LTS family always requires a deliberate change to the `baseurl`. Review the upgrade instructions and breaking changes before doing so, as each major release may introduce configuration changes, dependency upgrades, or migration steps that require preparation. For details, see the [Upgrading Squirro](upgrade.md#install-upgrade) and [Upgrading Configuration Conflicts](upgrade-conflicts.md#install-upgrade-conflicts) pages.

### Squirro

Create the file `/etc/yum.repos.d/squirro.repo`. Squirro provides the exact content of the file when delivering your license. To obtain it, visit the [Squirro Support website](https://go.squirro.com/support) and submit a technical support request.

Use the following example, but note that the username and password are not filled in:

**RHEL 8**

/etc/yum.repos.d/squirro.repo

```text
[squirro-stable]
name=Squirro - Red Hat Enterprise Linux 8 - Stable
baseurl=https://<user name>:<password>@mirror.squirro.net/el/8/stable/$basearch/<specific version or 'latest' or '3.2-lts'>/
enabled=1
gpgkey=https://mirror.squirro.net/el/RPM-GPG-KEY-squirro-stable
gpgcheck=1
sslverify=1
metadata_expire=300
```

## Storage Node Installation

Installation of the storage node happens with two separate packages. The first package installs the required Linux users and the second installs the services. Furthermore, we also need to explicitly install a few dependencies. Use the following commands for the installation:

### Java

Squirro requires Java 21 on every node. For details, see the [System Requirements](system-requirements.md#install-system-requirements) page.

```text
sudo su
yum install java-21-openjdk
```

### Squirro Storage Node

```text
sudo su
yum install squirro-storage-node-users
yum install elasticsearch
yum install squirro-storage-node
```

### Network Connectivity

If you are setting up a dedicated storage node, instead of mixing cluster and storage node on the same server, then you need to change the Elasticsearch configuration so it listens on a network IP address.

To do this, edit `/etc/elasticsearch/elasticsearch.yml` to add the server’s IP address to the `network.host` setting.

When doing this, you also need to declare the `discovery.seed_hosts`. To set up a cluster, see [Cluster Expansion](cluster-expansion.md#install-cluster-expansion) - the following value will work only for the single-node cluster case.

/etc/elasticsearch/elasticsearch.yml

```text
network.host: <storage node IP>,127.0.0.1
discovery.seed_hosts: ["127.0.0.1"]
```

## Cluster Node Installation

Installation of the Squirro cluster node happens with two separate packages. The first package installs the required Linux users and the second installs the services. Furthermore, we also need to explicitly install a few dependencies. Use the following commands for the installation:

### Java

Squirro requires Java 21 on every node. For details, see the [System Requirements](system-requirements.md#install-system-requirements) page.

```text
yum install java-21-openjdk
```

### Running MySQL and Redis Remotely

You can choose to run MySql server and Redis servers remotely, i.e. not residing on the Squirro Cluster Node if you go through the trouble of setting up MySql and Redis Server installations with a specific configuration.

To set up Squirro with remote MySql server and Redis server “Backends”, please create a readable file `/etc/squirro/backends.ini` with the content:

```text
is_mysql_server_remote = true
is_redis_server_remote = true
```

### Squirro Cluster Node

Installation of the Squirro cluster node happens with two separate packages. The first package installs the required Linux users and the second installs the services. Use the following commands for the installation:

```text
yum install squirro-cluster-node-users
yum install squirro-cluster-node
```

If the Storage and Cluster node are not the same physical machine, then you need now to adjust the file `/etc/nginx/conf.d/upstream-elastic.inc` to point to the IP or Hostnames of the storage node(s).

Examples:

**Single Server Both Roles**

```text
upstream elastic {
   server 127.0.0.1:9200;
   keepalive 32;
}
```

**Dedicated Storage Node**

```text
upstream elastic {
server 192.168.0.20:9200;
keepalive 32;
}
```

**Multiple Storage Nodes**

```text
upstream elastic {
server 192.168.0.20:9200;
server 192.168.0.21:9200;
server 192.168.0.22:9200;
keepalive 32;
}
```

If changes have been made to this file, reload the Nginx configuration as follows:

```text
service nginx reload
```

_Caution_: If you have chosen to rely on remote MySql and/or Redis server installations, follow the steps in the [Set Up on Linux with Remote MySql and Redis Servers](#install-linux-mysql-redis) section later on this page.

### Configuring the PDF Viewer License Key

On Squirro 3.15.4 and later, the built-in PDF viewer requires a license key. Without a valid key, PDF documents render with a watermark. Add the key to `/etc/squirro/frontend.ini` before starting the Squirro services. For instructions on obtaining and applying the key, see the [PDF Viewer](../../technical/admin/configuration/pdf-viewer.md#pdf-viewer-introduction) page.

### Starting Squirro Services

Finally, start the Squirro Services using the `squirro_start` command.

**RHEL**

```text
source /etc/profile.d/squirro-aliases.sh
squirro_restart
```

### Multi-node Cluster Setup

Running Squirro across multiple nodes requires additional steps to add and configure the cluster and storage nodes. For the full procedure, see the [Cluster Expansion](cluster-expansion.md#install-cluster-expansion) page. For high-availability and multi-node deployments, visit the [Squirro Support website](https://go.squirro.com/support) and submit a technical support request.

## Setting up the License Key

Squirro packages are provided with a YUM repository. To configure a new repository, follow the steps below:

1. Set up using a file in the folder `/etc/yum.repos.d`.
2. Create or edit /etc/yum.repos.d/squirro.repo :

   ```ini
   [squirro-stable]
   name=Squirro - EL-$releasever - Stable
   baseurl=https://FILL_IN_USERNAME_AND:PASSWORD@mirror.squirro.net/el/$releasever/stable/$basearch/latest
   enabled=1
   gpgkey=https://mirror.squirro.net/el/RPM-GPG-KEY-squirro-stable
   gpgcheck=1
   sslverify=1
   metadata_expire=300
   ```
3. Insert the repository credentials you got for accessing the Squirro YUM repository in this file. Replace FILL_IN_USERNAME_AND by the username and PASSWORD by your password. Also enable the repo by changing the value for “enabled”.

## Set Up on Linux with Remote MySql and Redis Servers

In some cases, e.g. when deploying Squirro to Amazon AWS or MS Azure, you may elect to rely on a remote MySql Server installation (e.g. RDS in case of AWS) and/or remote Redis Servers (e.g. RedisLabs). This page describes how you would configure such remote servers for use by Squirro and how to configure Squirro Cluster Nodes accordingly.

### Preparation

Before you install Squirro RPMs, ensure that you put in place the file `/etc/squirro/backends.ini` with the following content:

To set up Squirro with remote MySql server and Redis server “Backends”, please create a readable file `/etc/squirro/backends.ini`:

```python
mkdir -p /etc/squirro
touch /etc/squirro/backends.ini
```

And add the following content depending on whether you want to run MySql server or Redis servers remotely or both. The example below show the content for both MySql and Redis servers to be configured remotely:

`/etc/squirro/backends.ini`

```python
# Rely on a remote MySql Server installation
is_mysql_server_remote = true

# Rely on remote Redis Servers
is_redis_server_remote = true
```

### MySql Server Setup

On the MySql Server front there are server-level configurations, database-level configurations, and Squirro Cluster Node-level configurationsServer-level configuration

#### MySql Server-level Configuration

Configure MySql on User level scope, make sure your ~/.my.cnf file content looks like this:

```python
[client]
host="remote mysql server"
port=3306
user="user for the remote mysql server"
password="password for the remote mysql server"
ssl_ca="location of your SSL certificate"
```

> **Note**
>
> The default MariaDB/MySQL port is 3306. If your database server uses a non-default port, update the `port` parameter accordingly. For example, if your MariaDB server runs on port 3307, set `port=3307`.

If there is a requirement, follow the instructions in the [securing Elasticsearch](../../technical/admin/configuration/secure-config-guide.md#admin-securing-elasticsearch) section.

Following configuration are required by Squirro on MySql server level.

```python
[mysqld]
character_set_server=utf8
collation_server=utf8_unicode_ci
innodb_flush_log_at_trx_commit=1
```

Make sure you configure the MySql client on all Squirro Cluster Nodes to point to the server installation. Also note the name of the MySql administrator account - often called root.

#### MySql Database and User Creation

Run the following on a Squirro Cluster Node on which `/etc/squirro/backends.ini` has been set up and `squirro-cluster-node` has been installed.

```bash
MYSQL_ADMIN=root  # or the name of the MySql Administrator account able to create users and databases
DATABASE_PASSWORD="password of your choosing"

squirro_mysql_databases=`cd /etc/squirro/; grep "^db *=" *.ini    | sed -e "s/\..*$//"`
for mysql_db in $squirro_mysql_databases; do
    . /opt/squirro/setup/ensure_mysql_user_and_database_exist.sh $MYSQL_ADMIN $mysql_db $DATABASE_PASSWORD
done
```

### Squirro Cluster Node Configuration

Before starting Squirro services, the configuration files under `/etc/squirro/` need to be made to point to the remote installation:

```python
MYSQL_SERVER_ADDRESS="address or name of MySql installation"
sed -e "s|squirro/4u|$DATABASE_PASSWORD|" -e "s|localhost|$MYSQL_SERVER_ADDRESS|" -i /etc/squirro/*.ini
```

Note that the script above depends on the same DATABASE_PASSWORD variable from the “MySql Database and User Creation” section above.

### Redis Server Setup

#### Redis Server-level Configuration

Squirro writes into about a dozen Redis databases.

Ideally, your Redis Server setup would have multiple Redis databases, although it is possible, somewhat experimentally, to use a single Redis Server.

Squirro recommends running at least two Redis Server instances, one for key/value storage and one for caching. If SSL is required, follow the instructions on the [Securing Redis Instance over SSL](../../technical/admin/configuration/securing-redis.md#admin-securing-redis) page.

Each Redis instance also requires a specific eviction policy. The default `maxmemory-policy` value of `volatile-lru` is not suitable for Squirro and must be changed:

- **Storage instance**

  Set `maxmemory-policy noeviction` so that queued tasks and stored key/value data are never dropped when the instance reaches its memory limit.
- **Cache instance**

  Set `maxmemory-policy allkeys-lru` so that the least recently used keys are evicted first when the instance reaches its memory limit.

Set the policy in the configuration file of each Redis instance, for example `/etc/redis/redis.conf` for the storage instance and `/etc/redis/cache.conf` for the cache instance:

```ini
maxmemory-policy noeviction
```

### Squirro Cluster Node Configuration

You would point the Squirro services to the two Redis Server installation like so:

```bash
REDIS_STORAGE_HOST="Redis Server Storage host"
REDIS_STORAGE_PORT="Redis Server Storage port number"
REDIS_STORAGE_PASSWORD="Redis Server Storage password"
REDIS_CACHE_HOST="Redis Server Cache host"
REDIS_CACHE_PORT="Redis Server Cache port number"
REDIS_CACHE_PASSWORD="Redis Server Cache password"

redis_storage_dbs=`grep "^\[redis.*]" /etc/squirro/*.ini    | grep -v "_cache.*]"    | grep -v "redis_key_value"    | sed -e "s/^.*\[//" -e "s/] *$//"    | sort -u`
redis_cache_dbs=`grep "^\[redis.*_cache.*]" /etc/squirro/*.ini    | grep -v "redis_key_value"    | sed -e "s/^.*\[//" -e "s/] *$//"    | sort -u`
redis_key_value_dbs=`grep "^\[redis_key_value_store.*]" /etc/squirro/*.ini    | sed -e "s/^.*\[//" -e "s/] *$//"    | sort -u`
redis_key_value_cache_dbs=`grep "^\[redis_key_value_cache.*]" /etc/squirro/*.ini    | sed -e "s/^.*\[//" -e "s/] *$//"    | sort -u`

# remove old redis passwords:
sed -e "/^password *=/d" -i /etc/squirro/*.ini
sed -e "/^redis_password *=/d" -i /etc/squirro/*.ini

for storage_db in $redis_storage_dbs; do
    sed "s|^\(\[${storage_db}] *\)$|\1\nhost = $REDIS_STORAGE_HOST\nport = $REDIS_STORAGE_PORT\npassword = $REDIS_STORAGE_PASSWORD|" -i /etc/squirro/*.ini
done

for cache_db in $redis_cache_dbs; do
    sed "s|^\(\[${cache_db}] *\)$|\1\nhost = $REDIS_CACHE_HOST\nport = $REDIS_CACHE_PORT\npassword = $REDIS_CACHE_PASSWORD|" -i /etc/squirro/*.ini
done

for storage_db in $redis_key_value_dbs; do
    sed "s|^\(\[${storage_db}] *\)$|\1\nredis_host = $REDIS_STORAGE_HOST\nredis_port = $REDIS_STORAGE_PORT\nredis_password = $REDIS_STORAGE_PASSWORD|" -i /etc/squirro/*.ini
done

for cache_db in $redis_key_value_cache_dbs; do
    sed "s|^\(\[${cache_db}] *\)$|\1\nredis_host = $REDIS_CACHE_HOST\nredis_port = $REDIS_CACHE_PORT\nredis_password = $REDIS_CACHE_PASSWORD|" -i /etc/squirro/*.ini
done

if grep -Fxq "[queues_local]" /etc/squirro/common.ini
then
    sed "s|^\(\[queues_local] *\)$|\1\nredis_host = $REDIS_STORAGE_HOST\nredis_port = $REDIS_STORAGE_PORT\nredis_password = $REDIS_STORAGE_PASSWORD|" -i /etc/squirro/common.ini
else
    cat >>/etc/squirro/common.ini <<FOO

[queues_local]
redis_host = $REDIS_STORAGE_HOST
redis_port = $REDIS_STORAGE_PORT
redis_password = $REDIS_STORAGE_PASSWORD
FOO
fi
```

### Configuring SSL for Remote Redis Servers

Managed Redis services, such as Amazon ElastiCache or Azure Cache for Redis, typically require SSL/TLS connections. In that case, add the following settings to the `[redis]` section of `/etc/squirro/common.ini`:

```ini
[redis]
ssl = True
ssl_ca_certs = /path/to/ca.crt
ssl_verify = True
```

Settings placed in the `[redis]` section are inherited by the other Redis sections that do not define their own values, so in most cases you do not need to repeat them in each Redis block. For the complete SSL resolution rules, the full list of available settings, and how to configure certificates on the Redis server side, see the [Securing Redis Instance over SSL](../../technical/admin/configuration/securing-redis.md#admin-securing-redis) page.

### Return to the Linux Installation Steps

Congratulations. You are now ready to continue with the Squirro Cluster Node installation steps.

## Linux Users for Squirro

The following table represents a list of user accounts Squirro creates at installation time.

|  |  |  |  |
| --- | --- | --- | --- |
| **User Name** | **Primary Group** | **Description** | **Related Services** |
| elasticsearch | elasticsearch | Main data storage for indexed data. | elasticsearch |
| mysql | mysql | MySQL database used for metadata storage. To investigate this metadata, MySQL can be accessed directly. | mysql |
| postgres | postgres | PostgreSQL database used for metadata storage when PostgreSQL is selected as the database backend. See [Configure PostgreSQL as the Database Backend](postgresql.md#install-postgresql) for more information. | postgresql |
| nginx | nginx | Manages nginx web server. | nginx |
| redis | redis | NoSQL database used for caching. | redis-server, redis-server-cache |
| sqconfig | squirro | Manages application configuration. | sqconfigurationd |
| sqcontnt | squirro | Responsible for extracting relevant content from web pages. See [Noise Removal](../../technical/data-processing/pipeline-steps/enrich/noise-removal.md#pipeline-steps-noise-removal). | sqcontentd |
| sqdatasource | squirro | Manages dataloading process. | sqdatasourced |
| sqeauth | squirro | Responsible for managing authentication. | squserd |
| sqemails | squirro | Responsible for sending out emails on behalf of Squirro | sqemailsenderd |
| sqfilter | squirro | Handles search tagging rules.See [Search Tagging and Alerting](../../technical/data-processing/pipeline-steps/index/search-tagging.md#pipeline-steps-search-tagging). | sqfilteringrd |
| sqfront | squirro | Manages frontend servcice, implements main web interface of Squirro. | sqfrontendd |
| sqingest | squirro | Manages process for enriching and indexing data. | sqingesterd |
| sqmailer | squirro | Used for preparing the email digest | sqdigestmailerd |
| sqplumbr | squirro | Responsible for executing pipelets as part of the pipeline. See [Pipelets](../../technical/pipelets/index.md#pipelets) for more information. | sqplumberd |
| sqprovid | squirro | Manages endpoints for the providers that import data into the system. | sqproviderd |
| sqptopic | squirro | Manages the Squirro API. | sqtopicd |
| sqsched | squirro | Responsible for executing scheduled jobs that the Squirro application needs to run. | sqschedulerd |
| sqthumb | squirro | Calculates thumbnails for item images that are displayed in Squirro. | sqthumblerd |
| sqtproxy | squirro | Manages the web service access to some project metadata. | sqtopicproxyd |
| squproxy | squirro | Responsbile for manages the web service which gives access to some user data. | squserproxyd |
| squser | squirro | Endpoint for user management and authentication. See [Authentication](../../technical/api/authentication.md#api-authentication). | squserd |
| sqwebsht | squirro | Analyzes web stories and identifies the most relevant image to use for visualization. | sqwebshotd |
| tika | squirro | Responsible for extracting text from a variety of binary-based documents. | tika |

## Setup on Linux (Offline)

This section covers installing Squirro on a Linux system without internet access.

### Prerequisites

For an offline installation, Squirro will provide you with a `tar.gz` file which contains all the required RPM files. Make that file available on the server, for example by copying it over using scp.

### YUM Repositories

#### Extract Repository

The `tar.gz` file above contains a YUM repository that can be used offline. To use this on the server, extract it:

```bash
sudo su
mkdir -p /opt/squirro
cd /opt/squirro
tar -xvf /tmp/squirro-xxx.tar.gz
```

This creates the folder `/opt/squirro/repo` and populates it with all the RPM files.

#### Squirro

Create the file `/etc/yum.repos.d/squirro.repo`.

```ini
[squirro]
name=Squirro
baseurl=file:///opt/squirro/repo/
enabled=1
metadata_expire=300
gpgcheck=0
```

### Storage Node Installation

Follow the steps in [Storage Node Installation](#install-linux-storage) to complete the storage node installation.

### Cluster Node Installation

Follow the steps in [Cluster Node Installation](#install-linux-nodes) to complete the cluster node installation.

## Disabling User Sign-Up After Installation

You will create your first user account using the Sign Up button that appears with the first load of a Squirro installation.

The first user created will be a server-level Administrator.

Reference: For further details on server-level roles, see [Server-Level Permissions](../../technical/ui/permissions.md#ui-permissions-server).

After creating your first Administrator account, Squirro recommends disabling the `Sign Up` feature to prevent unauthorized account creation.

To disable the feature, follow the steps below:

1. Log in to your Squirro installation as a server administrator.
2. Navigate to the Server space.
3. Click Configuration in the left menu.
4. Search for _frontend.userapp_; the results list should include `frontend.userapp.signup`.
5. Hover over `frontend.userapp.signup` and click Edit.
6. Uncheck the box to disable the option, then click the checkmark button to save, as shown in the screenclip below:

[![Squirro Disable Sign-Up Configuration Option](https://s3.amazonaws.com/download.squirro.net/docs/technical/ui/disable-signup.png)](https://s3.amazonaws.com/download.squirro.net/docs/technical/ui/disable-signup.png)

## Next Steps

After the installation completes, finish securing and configuring the instance. For the full list of steps, see the [Post-Installation Checklist](post-installation-checklist.md#install-post-installation-checklist) page.

## Troubleshooting & Known Issues

There is an issue with MariaDB Version 10.6 on RedHat 8.5: Squirro was not able to configure the required SQL tables on MariaDB Version 10.6. This was resolved by downgrading to MariaDB 10.3.