Activity Tracking

This page provides an overview of Activity Tracking, which is used to track data about user interactions on the platform.

Note

Activity Tracking is visualized as part of Squirro Monitoring. If you are looking to view project user activity, Squirro Monitoring is the recommended place to start.

Overview

You can configure Squirro to collect data on user interactions with Squirro.

The collected activity data provides insights on the overall usage of the system. For example, what dashboards were visited or what queries a user submitted.

Squirro uses the recorded queries to improve the overall user experience by suggesting popular queries in typeahead or recommending similar queries when a user types into the search bar.

Configuration

Activity tracking is enabled by default on every new Squriro instance.

If you would rather not collect activity logs, go to Server→Configuration and uncheck the (frontend.userapp.log-activity) option.

The activity logs are by default stored under /var/log/squirro/frontend/$hostname/activity.$year-$month-$day.jsonl.

The path can be changed in the common.ini file by adjusting the path config in the activity section:

[activity]
path = /var/log/squirro/frontend/

Activity Tracking Project

Automatic Creation

Self-service instances are shipped with a preconfigured monitoring project space. On new deployments, Squirro creates the monitoring project named Squirro Monitoring upon first admin-user sign up (for each tenant).

You can configure the behaviour of the automatic monitoring project creation in the Server space under Configuration by setting the value of topic.monitoring.default-monitoring-project to true or false (default is true)

Monitoring spaces come with the following:

• Preconfigured sources and pipelines to load activity logs and compute activity insights.

• Dashboard to display user activity within the project.

Manual Creation

On existing deployments, you can manually create a monitoring project by using the Squirro Monitoring project template, as shown in the example screenshot below:

For manually created monitoring projects, the Squirro Activity Log data loader plugin uses a custom index with the name squirro_v9_${project_id}. The index is required to set up the popular queries suggestions.

Activity Tracking Components

The Squirro Monitoring uses the following components to collect user activity data and extract knowledge about user behavior:

• Squirro Activity Log: A Data Loader plugin that continuously loads activity data from the activity logs.

• Activity Tracking: A pipelet that enriches activity data with additional user information.

• Squirro Activity Insights: A Data Loader plugin that generates enriched insights about user behavior.

Squirro Activity Log Data Loader Plugin

The Squirro Activity Log Data Loader plugin loads the activity logs saved in the filesystem and inserts them to the Elasticsearch index.

By default, the Data Loader plugin reads the path with activity files from the common.ini config.

If you would prefer to provide a custom path where your activity files are stored, you can specify the Activity path option in the advanced options.

Running Data Loader on Multi-Node Instances and Storing Files in the EFS

The Squirro Activity Log Data Loader plugin works both on multi-node instances and with activity files placed in the EFS. It takes care of loading proper data by resolving the hostname folder where the activity files are stored.

The plugin gathers the activity files from the root level of the activity path and then checks for files inside the folder with the corresponding hostname. If it finds a folder with the same hostname as the hostname that it is running on, it also searches for files inside this folder.

Example

Assuming the Data Loader runs on the instance with the hostname: hostname-1, it will gather the following files:

• /var/log/squirro/frontend/activity.2022-01-01.jsonl

• /var/log/squirro/frontend/hostname-1/activity.2022-01-02.jsonl

But won’t gather the file below:

• /var/log/squirro/frontend/hostname-2/activity.2022-01-03.jsonl

Activity Tracking Pipelet

The Activity Tracking pipelet is part of the Squirro Activity Log Data Loader plugin’s ingestion pipeline.

It enriches loading activity logs with additional user metadata like the user’s name, email, role, and groups that the user belongs to.

If you want to enrich activity logs with custom user-related information, you can configure the pipelet by going to the Pipeline Editor and clicking on the Activity Tracking step.

Squirro Activity Insights Data Loader Plugin

When the Squirro Activity Log Data Loader plugin provides information on user actions, the Squirro Activity Insights Data Loader plugin combines this information to give a bigger picture overview of user behavior.

The Data Loader plugin collects activity data from Elasticsearch and generates insights called sessions.

These sessions allow Squirro to understand how the user behaves on the platform.

For example, it can answer a question like, “How many times does the user type a query but then cannot find a relevant document and leave?”.

The Data Loader computes the following three types of sessions:

• Query sessions

• Dashboard sessions

• Browsing sessions

Query Session

A query session is defined as the activities that are logged between a user’s searches.

It answers the question, “What does a user do after performing a search?”

Dashboard Session

A dashboard session is defined as the activities that are logged on a particular dashboard before the user switches dashboards.

It answers the question: “What does a user do on a particular dashboard?”.

Browsing Session

A browsing session is defined as the activities that are logged between a user’s activity sessions (see below).

It answers the question, “How does a user interact with the Squirro product in general?”

Activity Session

An activity session is a top-level session that enables Squirro to detect when the user entered and left the platform.

It helps produce more detailed insights by creating boundaries between user visits.

The activity session ID is stored in the Redis database and attached to each activity event.

The session starts at the first visit to the platform and ends after an explicit logout or after some period of inactivity.

Each user’s activity extends the session time.

After no activity within the specified time, it is assumed to be the end of the session and further activities will generate a new one.

To configure the session expiration time, you can use the configuration service shown below: