Activity Tracking
Contents
Activity Tracking#
This page provides an overview of Activity Tracking, which is used to track data about user interactions on the platform.
Note
Activity Tracking is visualized as part of Squirro Monitoring. If you are looking to view project user activity, Squirro Monitoring is the recommended place to start.
Overview#
You can configure Squirro to collect data on user interactions with Squirro.
The collected activity data provides insights on the overall usage of the system. For example, what dashboards were visited or what queries a user submitted.
Squirro uses the recorded queries to improve the overall user experience by suggesting popular queries in typeahead or recommending similar queries when a user types into the search bar.
Configuration#
Enable activity tracking in the Server Configuration (frontend.userapp.log-activity).
The activity logs are by default stored under /var/log/squirro/frontend/$hostname/activity.$year-$month-$day.jsonl.
The path can be changed in the common.ini file by adjusting the path config in the activity section:
[activity]
path = /var/log/squirro/frontend/
Activity Tracking Project#
The Squirro Monitoring uses the Squirro Activity Log data loader plugin with a pre-configured Activity Log Enrichment pipeline to continuously load the activity data from the activity logs.
Automatic Creation#
Self-service instances are shipped with a pre-configured monitoring project space. On new deployments, Squirro creates the monitoring project named Squirro Monitoring upon first admin-user sign up (for each tenant).
You can configure the behaviour of the automatic monitoring project creation in the Server space under Configuration by setting the value of topic.monitoring.default-monitoring-project to true or false (default is true)
Monitoring spaces comes with the following:
A pre-configured
Squirro Activity Logdata sourceA custom index for the activity data with the name
squirro_v9_activity_tracking_${tenant}. The index is required to set up the popular queries suggestions.
Current limitations:
- The monitoring project has no pre-configured dashboards. For a template, see the delivery repository.
Manual Creation#
On existing deployments, you can also manually create a monitoring project:
Create a new project
Add the Squirro Activity Log data loader plugin as data source
Click the SAVE button to set up the source as 1-click connector.
By setting up the source as a 1-click connector, all necessary facets are automatically mapped correctly. If you need to map the facets manually, click the NEXT button (step 3 above). The relevant facets are listed in the facets.json below.
{
"source_type": {
"name": "source_type",
"display_name": "Source Type",
"default_value": "Activity",
"visible": true,
"searchable": true,
"typeahead": true,
"analyzed": true
},
"hostname": {
"name": "hostname",
"display_name": "Hostname",
"analyzed": false,
"group_name": "Activity"
},
"session_id": {
"name": "session_id",
"display_name": "Session",
"analyzed": false,
"group_name": "Activity"
},
"pageview_id": {
"name": "pageview_id",
"display_name": "Pageview",
"analyzed": false,
"group_name": "Activity"
},
"source": {
"name": "event_source",
"display_name": "Source",
"analyzed": false,
"group_name": "Activity"
},
"action": {
"name": "action",
"display_name": "Action",
"analyzed": false,
"group_name": "Activity"
},
"user_id": {
"name": "user_id",
"display_name": "User ID",
"analyzed": false,
"group_name": "User"
},
"query.query": {
"name": "query",
"display_name": "Query",
"group_name": "Activity",
"searchable": true
},
"query_context.parse.user_terms": {
"name": "query_user_terms",
"display_name": "Query User Terms",
"group_name": "Activity",
"analyzed": false
},
"query_context.parse.facet_filters": {
"name": "query_user_filters",
"display_name": "Query User Filters",
"group_name": "Activity",
"analyzed": false
},
"query_context.parse.language": {
"name": "query_language",
"display_name": "Query Language",
"group_name": "Activity",
"analyzed": false
},
"query_context.parse.type": {
"name": "query_type",
"display_name": "Query Type",
"group_name": "Activity",
"analyzed": false
},
"item.id": {
"name": "item_id",
"display_name": "Item ID",
"analyzed": false,
"group_name": "Activity"
},
"item.link": {
"name": "item_link",
"display_name": "Item Link",
"analyzed": false,
"group_name": "Activity"
},
"drop_to": {
"name": "drop_to",
"display_name": "Item Dropped To",
"analyzed": false,
"group_name": "Activity"
},
"widget.type": {
"name": "widget_type",
"display_name": "Widget Type",
"analyzed": false,
"group_name": "Activity"
},
"dashboard.title": {
"name": "dashboard_title",
"display_name": "Dashboard",
"analyzed": false,
"group_name": "Activity"
},
"layer_title": {
"name": "layer_title",
"display_name": "Layer",
"analyzed": false,
"group_name": "Activity"
},
"dashboard.id": {
"name": "dashboard_id",
"display_name": "Dashboard ID",
"analyzed": false,
"group_name": "Activity"
},
"project.title": {
"name": "project_title",
"display_name": "Project",
"analyzed": false,
"group_name": "Activity"
},
"project.id": {
"name": "project_id",
"display_name": "Project ID",
"analyzed": false,
"group_name": "Activity"
},
"edit": {
"name": "edit",
"display_name": "Edit",
"analyzed": false,
"group_name": "Activity"
},
"microseconds": {
"name": "microseconds",
"display_name": "Microseconds",
"data_type": "int",
"group_name": "Activity"
},
"result.start": {
"name": "result_start",
"display_name": "Result List Offset",
"data_type": "int",
"group_name": "Activity"
},
"result.total": {
"name": "result_total",
"display_name": "Results",
"data_type": "int",
"group_name": "Activity"
},
"type": {
"name": "type",
"display_name": "Type",
"group_name": "Activity"
}
}
For manually created monitoring projects, the Squirro Activity Log data loader plugin uses a custom index with name squirro_v9_${project_id}. The index is required to set up the popular queries suggestions.
Activity Dataloader Plugin#
By default, the data loader plugin reads the path with activity files from the common.ini config.
If you would like to provide a custom path where your activity files are stored, you can specify the Activity path option in the advanced options.
Running Data Loader on Multi-Node Instances and Storing Files in the EFS#
The Squirro Activity Log data loader plugin works both on multi-node instances and with activity files placed in the EFS. It takes care of loading proper data by resolving the hostname folder where the activity files are stored.
The plugin gathers the activity files from the root level of the activity path and then checks for files inside the folder with the corresponding hostname. If it finds a folder with the same hostname as the hostname that it is running on, it also searches for files inside this folder.
So assuming the data loader runs on the instance with the hostname: hostname-1, it will gather the following files:
/var/log/squirro/frontend/activity.2022-01-01.jsonl/var/log/squirro/frontend/hostname-1/activity.2022-01-02.jsonl
But won’t gather the file below:
/var/log/squirro/frontend/hostname-2/activity.2022-01-03.jsonl
Session Tracking#
To get more insights of how users interact and use the Squirro product, a session ID is attached to each activity entry. Session is stored in the Redis database and has an expiration time that is used for defining when the session starts and ends. Each user’s activity extends the session time. After no user’s activity within the specified time, it is assumed to be the end of the session. Further activities will generate a new session.
To configure the session expiration time, you can use the configuration service:
