Squirro Monitoring#

This page describes the Squirro Monitoring space, which allows you to monitor user and data ingestion activity throughout your Squirro instance.

By default, a Squirro Monitoring project is automatically installed on new Squirro instances.

Important: Squirro Monitoring covers an entire instance, not an individual project. All projects on an instance share the same Squirro Monitoring.

Note

Squirro Monitoring was introduced as a default part of new instances as of Squirro 3.6.3 (September, 2022). Instances created before the September 2022 will need to manually add Squirro monitoring as described in Installing Squirro Monitoring.

Accessing Squirro Monitoring#

Squirro Monitoring is accessed through the Spaces menu once you’ve logged into a project.

Relevant User Profiles#

Squirro Monitoring provides valuable insights to two groups of people.

The first group includes anyone wanting to understand user activity trends within a project. Typically, this is the project creator.

The second group includes anyone wanting to understand data ingestion activity within a project. This is often the project creator, but can also include model creators and data scientists connecting data sources to the Squirro platform.

Using Squirro Monitoring#

Two dashboards are available within Squirro Monitoring:

  • Activity Logs

  • Data Ingestion Logs

Both dashboards will show all historical project data by default.

Note: You can change the active time interval by clicking All Time in the dropdown menu and selecting your desired interval.

Changing time interval within Squirro Monitoring

Activity Dashboard#

The Activity dashboard displays user activity within the project. This is where you can identify usage trends within the Squirro platform and within your project documents.

The following is an example Activity Dashboard:

Activity Dashboard within Squirro Monitoring

You can use this dashboard to identify:

  • Which dashboards project users are using most frequently

  • Dashboard usage trends over time

  • Which groups are the most active

  • Search trends within groups

  • The total number of items opened and trends in item utilization over time

If you are looking for further information about tracking user behavior within Squirro, see Activity Tracking.

Data Ingestion Logs Dashboard#

The Data Ingestion Logs dashboard displays data ingestion activity. On this dashboard, you can see if any errors occurred during the ingestion of data from the configured data sources into your Squirro instance. By default, the dashboard presents an activity chart and summary figures at top, with the most recent data ingestion logs below.

The following is an example Data Ingestion Dashboard:

Data Ingestion Dashboard within Squirro Monitoring

You can filter logs by selecting from the dropdown menus below the navigation bar as shown in the screen capture below:

Filtering the Data Ingestion Dashboard within Squirro Monitoring

By default, you can filter logs by the following parameters:

  • Project ID (project_id): Display only the data ingestion logs related to the selected project.

  • Source ID (source_id): Display only the data ingestion logs related to the selected data source.

  • Batch ID (batch_id): Display only the data ingestion logs related to the selected batch (a batch is a set of items which belong to a specific data source and project).

  • Pipeline Step Name (step_name): Display only the data ingestion logs produced by a selected pipeline step. This can be a specific pipelet, ML model, or built-in step.

  • Pipeline Step Execution Mode (step_execution_mode): There are two execution modes for the pipeline steps: batch, parallel. With this filter you can see the data ingestion logs produced by the pipeline steps of a selected execution mode.

  • Pipeline Step Exit Status (step_exit_status): There are two exit statuses: error, success. With this filter you can see the data ingestion logs produced by the pipeline steps of a selected exit status.

Note

By default, only logs of INFO level and above are logged and displayed in the dashboard. If you would like to see all the emitted logs, then you would need to modify the log level of the Ingester service to DEBUG. This is not recommended for production instances as the amount of logs will be increased significantly, but it is helpful for development purposes.

Modifying the Squirro Monitoring Dashboards#

You can customize both the Activity and Data Ingestion Logs dashboards by following the steps below:

  1. Click Setup from the Navigation menu.

  2. Click Visualization from the top menu.

  3. Hover over the dashboard you want to modify and click Edit.

Installing Squirro Monitoring#

Squirro Monitoring is a default part of every new project.

However, in releases prior to Squirro 3.6.8 LTS, Squirro Monitoring was not installed by default.

If your instance does not have Squirro Monitoring installed and you would like to add it, follow the instructions below:

  1. Log in with your Squirro ID at start.squirro.com.

  2. Click My Instances in the upper-right corner of the page.

  3. Select your instance.

  4. Click your project title in the upper-left corner of the page.

  5. Click the Plus icon to create a new project.

  6. Select Squirro Monitoring from the list of available project templates.

Installation Limits#

Only one Squirro Monitoring project is allowed per instance.

If you try to create a new Squirro Monitoring project, but a monitoring project already exists, an error message displays informing you which project is the current monitoring project. sm-delete-pro

You can do this by opening your existing project, then navigating to Setup → Settings → Properties and selecting Delete Project.

Deleting a Squirro Monitoring Project