3.10.0 - Release Notes

3.10.0 - Release Notes#

Squirro 3.10.0 was released March 14, 2024.

Learn more about the Squirro Release Process.

What’s New#

  • New “My Communities” Filter Widget: A new filter widget for “My Communities” has been added to enhance user navigation.

  • Redesigned “Community List” Widget: The “Community List” widget has undergone a redesign for improved usability.

  • SquirroGPT Project Templates: Users can now access SquirroGPT project templates from the sidebar, enabling the creation of new projects with OpenAI and Mixtral LLMs.

  • Improvements to Squirro Ansible: We’ve improved support for build_mode, added control for cache settings in common.ini, and introduced support for hourly/daily snapshots, among other enhancements.

Improvements#

  • Initial support for rerun from index job monitoring: Use the rerun_job_id label in the Data Ingestion Logs dashboard of the Squirro Monitoring project to track pipeline steps. Find the ID of the triggered job in the datasource service logs at INFO level (e.g., tail -f /var/log/squirro/datasource.log | grep 'rerun_from_index').

  • Embedding models are now lazy-loaded: This change is part of the genai embedding models’ deprecation process, ensuring functionality for older projects still using the genai service retriever.

  • Item indexing: Index pipeline step now offers options to index Squirro items or associated paragraphs. Rerun from index now includes sub-items by default, controlled by the datasource.rerun.index.include-sub-items setting. Override this in the Python SDK with the include_sub_items parameter in the rerun_pipeline_workflow method.

  • New community chip: Introduced a special chip feature for users to easily identify their subscribed communities.

  • Squirro Assets: Introduced a configuration option to use the --no-dependencies flag with pip when installing packages from a requirements.txt file for Squirro Assets.

  • Squirro Ansible: Now ensures consistent tooling by installing Python virtualenv, Elasticsearch client, requests, and certain JSON packages on storage-only nodes. Improved Squirro Ansible documentation.

  • SQGPT Configuration: Introduced SQGPTConfig schema to the configuration service for easier setting and validation. The service can now interpolate values from .ini files in sections or keys with dots (e.g., ${sqgpt\.settings.api_key} for the api_key under sqgpt.settings section).

  • Performance improvement: Enhanced Redis integration as the caching backend for the @cached decorator, now supporting prefixes/namespaces, auto-expiring keys, and asynchronous cache insertion. Ehanced efficiency of individual backend requests by enabling multiple caches by default, complete with a proper cache invalidation strategy. These caches pertain to authentication handling, project metadata retrieval (including facets data), and user information (including project permissions).

  • Global Search: Facets are highlighted in ItemCard and LabelsWidget.

  • Project Export: Optimized memory usage and increased speed during project export that includes items.

  • Widgets: Added overrides, documentation and tests for CommunityItem widget**.

  • Generative AI: Query context and paragraph id/field mapping have been added to the genai service.

  • Dependencies: Bumped redis-py client from 4.3.4 to 5.0.2 (latest version).

Bug Fixes#

  • Improved monitoring plugin: Monitoring plugin has been updated to function even without access to the topic.ini configuration file.

  • Improved Label Display: When assigning a value to the Sources label in a Note, the system will now display the name of the data source, rather than its ID.

  • Resolved Issues with Tabs Widget: Fixed multiple issues that were preventing the selection and deselection functionality in the Tabs widget.

  • Fixed Item ID Mapping: Corrected the mapping of Item ID in the activity plugin when search feedback is sent.

  • Global Search Facets Removal: Fixed an issue where some global search facets couldn’t be removed from ItemCard keywords or LabelsWidget.

  • Default Pipeline Deletion: Resolved an issue where the previous default pipeline couldn’t be deleted without refreshing.

  • URL Query Facets Removal: Fixed an issue where facets applied by URL query couldn’t be removed from GlobalSearch/SearchWidget/ItemCard/LabelsWidget.

  • Cache Value Serialization: Ensured value serialization before starting the executor that caches it to prevent potential changes to the value before serialization.

  • SQLAlchemy Dependency: Fixed an issue where a missing SQLAlchemy dependency (tzdata) rendered Squirro Toolbox tools (e.g., squirro_data_load) unusable.

  • Squirro Status Command: Prevented the squirro_status command from failing when the topic.ini file is not readable.

  • GlobalSearch: after applying the favorite filter, additional query is not submitted from the first time

  • Processed Parameter: Fixed a bug where the optional processed parameter would throw an error if left empty.

  • Feedback Overwriting: Fixed a bug where providing feedback with different models would result in the feedback overwriting each other.

  • GenAI Plugin: Fixed the starter-questions and summarization endpoints in the GenAI plugin which were not working when a valid token was provided in the URL and no session existed.

  • Bulk Subscription Success Message: Fixed an issue where the success message was missing for bulk subscription in the communities list widget.

  • Dashboard Tabs Navigation: Removed extra space from dashboard tabs navigation.

  • Cached Project API Responses: Ensured that cached project API responses are cleared out when a project is deleted.

Breaking Changes#

  • Duplicate Item Deletion Task Removed: We’ve eliminated a task that previously deleted duplicate items each time project data was fetched. Consequently, the following parameters in the topic.ini configuration file are now obsolete: enable_duplicate_item_deletion, duplicate_item_deletion_shard_size, and duplicate_item_deletion_size.

  • Service Initiation Change: We’ve implemented a change to halt the initiation of a service if its configuration file is inaccessible to the service user. This modification is designed to highlight issues where incorrect permissions lead to inconsistent or malfunctioning service behavior. Typically, the error will be logged in /var/log/squirro/{service_name}/stderr.log.

Installation and Upgrade#

For new installations, find step-by-step instructions in Install and Manage Squirro with Ansible (recommended) or Installing Squirro on Linux.

To upgrade an existing installation, see Upgrading Squirro.

Contents