3.8.0 - Release Notes

3.8.0 - Release Notes#

Squirro 3.8.0 was released May 30, 2023.

What’s New#

  • The ZIP connector has been renamed to Documents and now, in addition to creating a new source using a ZIP file of supported documents, it supports creating a new source using a single document (a single PDF document, for example).

Document Import

  • A new Query Syntax feature rank_by:{}: allows you to rank on optional scoring signals to boost matching documents without applying an overall filter. For example, guest wifi rank_by:{ source:faq } enforces matches on guest wifi but ranks documents higher that are tagged with source:faq.

  • A new Query Syntax feature profile:{}: allows you to inject Scoring Profiles dynamically within the query syntax. For example, to filter items that have been read by the user in the last 10 days, the syntax is profile:{ plugin:last_read $last_days:10 }. Another use case is to apply scoring profiles as additional ranking signals by combining the literals rank_by & profile, for example genai how to rank_by:{ profile:{last_read} }. This matches documents with ``genai how to `` and further boosts last read items.

  • Added a new type of content streamer called QueueFileSystemStreamer. This streamer extends the existing FileSystemStreamer with the idea that each data source has its own distinct queue. It can be enabled by modifying the /etc/squirro/common.ini file, by setting the stream_type option of the content section to filesystem-queue from filesystem.

  • Introduced AutoML Templates within AI Studio. It works by training multiple models with automated hyperparamater tuning and model ensemble and automatically returns the best model.

  • Added documentation for the content streamers of the data processing pipeline. See Content Streamers.

  • Added an autogluon classifier to the libNLP classification steps.

  • You can now store metadata information on sub-items (pages) in the same way you can for top-level items. This metadata information can be used for internal operations and is not exposed via the Facet Aggregation API, which only considers top-level items.

Improvements#

  • Upgraded Elasticsearch to 8.7.1.

  • Added a hover state to the Candidate Set list.

  • When searching, improved the retrieval of relevant documents and highlighting of matching tokens (including lemmas & stems) based on the detected language of the provided query. The language distribution across indexed documents is considered to achieve stable language detection for short keyword queries.

  • Search indexing now uses the recommended (more aggressive) stemmers for all supported languages.

  • Removed typeahead suggestions that don’t include meaningful content, a standalone asterisk symbol (*) for example.

  • Now the horizontal mode is available without relevant communities in the Communities List widget.

  • Added a widget query to the Communities List widget.

  • Users scroll to the first highlighted section within a document after clicking on a QA widget answer.

  • Added support for custom templates in AI Studio and improved template handling generally.

Bug Fixes#

  • Fixed issues with entity aggregation and queries sorted by date.

  • Fixed an issue with event propagation being prevented when clicking within the Items card actions.

  • Changed typing_extensions to typing-extensions in requirements for FPM.

  • Fixed an issue with /v0/notes/query where non-English items returned with missing titles.

  • Fixed an issue with the Labels widget not considering additional queries.

  • Fixed issues with React Custom Widgets rendering.

  • Fixed an issue with entities scroll occurring off the screen inside the PDF viewer.

  • Fixed an issue with entity feedback not working inside PDF documents.

  • Fixed entity feedback suggested labels not showing.

  • Fixed a scripting issue with reindexed Elasticsearch indices when the source index has no analysis section.

  • Now the React version of the ItemDetail modal is used in the QA widget.

  • Inside the SharePoint dataloader plugin, replaced yield from () with return iter([]).

  • Improved entity choice for feedback when multiple entities are overlapping.

  • Fixed an issue with the Community Banner widget not reflecting dashboard queries and additional widget queries, ensuring that the Community Banner widget correctly uses a configured dashboard query to render the number of documents within a community.

  • Previously, when new monitoring projects were created, the popular queries were using old project IDs because of the cached value. This has been fixed so that the new project IDs are used.

  • Fixed a bug where Entities Stacked Bar Charts in a GlobalSearch wasn’t filtering properly with queries.

  • Fixed an issue with Notes comments not hiding when toggled off in the side panel top bar.

Breaking Changes#

  • The public methods enqueue and enqueue_data of the FileSystemStreamer will now return the absolute path to the enqueued data file.

  • If you are using custom templates within AI Studio, they must be handled using the new structure.

Installation and Upgrade#

You will have to resolve at least the following config files when upgrading from Squirro 3.3.0:

  • /etc/nginx/conf.d/ssl.inc

  • /etc/squirro/common.ini

For new installations, please follow the Installing Squirro on Linux instructions.

To upgrade an existing installation, please consult Upgrading Squirro.

Contents