Query Syntax#

Profiles: Project Creator, Search User

You can use query syntax within Squirro search bars to return better, more refined search results.

Query syntax is the way you structure a search input. Operators (AND, OR, NOT, +, and -), brackets, wildcards (* and ?), and quotation marks let you narrow or broaden which documents match. Bookmarking, boosting, sorting, and time increments let you adjust how the matching results are presented.

This page details the query syntax techniques available within Squirro. To run the same queries from an integration and choose which item fields are returned, see the Search API page.

Project creators can configure query syntax options. Search users can use the techniques outlined in this page to improve search accuracy.

Defaults#

By default, the title and body fields are taken into account when searching.

A run of plain search terms is treated as one unit. Inside that unit the terms are combined with OR, and the minimum-should-match strategy configured for the project decides how many of them a document has to contain. A phrase, a field filter, or a group forms a separate unit, and units that sit next to each other are combined with AND.

For example, in one two "three four" the run one two is one unit and the phrase "three four" is another. The phrase has to match, while minimum-should-match decides whether one, two, or both are also required.

Dynamically tagged text labels can also be made searchable.

For more information, see the How To Handle User Query Terms Correctly page.

Boolean Operators#

Use AND, OR, NOT, + (plus sign) or - (minus sign) to explicitly combine terms. Be aware that the operators need to be in all capital letters.

The following restrictions apply:

  • The + or required operator requires that the term after the + symbol exists somewhere.

  • The - or prohibit operator excludes documents that contain the term after the - symbol. The term has to follow immediately, since - squirro with a space is read as plain text.

Example query

Description

squirro AND memonic

Search documents that contain squirro and memonic.

squirro OR memonic

Search documents that contain either squirro or memonic.

+memonic -squirro

Search documents that contain memonic but do not contain squirro.

squirro NOT memonic

Search documents that contain squirro but do not contain memonic.

Grouping#

Use parentheses to group terms, so that a Boolean operator applies to the whole group instead of to a single term. Grouping is what lets you combine AND and OR in one query without the result depending on operator precedence.

Example query

Description

(java AND solr) OR (python AND elasticsearch)

Search documents that contain both java and solr, or documents that contain both python and elasticsearch.

nektoon AND (squirro OR memonic)

Search documents that contain nektoon and either squirro or memonic.

Comparison Operators#

A comparison operator can replace the colon after a field name. It is meant for dates and numbers, including the numeric entity fields covered in the Entity Search section. Nothing stops you from using one on a text label, but the comparison is then alphabetical rather than numeric:

Operator

Meaning

>

Greater than

>= or =>

Greater than or equal to

<

Less than

<= or =<

Less than or equal to

Spaces around these operators are optional, so $_size > 100000 and $_size>100000 are equivalent.

= is not in that list, because on its own it is not a comparison operator. It is a second way to write the separator, so $item_language=en matches an exact value exactly as $item_language:en does. Unlike the operators above it takes no surrounding spaces: $_size = 100 is read as plain search terms rather than as a query on the field.

To bound a field on both sides, use the range syntax described in the Range Search section rather than combining two comparison operators with AND.

Term-Level Boosting#

Note

This feature only boosts individual terms or facets, to boost the full query clause see Boosting Queries By Optional Ranking Signals.

You can prioritize individual elements of a query by boosting them. Sort by relevance to see the changed relevance scores.

Example query

Description

France^10 Europe

Search for France and Europe, but boost matches of France.

France OR Country:France^10

Search for France in full text as well as in the Country label, and boost the items that carry the value in that label.

France^0.1 Europe

Search for France and Europe, but de-prioritize matches of France. The default boost is 1.0.

Sorting#

You can use the following query syntax to sort the result. The square brackets mark the optional parts and are not typed as part of the query:

sort:<field_name>[:<order>]

Where <order> is either asc for ascending or desc for descending. The order suffix is optional, and the default order is descending.

<field_name> is either a label name, an item field name written with the $ prefix, or one of the following sort keywords:

Keyword

Sorts by

date

Date the item was created. This is the default, and it sorts on $item_created_at.

relevance

Relevance score of the item.

author

Item author.

language

Item language.

provider

Item provider.

source

Data source the item came from.

sort:read and sort:starred are also accepted, but neither orders the results by your own read or bookmark state, and both return the same order. To narrow the results to items you have read or bookmarked, see the Bookmarked and Read Items section.

Additionally, you can add a second (or third etc) sorting criteria by adding

[;<2nd_sort_field>[:<2nd_order>]]

to the query syntax.

Example query

Description

sort:date

Sort by date (descending order by default).

sort:date:asc

Sort by date in ascending order.

sort:relevance:desc

Sort by relevance in descending order.

sort:my_sortable_facet:desc;date:desc

Sort by my_sortable_facet in descending order, and then by date in descending order.

(Moon landing) sort:date

Sort query by date (default order is descending).

Multiple Sorting Criteria#

You can sort on more than one criterion. The criteria apply in the order the query lists them: the first is the primary criterion, and each one after it breaks ties among the items the previous criteria rank equally.

Both of these queries sort by date in descending order, then by title in ascending order:

sort:date:desc;$title:asc
sort:date:desc sort:$title:asc

The first form lists the criteria inside one sort keyword, separated by a semicolon. The second repeats the sort keyword for each criterion. Both forms behave the same.

Important

Check the spelling of every label you sort on. A criterion that names a label the project does not have is not reported as an error: the query returns no results at all, whichever form you use. An unknown item field written with the $ prefix is ignored instead, and the other criteria still apply.

Time Increment#

You can control the time increments shown in the main timeline and in the dashboard widgets. To do so, add time_increment:<value> to a query.

A value is either a unit on its own, such as time_increment:day, or a number followed by a unit, such as time_increment:12hours. Eight units are available:

Unit

Short form

Example with a number

second

none

time_increment:30seconds

minute

m

time_increment:15minutes

hour

h

time_increment:12hours

day

d

time_increment:4days

week

w

time_increment:8weeks

month

none

time_increment:6months

quarter

q

time_increment:2quarters

year

y

time_increment:3years

Each unit accepts its singular and plural spelling, so time_increment:day and time_increment:days are equivalent. Two units have no short form: month, because m already means minute, and second, where the single letter is not handled correctly. Write second or seconds in full.

Put only one time_increment clause in a query. When a query holds more than one, only one of them takes effect, and which one depends on how the query is written. That also applies when a query is assembled from several parts, such as a search query combined with dashboard filters, so set the increment in one of them only.

Choose an increment that keeps the number of buckets small. The finer the increment, the more buckets the results split into, and the slower both the query and the timeline become. Splitting a year of data into hourly increments, for example, is far more costly than splitting it into weekly ones.

Bookmarked and Read Items#

Bookmarked items are the items a user has added to their Bookmarks collection. The query syntax still names them with the legacy starred keyword, carried over from the previous starred system.

is:starred      # Items in the user's Bookmarks collection
is:unstarred    # Items not in the user's Bookmarks collection

is:starred reads the Bookmarks collection directly, so it works in any project. It matches nothing when the collection is empty.

is:unstarred reads the same collection, so it works everywhere too.

is:read and is:unread are also accepted, but they do not reflect which items you have read, so do not rely on their results. To find the items you have read, use the last_read scoring profile instead. By default it covers the 10 items you read most recently within the last 30 days, and count and last_days widen that, as in profile:{ last_read count:1000 last_days:365 }. It requires activity tracking. Without it, or when you have read nothing within last_days, the search returns no results. For more information, see the Scoring Profiles and Queries section.

Write these keywords in lower case and check them for typos. An unrecognized name such as is:STARRED is never reported as an error. Depending on the project configuration, it is either dropped from the query, so the search comes back unfiltered, or it matches nothing.

Scoring Profiles and Queries#

Accessing Scoring Profiles#

Scoring profiles use document metadata as additional filtering criteria to return the most relevant documents.

For an introduction to scoring profiles, see the How to Use Scoring Profiles to Customize Document Relevancy Scoring page.

Those profiles can be directly used in the query syntax using the profile:{} literal.

Scoring profiles can either reference a configured profile from the project configuration by name or leverage a plugin without any project configuration required.

Out-of-the-Box Plugins#

The plugins available out of the box include the following:

  • is_new

    Marks items as new when they were created after the last browsing session of the user. Requires activity tracking.

  • last_read

    Boosts the items the user read recently. The more recently the item was read, the higher the score. Requires activity tracking.

  • popular_item

    Boosts popular items. The more popular the item is within the project, the higher the score.

  • concept

    Runs concept search from within the search bar.

  • recommend_on_searches

    Recommends items based on the search history of the user.

  • subscribed_communities

    Adds community filter queries for the communities the user subscribes to.

  • recency_boost

    Makes recent documents more relevant.

For more information about these plugins, see the Scoring Plugins page.

Example Queries#

The examples below show how to reference different scoring profiles in the search bar.

Identify New Items#
# Mark items as new with default settings (1 hour minimum session age)
profile:{ is_new }

# Configure minimum session age (only consider sessions older than 2 hours)
profile:{ is_new min_session_age:2h }

# Use different time units for session age
profile:{ is_new min_session_age:1d }    # 1 day
profile:{ is_new min_session_age:30m }   # 30 minutes

# Combine with keyword search to find new items matching a topic
technology profile:{ is_new }

The is_new profile adds a new flag to the runtime_labels field for items created after the last browsing session of the user. The min_session_age parameter specifies the minimum age of a session to be considered for calculating the last activity date.

This profile is useful for:

  • Highlighting fresh content for returning users.

  • Creating “what you missed” views.

  • Surfacing recently ingested items.

The is_new functionality requires activity tracking to be turned on in the monitoring project. Without activity tracking, this profile has no effect.

For detailed information about new items, see the Read, Unread, New, and Bookmarked Items page.

Filter by Last Read Items#
# Filter on last read user items with default settings
profile:{ last_read }

# Filter on last 5 read user items
profile:{ last_read count:5 }

# Filter on last 50 read user items
profile:{ last_read count:50 }

# Combine last read filter with keyword search
artificial intelligence profile:{ last_read count:20 }

The last_read profile restricts the results to items the current user has read, and scores the more recently read ones higher. The count parameter sets how many of the most recently read items are taken into account, and defaults to 10. The last_days parameter limits how far back to look, and defaults to 30 days.

This profile is useful for:

  • Finding items the user read previously.

  • Building reading history views.

  • Creating personalized dashboards based on user activity.

Read state is tracked per item, both when the item was read and how many times.

For detailed information about read and bookmarked items, see the Read, Unread, New, and Bookmarked Items page.

Recency Boosting#
# Boost documents by their item_creation date (default date field ranking)
profile:{ recency_boost }

# Boost documents by a custom date_time label, for example `last_updated`
profile:{ recency_boost date_field:last_updated }
Combining Profiles#
# Search for specific documents within the search history of the user, and boost resulting items by their date recency
elasticsearch tutorial profile:{ last_read count:1000 } profile:{ recency_boost }

# Run concept search on a given text with additional boosting of recently trendy items
profile:{ concept text:"covid outbreak" } profile:{ popular_item last_months:1 }

# Filter documents that belong to the communities the user subscribes to, and boost the resulting items by their date recency
profile:{ subscribed_communities } profile:{ recency_boost }

Boosting Queries By Optional Ranking Signals#

Warning

This feature only works when sorting by relevance (sort:relevance).

Besides ranking by profile, you can add extra scoring clauses with the rank_by:{} and scale_by:{} literals.

A document does not have to match these clauses to be retrieved. Documents that do match receive a boost to their relevance score.

A rank_by:{} clause adds the score of the wrapped query to the total score of each document, so its contribution is additive.

A scale_by:{} clause multiplies the total score of a matching document by the factor given after the closing brace, as in scale_by:{...}^2. Leaving the factor off applies 1.1, which raises the score by 10%. A rank_by:{} clause takes the same trailing factor.

These clauses always apply to the whole query, even when they sit inside a group. So (wifi rank_by:{source:official}) OR (login rank_by:{source:unofficial}) returns documents matching wifi or login, and raises the score of those tagged official or unofficial in the source field.

Example query

Description

cheese rank_by:{ Country:France^10 }

Search for the term cheese but boost items that additionally are tagged with Country:France. The final score per document is the score for the term cheese added to the score for the country tag (which is multiplied by 10 before being added). Items that do not match France are still returned.

wifi login rank_by:{ source:official^5 OR is_faq:True^10}

Search for the term sequence wifi login with most important relevancy signal being is_faq and the second signal being items coming from the official named datasource.

rank_by:{ entity:{"properties.Sentiment":"negative"} } rank_by:{ source:official^5 OR is_faq:True^10}

Boost items that are tagged on sentence level with a classification output (entities), in this case negative sentiment.

arbitrage scale_by:{ $item_created_at>="2023-01-01" }^2

Search for the term arbitrage and boost the score of documents that are created after 2023 by a factor of 2.

You can also put a scoring profile inside a rank_by clause, for ranking specific to one use case such as a single dashboard or widget.

Example query

Description

rank_by:{ profile:{plugin:last_read $last_days:7} }

Match all available items in the project, but boost the user’s recently-read items according to read time. The more recent, the more relevant.