How to Use Scoring Profiles to Customize Document Relevancy Scoring

How to Use Scoring Profiles to Customize Document Relevancy Scoring#

Profiles: Project Creator, Search Engineer

Warning

This feature is in technical preview and may change in a future release.

This page presents an overview of how to use Scoring Profiles and Scoring Roles within Squirro Cognitive Search.

Scoring profiles and roles are configured by search engineers, then used by project creators to finetune the search experience for end users.

Reference: Learn more about Scoring Profiles and Roles.

When Scoring Profiles Should be Used#

Scoring Profiles should be added whenever the default scoring algorithm based solely on text matching does not meet your expectations and you want to consider additional document features for document relevancy ranking.

Figure 1: Introducing Relevancy Signals into Document Retrieval

https://s3.amazonaws.com/download.squirro.net/docs/technical/search/relevancy/search-pipeline_scoring_profile_overview.drawio.png

How to add Scoring Profiles#

Scoring Profiles are maintained within the Configuration Service and can be configured for each project individually.

Project Configuration topic.search.document-scoring-profiles

{
    {{profile_name}}: {
        {{profile_type}}: {{profile_value}}
        stage: string
        boost: float
        debug: bool
        config: dict
    }
}
Reference: Scoring Profile Schema
{{profile_name}}
Type: string
Required: True

The profile name ({{}} substituted with actual name). Scoring Roles are linked with profiles per name.

{{profile_type}}
Type: string
Required: True

With {{profile_type}} either query, script or asset_name.

stage
Type: string
Required: False
Default: "query"

Control if the profile is applied on all matching documents or on the most relevant subset. Can be either query or rescore

boost
Type: float
Default: 1.0

Multiply the profiles document matching score with the provided boosting factor to increase the Profile’s overall impact.

debug
Type: bool
Required: False
Default: False

If produced profile query clauses get logged. Logs are located at topic.log.jsonl or machinelearning.log.jsonl respectively.

config
Type: dict
Required: False
Default: {}

Profile specific additional configuration, useful to provide configuration of custom plugins or built-in rescore stage.

How to Apply Profiles Using Scoring Roles#

Project Configuration topic.search.document-scoring-roles

Scoring Roles define what Scoring Profiles should actually get executed.

Certain Profiles might make sense to get applied to all users, whereas others only to a certain group of people.

The role configuration allows a versatile way of configuring the mapping between user’s and Scoring Profiles. Two options for role-profile mapping are supported:

  • Groups: Squirro maintains group relationship for all users. This information can be used to enable Profiles only for users members of a certain Squirro Group.

  • User Data: Squirro can also load and store key-value data from a 3rd party system, for example to what department a User belongs to in the Source System. This data can also be extended with custom user-preferences

{
    {{role_name}}: {
        enabled: bool
        applies_to: {
            groups: []
            "${{user_value}}": []
        }
        profiles: []
    }
}
Reference: Scoring Role Schema
{{role_name}}
Type: string
Required: True

The role name ({{}} substituted with actual name)

enabled
Type: bool
Required: False
Default: True

Control if role should get evaluated and used.

applies_to
Type: dict
Required: False
Default: {}

Control for whom the role should be applied. Works with Squirro Group (key groups) mapping and dynamically evaluated User Information coming from 3rd party authentication providers (key ${{user_value}} like $department). Dynamically mapped keys have to start with $ to be evaluated.

Per default the role gets applied for all users.

profiles
Type: List[str]
Required: True

List of associated Scoring Profiles that should be executed.

Role for all users#

Boost recently modified documents for all users#
{
    "boost_recently_modified": {
        "enabled": true,
        "applies_to": {},
        "profiles": ["recently_modified__boost_decay"]
    }
}

Note: A missing or empty applies_to key enables the role for everyone.

Role for specific users / groups#

Arbitrary user-metadata can be referenced using the $ prefix like applies_to.$department. Squirro Groups are referenced like applies_to.groups

Boost popular items only for users belonging to department “Engineering”#
{
    "boost_popular_items_for_IT": {
        "enabled": true,
        "applies_to": {
            "$department": "engineering"
        },
        "profiles": "popular_items"
    }
}
Boost popular items for users belonging to group `internal`, department `engineering` or `sales`. Role gets applied as soon as one of the `applies_to` condition matches#
{
    "another_example": {
        "enabled": true,
        "applies_to": {
            "groups": ["internal"],
            "$department": ["engineering", "sales"]
        },
        "profiles": "popular_items"
    }
}
Contents