SemanticSearch

SemanticSearch#

class SemanticSearch#

semantic | Perform approximate k-NN search on the paragraph embeddings using the HNSW vector index, and optionally rescore the results.

Leverages registered embeddings service to encode the query. Note: index embeddings have to be compatible with query embeddings

pydantic model PluginConfig#
Fields:

  • apply_phrases_as_knn_filter (bool)

  • apply_phrases_to_encoder (bool)

  • apply_rescore (bool)

  • embedding_type (squirro.common.clients.transformers.EmbeddingDataType | None)

  • filter_query (str)

  • k (int)

  • knn_boost (float)

  • knn_filter_stage (squirro.lib.search.relevancy.plugins.retrieve.embedding_retrievers.KnnFilterStage)

  • num_candidates (int)

  • perform_only_knn (bool)

  • similarity_threshold (float | None)

  • text (str)

  • vector_field (str | None)

  • worker (str)

PluginConfig.plugin_name: ClassVar[str] = 'semantic'#

Used to register and reference the plugin within a query.

field PluginConfig.text: str = ''#

Text snippet on which knn search should get applied on

field PluginConfig.filter_query: str = ''#

Squirro Query used to filter the KNN scope. Query against the paragraph index like filtering on facets, phrases or even terms. (no support of entities)

field PluginConfig.num_candidates: int = 100#

Amount of candidates to consider per shard

field PluginConfig.k: int = 20#

Amount of k nearest neighbours to retrieve

field PluginConfig.knn_boost: float = 100#

KNN Boosting factor

field PluginConfig.worker: str = 'query-fast'#

What deployed sentence-embeddings worker (@transformer-service) should be used

field PluginConfig.similarity_threshold: Optional[float] = None#

The required minimum similarity for a vector to be considered a match (optional). The scale of the threshold is dependent on the used similarity metric, and refers to the true similarity before it has been transformed into _score and boost applied - use the corresponding inverted score function.

field PluginConfig.embedding_type: Optional[EmbeddingDataType] = None#

The data type used to encode embeddings. Either float or byte. If set to byte, embeddings are quantized. If not set, the default type is read from the project configuration using the topic.search.default-embedding-settings config.

field PluginConfig.vector_field: Optional[str] = None#

The name of the vector field in Elasticsearch used for querying e.g. 384-byte-intfloat/multilingual-e5-small.

field PluginConfig.apply_phrases_as_knn_filter: bool = True#

Query Processing might rewrite the user query to match detected entities exactly as a phrase. If this is enabled, then semantic search is only applied on documents that match this phrase exactly (with configured proximity slop). All searchable fields are used for the pre-selection to fulfill the condition.

field PluginConfig.apply_phrases_to_encoder: bool = True#

Query Processing might rewrite the user query to match detected entities exactly as a phrase. If this is enabled, then the additional phrase is appended to the user terms and used in the query embedding call.

field PluginConfig.perform_only_knn: bool = False#

The query generator always adds some top-level query filtering options (project scoring roles, ACL, filtering on deleted sources etc). Those can be removed to only execute the standalone knn query. This is especially useful when RRF hybrid search is performend, and the underlying semantic search should be as fast & pure as possible.

field PluginConfig.apply_rescore: bool = False#

Whether to apply rescoring after the initial kNN search

field PluginConfig.knn_filter_stage: KnnFilterStage = KnnFilterStage.pre#
Contents