How to integrate remote models that power semantic search and other NLP tasks

How to integrate remote models that power semantic search and other NLP tasks#

Server Configuration topic.nlp.remote-services-connection

{
    "auth": {
        "password": "",
        "username": ""
    },
    "headers": [],
    "timeout": null,
    "service_variant": "transformers.reranker.*",
    "url": "https://<YOUR_HOST>/transformers/rerankers/baai-bge-v2-m3/"
}

Remote Services (service_variants) are registered following the format of <model namespace>.<language>.<worker>, for example spacy.en.fast. A fallback worker per namespace can be configured as the default in case a requested worker for a service variant cannot be resolved/reached. The fallback is configured using the * syntax (<model namespace>.*).

Supported model namespaces and tasks are:

  • reranker: transformers.reranker.* (To rerank passages for a given query)

  • sentence-embeddings: transformers.sentence-embeddings.* (To embedd queries and passages)

  • qa: transformers.qa.* (Extractive Question Answering)

  • spacy: spacy.* (Spacy NLP analysis)

API Specification for given namespaces#

Squirro provides docker images for each namespace, with pre-packaged models. It is possible to integrate any kind of model (on-premises, managed) by implementing a service that adheres to the API specification of the namespace, and registering it accordingly within topic.nlp.remote-services-connection.

The services are expected to implement a top-level _invoke API, with a specific payload for each task (request and response schema).

See the expected API parameters below:

Reranker#

RerankerPayload

CrossEncoderResponse

Sentence Embeddings#

SentenceEmbeddingsPayload

SentenceEmbeddingsOutputs

Contents