How to Connect Squirro Chat to a Third-Party LLM

Profile: System Administrator

This page describes how system or project administrators, or anyone with access to the Server or Setup spaces of a Squirro Chat project, can connect their Squirro Chat application to a third-party LLM. Squirro Chat requires a configured LLM to generate responses. Without one, the application does not function.

Connecting Squirro Chat to a third-party LLM involves two steps: obtaining the required information from your provider, then configuring your Squirro instance to use it.

Obtain Third-Party LLM Information

To set up Squirro Chat access to a specific third-party LLM, you need to set up the LLM externally and record certain information for use in the Squirro Chat configuration. Squirro Chat uses that information to authenticate your connection.

The information you need depends on the type of LLM you are connecting to. The table below outlines the required information for each type of supported LLM.

Model Type	Required Information
OpenAI	OpenAI API key
Azure	Deployment Name Base URL for API Azure API key API version (optional)
OpenAI-API compatible	API key (if required by the provider) Model name Base URL of the provider

Note

How you access this information depends on your specific third-party LLM provider.

The OpenAI-API compatible category covers any LLM that exposes an OpenAI-compatible endpoint, including open-source models hosted on third-party platforms and models running on your own infrastructure.

Configure Your Squirro Server or Project Settings

You have two options in terms of configuring usage of the third-party LLM:

• Configure your Squirro instance at the server level, meaning all existing and future Squirro Chat projects will use the third-party LLM you specify.

• Configure your Squirro Chat application at the project level, meaning only that particular project you are working within will use the third-party LLM you specify.

When both server-level and project-level configurations are present, the project-level configuration takes precedence for that specific project.

Note

The server-level and project-level configurations use different key names: genai.sqgpt.default-settings at the server level and genai.sqgpt.settings at the project level. That difference is intentional. The two keys serve different scopes and are not interchangeable.

Tip

Typically, Squirro recommends configuring your Squirro instance at the server level.

Configuring Your Squirro Instance at the Server Level

To configure your Squirro instance at the server level, follow the steps below:

1. Log in to your Squirro instance.

2. Click on Server in the spaces navigation bar.

3. Click on Configuration.

4. Search for the configuration named genai.sqgpt.default-settings.

5. Click on the edit icon to modify the configuration.

6. Adjust the default values following the LLM Configuration Values section.

You can also edit the configuration as a JSON dictionary by clicking the toggle:

Configuring Your Squirro Chat Application at the Project Level

To configure your Squirro Chat application at the project level, follow the steps below:

1. Log in to your Squirro instance.

2. Click on Setup in the spaces navigation bar.

3. Click on Settings.

4. Click on Project Configuration.

5. Search for the configuration named genai.sqgpt.settings.

6. Click on the edit icon to modify the configuration.

7. Adjust the default values following the LLM Configuration Values section.

Tip

To configure a certain setting to fall back to the server setting, select Default to Server Setting (or assign null to the setting in the JSON editor):

LLM Configuration Values

The following configuration values apply to both the server-level and project-level configuration steps above. Paste appropriate values for your third-party LLM into the configuration.

Configuration for OpenAI Models

{
    "llm": {
        "kind": "openai",                          # Indicates OpenAI model (required)
        "openai_api_key": "<your OpenAI API key>", # Your OpenAI API key (required)
        "model": "gpt-5.2"                         # Model name (required)
    }
}

Configuration for Azure-hosted OpenAI Models

{
    "llm": {
        "kind": "azure",                           # Indicates Azure-hosted model (required)
        "deployment_name": "<your deployment>",    # Your deployment name (required)
        "openai_api_base": "<your base URL>",      # Base URL for the API (required)
        "openai_api_key": "<your Azure API key>",  # Your Azure API key (required)
        "openai_api_version": "2023-05-15"         # API version (optional, default is "2023-05-15")
    }
}

For more information on connecting to a Microsoft Azure LLM, see the How to Connect Squirro Chat to a Microsoft Azure LLM page.

Configuration for OpenAI-API Compatible Models

Important

The LLM you connect must support tool calling. Squirro agents rely on tool calling to function. Models that do not support it cannot be used with Squirro Chat. For more information, see the Large Language Model Support page.

OpenAI-API compatible models are LLMs that expose an HTTP API following the same request and response format as the OpenAI API. That compatibility makes it possible to connect Squirro Chat to a wide range of models beyond standard OpenAI offerings, including open-source models hosted on third-party cloud platforms.

For organizations that already run LLMs on their own infrastructure, Squirro Chat can connect directly to that infrastructure using the same configuration, provided the deployment exposes an OpenAI-compatible endpoint. For compatibility requirements, see the Large Language Model Support page.

{
    "llm": {
        "kind": "openai",                          # Same value as for OpenAI. The base_url field determines the provider.
        "openai_api_key": "<your API key>",        # Your API key (required, or any string if no auth needed)
        "model": "<your model>",                   # Model name (required)
        "base_url": "<base URL of provider>"       # Base URL of the model provider (required)
    }
}

Advanced configuration parameters

All LLM configurations accept the following optional parameters in addition to the required fields shown above:

• temperature: Sampling temperature (float, default 0.0). Higher values produce more varied responses.

• top_p: Nucleus sampling threshold (float, default 1.0).

• seed: Optional integer seed for more reproducible outputs. Support varies by model.

• reasoning_effort: For GPT-5 and o-series models only. Accepted values are "low", "medium", "high", "none", and (for GPT-5 models) "minimal".

• request_timeout_s: HTTP timeout in seconds, as an object with connect and read keys (for example, {"connect": 5, "read": 120}).

• context_length: Override the model context window size (integer). Set to -1 to auto-detect.

• max_retries: Number of retry attempts on API failure (integer, default 3).

Verify Your Configuration

After saving the configuration, open a Squirro Chat project and send a test message. If the LLM responds correctly, the configuration is working.

If you see an error, check the following:

• The API key is correct and has not expired.

• The base URL is reachable from the Squirro server (not just from your local machine).

• The model name matches exactly what the provider expects.

• The LLM supports tool calling. For more information, see the Large Language Model Support page.

If the issue persists, visit the Squirro Support website and submit a technical support request.

Timeout Configuration for Reasoning Models

Some LLMs require significantly more time to generate responses than the default timeout allows. Common scenarios include reasoning models operating at high reasoning levels (for example, OpenAI GPT-5) and large models running on constrained hardware. When the timeout is exceeded, Squirro Chat returns an error before the LLM finishes generating.

To accommodate longer response times, increase the frontend.sqgpt.default-timeout project configuration. That setting controls the overall timeout for Squirro Chat functionality, including Batch Q&A, the Chat widget, Chat with Document, and Summarization.

1. Navigate to Setup → Settings → Project Configuration.

2. Search for frontend.sqgpt.default-timeout.

3. Edit the configuration and increase the value (in milliseconds).