Set Up the Proxy

Set Up the Proxy#

This page covers the first step of embedding Neo chat: installing the package, preparing the service account, and running the proxy behind your reverse proxy. For the overview and the requirements, see the Embedding Neo Chat in Your Own App or Site page.

Installation#

The SDK is the @squirro/neo-sdk package on the public npm registry. For the package page, see the npm website. The package holds the proxy and its templates, the neo-sdk.js script for the script-tag embed, and the React components. Your Squirro instance serves none of them. The server that runs the proxy needs Node.js 22.18 or later, which includes npm.

The package is licensed under the PolyForm Shield License 1.0.0, in the LICENSE file of the package.

Note

Node.js is a program that runs JavaScript on a server rather than in a web browser. With it, a JavaScript program can listen for web requests, read files, and call other services, like any other server program. The proxy is such a program: node proxy-node.mjs starts it, and Node.js runs it until you stop it.

Node.js comes with npm, the tool that downloads packages such as @squirro/neo-sdk or Express from the public npm registry into the node_modules/ folder of your project. Express is an optional package that makes building web servers on Node.js more convenient. Node.js can also run a web server on its own, which is what the proxy-node.mjs template does.

Like any server software, Node.js receives security fixes and new versions, and keeping it up to date is your responsibility. See the Components You Maintain section of the Go Live and Maintain page.

npm installs the matching @squirro/neo-core package with the SDK. Each SDK minor version is pinned to the core of one Squirro release, and the version of that core is the version of the release. To see which Squirro release an SDK version belongs to, run this command:

npm view @squirro/neo-sdk@<version> dependencies

Install the SDK with a tilde range on its minor version, for example @~0.1 for SDK minor version 0.1. The range keeps you on that minor version and brings its patch versions, which is how fixes for an LTS release arrive. Do not install with @lts. The lts dist-tag is shared by every LTS release, so it can point at the SDK of an LTS release other than yours.

Install the package on the server that runs the proxy, in the folder of your server project, for example a new one:

mkdir chat-proxy
cd chat-proxy
npm init -y
npm install @squirro/neo-sdk@~0.1

The same package holds neo-sdk.js, so the script-tag embed needs nothing more to install. The React components need the package in your React app as well. For both, see the Add the Chat to Your Page page.

Where to Run the Proxy#

The proxy runs on your own infrastructure, next to the site or app that embeds the chat. Two setups are possible:

  • On the server of your site or app

    The default that the templates expect. The proxy runs on the same machine as your site or app, behind the reverse proxy that serves it, for example at /api/chat. Your page and the proxy then share one origin, which is the combination of scheme, host name, and port, such as https://docs.example.com.

    Sharing one origin has two advantages. Browsers stop a page from reading the responses of another origin, unless that origin explicitly allows the page, a rule called Cross-Origin Resource Sharing (CORS). On one shared origin, that rule does not apply, so the proxy needs no CORS setting. The session cookie of your site or app also reaches the proxy, for a proxy that signs readers in with it.

  • On a separate server or in a container

    The proxy runs on its own host, behind a reverse proxy. Set HOST=0.0.0.0 so that the reverse proxy can reach it, and set TRUST_PROXY to the number of reverse proxies in front. Then make sure that only the reverse proxy can reach the port of the proxy, for example with a firewall rule or a private container network. A client that reaches the port directly bypasses HTTPS and the rate limit.

    If the proxy is then on another origin than your page, such as https://chat-proxy.example.com, list the origin of your page in ALLOWED_ORIGINS. Sign readers in with anonymous or token, because a session cookie does not reach another origin. See the Signing Readers In section of the Add the Chat to Your Page page.

For the reverse proxy settings that both setups need, see the Behind a Reverse Proxy section.

Get Squirro Credentials for the Proxy#

Do not give the proxy personal or administrator credentials. Instead, follow these steps:

  1. Create one dedicated, non-personal service account for the integration in your identity provider. Squirro Neo accepts sign-ins only through the identity provider that your Squirro instance uses for single sign-on (SSO), so a local account with a password cannot sign in to it. Make sure that the role mapping of your Squirro instance gives the account the reader server role, which the Squirro UI shows as Restricted, for example by mapping a group that contains only that account to reader, ahead of any broader rule. For the role mapping, see the External Authentication page.

  2. Sign in to Squirro Neo as the service account once, which creates the account on your Squirro instance. Then add it only to the projects that the chat answers from, with the Reader project role. In the Setup space of each project, add the account itself under Settings → Project Members, not a Squirro group that holds it, because each sign-in removes the group memberships that your identity provider does not assert.

  3. Sign in to Squirro Neo as the service account, select the profile picture, navigate to Settings → API access, and select Generate token. Copy the token, and set it as SQUIRRO_REFRESH_TOKEN on the proxy. Keep it nowhere else.

Create the token only after the server role of the service account is set, because changing the role later revokes the token. Signing in again, for example for the memory steps, does not revoke the token. Anyone who signs in as the service account can display the token again, so protect the sign-in of the service account in your identity provider as well as the token, and limit who can use it.

If your identity provider cannot hold such an account, create a local account and its token through the API instead, as described on the Create a Service Account page, and use the API equivalents of the memory steps. That route needs OAuth client credentials that a platform administrator registers on the server.

Prepare the Service Account and the Project#

Every reader of the chat works as the service account, so readers share the memory, the scheduled tasks, and the file storage that your Squirro instance keeps for that account. Before you start the proxy for the first time, turn off memory, accept that readers share the scheduled tasks, and either accept that they share the file storage or turn it off in the project. The proxy does not start while one of the three is on and not explicitly allowed.

Memory#

Turn off memory for the service account in every project that the proxy serves. With memory on, something one reader tells the agent comes back in the answers of another reader, and any reader can ask the agent to forget everything.

To turn off memory, follow these steps:

  1. Sign in to Squirro Neo as the service account.

  2. In each project that the proxy serves, select the profile picture, navigate to Settings → Memory, and turn off Use memory in this project. The API equivalent is PUT /service/genai/v1/projects/{pid}/memories/settings with {"enabled": false}. If Settings does not list Memory, memory is not available in that project, and there is nothing to turn off.

  3. Delete what was remembered while memory was on, because turning off memory keeps those entries. Select Clear memory on the same page, and then Delete all under Recently deleted. While memory is off, cleared entries are otherwise kept, and anyone who signs in as the service account can restore them. With the API, call POST /service/genai/v1/projects/{pid}/memories/_clear with {"mode": "erase"}, and then with {"mode": "erase", "scope": "recoverable"}, which also deletes the entries that were already under Recently deleted.

To check the setting, call GET /service/genai/v1/projects/{pid}/memories/settings. It returns {"enabled": false, ...}, or 404 for a project without memory.

Set ALLOW_SHARED_MEMORY=1, or allowSharedMemory: true, only if every reader may share one memory. The proxy also does not start while it cannot read the memory setting of a project in PROJECT_ID.

Scheduled Tasks and File Storage#

One reader can list, change, run, or delete the tasks that another reader scheduled. While file storage is on, the conversation of one reader can also read what the conversation of another reader wrote to the file storage. The agent can write there on its own, for example notes that it keeps for later questions, so what one reader tells the agent can reach the answers of another reader.

A project cannot turn off scheduled tasks, so every reader always shares those of the service account.

To accept both, set ALLOW_SHARED_AGENT_TOOLS=tasks,filesystem, or allowSharedAgentTools: ['tasks', 'filesystem']. Without tasks, the proxy does not start.

To keep readers from sharing the file storage, turn it off in every project that the proxy serves. That setting applies to every user of the project, not only to the service account. The agent then cannot hand over a file that it wrote to its storage, but answers still carry the files that the code interpreter produces.

To turn off file storage, follow these steps:

  1. In the agent configuration of each project, pin capability.filesystem.enabled to false in the project manifest, project.yaml, and in every agent that sets it. For the setting, see the Preference Reference page.

  2. Deploy the agent configuration with the genai command-line tool. Outside Squirro, the tool is available on request: visit the Squirro Support website and submit a technical support request. For how to install the tool, see the Installation page.

  3. Set ALLOW_SHARED_AGENT_TOOLS=tasks, or allowSharedAgentTools: ['tasks'].

Without filesystem in ALLOW_SHARED_AGENT_TOOLS, the proxy does not start until capability.filesystem.enabled is pinned to false in the project manifest and in every agent that sets it. Its error names the project, the setting, and what is wrong.

If someone turns on memory or file storage later, without allowing it on the proxy, the proxy stops serving at its next check and responds to every request with 503 proxy_not_ready. Until then, readers share what was turned on, so do not change these settings while the proxy serves.

Proxy Environment Variables#

The templates read these environment variables. Without a required one, the proxy stops at startup with an error that names it:

Variable

Required

Description

SQUIRRO_API_URL

Yes

The origin of your Squirro instance, for example https://acme.squirro.cloud. Use the final https address, because the proxy does not follow a redirect. Plain http works only on a private network. For a private certificate authority, also set NODE_EXTRA_CA_CERTS to its certificate file.

SQUIRRO_REFRESH_TOKEN

Yes

The refresh token of the service account. See the Get Squirro Credentials for the Proxy section. Keep it on the server.

ALLOWED_ORIGINS

Yes

The comma-separated origins of the pages allowed to use the proxy, for example https://docs.example.com.

ANON_TOKEN_SECRET

Yes

A random secret of at least 32 characters that signs the anonymous tokens, for example from openssl rand -base64 32. Give every replica of one proxy the same secret, use a different one for each environment, such as staging and production, and keep it out of source control.

PROJECT_ID

Yes

The ID of the Squirro project that the chat uses, or several IDs, comma-separated. The proxy refuses every other project. The ID appears in the address of any Neo page of the project, after /neo/.

ALLOW_SHARED_AGENT_TOOLS

Yes

Set it to tasks,filesystem, or to tasks if file storage is turned off in every project that the proxy serves. See the Scheduled Tasks and File Storage section.

SQUIRRO_TENANT

With Workspaces

The tenant of the workspace that holds the project, used to open documents. Required when the Workspaces feature is turned on. See the Which Tenant to Set section.

ALLOW_SEARCH

No

Set to 1 to let readers search your documents with the search widget. See the What Readers Share page.

ALLOW_SHARED_MEMORY

No

Set to 1 only if every reader may share one memory. See the Memory section.

NUTRIENT_LICENSE_KEY

No

The license key of the PDF viewer. See the Serve the PDF Viewer section.

PORT

No

The port the proxy listens on. Defaults to 8787.

HOST

No

The address the proxy listens on. Defaults to 127.0.0.1, so that only a reverse proxy on the same machine reaches it.

TRUST_PROXY

No

The number of reverse proxies in front, so that the rate limit counts each reader separately. Defaults to 1 when HOST is a loopback address, and to 0 otherwise.

Which Tenant to Set#

To open documents, the proxy needs the tenant that holds the project, which appears in Squirro API paths. A tenant keeps the data of one organization separate from the others on the same Squirro instance. For more information, see the Authentication page.

  • Without the Workspaces feature

    The proxy reads the tenant from its own token exchange, so you set nothing.

  • With the Workspaces feature turned on

    Each workspace is a tenant of its own, and the service account signs in to a global tenant that holds no project. Set SQUIRRO_TENANT, or tenant on createChatProxy, to the tenant of the workspace that holds the project. Otherwise, every request for a document responds with 404. For more information about workspaces, see the Workspaces page.

Start from a Template#

The package ships two example proxies, called templates, in its templates/ folder:

  • proxy-node.mjs

    Runs on the HTTP server built into Node.js, and needs no other package.

  • proxy-express.mjs

    Runs in an Express app. Use it if your server already runs on Express, and install Express in the same server project with npm install express.

Each template runs unchanged in the setup that this guide describes. Copy one into your server project, set the environment variables, and run it:

cp node_modules/@squirro/neo-sdk/templates/proxy-node.mjs .
node proxy-node.mjs

To check that the proxy runs, look for this line in its output. The template prints it only after the startup check has passed:

neo chat proxy listening on http://127.0.0.1:8787

Then, on the same machine, run curl http://127.0.0.1:8787/healthz. A proxy that serves responds with status 200 and {"ok":true,...}. If the startup check fails, the template exits instead, with an error that names what to fix. For the errors, see the Troubleshooting page.

Both templates are built around one call:

import { anonymous, createChatProxy } from '@squirro/neo-sdk/server';

const allowedOrigins = process.env.ALLOWED_ORIGINS.split(',');

const { handler, stats, dispose, destroy, ready } = createChatProxy({
  squirroUrl: process.env.SQUIRRO_API_URL,
  refreshToken: process.env.SQUIRRO_REFRESH_TOKEN,
  tenant: process.env.SQUIRRO_TENANT || undefined,
  projectIds: process.env.PROJECT_ID.split(','),
  // tasks, or tasks,filesystem: what every reader shares with the service account.
  allowSharedAgentTools: process.env.ALLOW_SHARED_AGENT_TOOLS.split(','),
  allowSharedMemory: process.env.ALLOW_SHARED_MEMORY === '1',
  allowSearch: process.env.ALLOW_SEARCH === '1',
  pdfViewer: { licenseKey: process.env.NUTRIENT_LICENSE_KEY || undefined },
  mountPath: '/api/chat',
  // Only needed when your page and the proxy are on different origins.
  corsOrigins: allowedOrigins,
  // The proxy prints nothing on its own: this shows a weak secret or a missing X-Forwarded-For.
  onWarning: (message) => console.warn(message),
  authorize: anonymous({
    secret: process.env.ANON_TOKEN_SECRET,
    allowedOrigins,
    rateLimit: { windowMs: 60_000, max: 60 },
    // Behind your reverse proxy on this machine: count each reader by X-Forwarded-For.
    trustProxyHeader: true,
  }),
});

// The startup check: a refused refresh token, or memory that is on
// or cannot be checked, stops the proxy here.
await ready;

To adapt a template, for example for signed-in readers, edit its createChatProxy call. In a server of your own, wait for ready before it listens, as the templates do. Keep reading each setting from its environment variable. A value written into the code ignores the variable, so that changing ALLOW_SHARED_AGENT_TOOLS later, for example, has no effect. For every option and return value, see the docs/proxy.md file in the package.

In a container, set the stop timeout above 30 seconds, for example stop_grace_period: 35s in Docker Compose, so that answers that are still streaming can finish when the proxy stops.

Behind a Reverse Proxy#

A reverse proxy is the web server that receives the requests that browsers send to your site, such as nginx, Apache, Caddy, or the load balancer of your hosting provider. It handles HTTPS, and passes each request on to the program that responds to it. Do not confuse it with the chat proxy of the SDK: the reverse proxy sits in front of the chat proxy.

The chat proxy must run behind a reverse proxy. The templates speak plain HTTP, and a page served over HTTPS cannot call a plain HTTP address. By default, they also listen only on 127.0.0.1, the address of the machine itself, so browsers cannot reach them directly. The web server or load balancer that already serves your site usually fills that role, so you only add the settings below to it.

Your reverse proxy forwards the requests of the chat to the chat proxy, and adds HTTPS on the way. With nginx, a location block for the mount path of the chat does that:

location /api/chat/ {
    proxy_pass http://127.0.0.1:8787;
    proxy_set_header Host $host;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_buffering off;
    proxy_read_timeout 600s;
}

The directives do the following:

  • proxy_pass, without a path after the port, forwards the full request path, which the chat proxy expects.

  • Host $host lets the chat proxy recognize requests from your own pages. Without it, questions from a page whose origin is not in ALLOWED_ORIGINS fail with 403 cross_site_request.

  • X-Forwarded-For lets the rate limit count each reader separately.

  • proxy_buffering off and proxy_read_timeout 600s pass each answer on as it streams, and wait up to 10 minutes between two parts of an answer.

Other reverse proxies need the same settings under other names: forward the full path, pass on the host name and the address of the reader, and turn off response buffering.

The request paths carry conversation IDs, which give access to the conversations. Keep them out of the access logs of your reverse proxy, for example with access_log off; in that location block. See the What Readers Share page.

To check the reverse proxy, request a token through it from any machine, with the origin of your page:

curl -H 'Origin: https://docs.example.com' https://docs.example.com/api/chat/token

The response is a JSON object with a token. A 403 with origin_not_allowed means that the origin is missing from ALLOWED_ORIGINS. A 502 means that the reverse proxy cannot reach the chat proxy.

Decide Who May Chat#

authorize decides which requests the proxy lets through. Pick one of the following:

  • Anonymous tokens, for public sites

    With anonymous(...), the proxy issues short-lived tokens at GET /api/chat/token to pages on your allowed origins, and rate-limits each reader address. That keeps other websites from using your proxy. It does not identify anyone, though: a script can still call it, so size the rate limit deliberately.

    The 60 requests per minute of the templates allow about 20 page loads or questions per reader address per minute, because the limit counts every request of the chat, not only the token. Questions, searches, and downloads also have budgets of their own. For those limits, and for readers behind one corporate gateway, see the Troubleshooting page.

    Every question uses the language model of the project, and these limits apply per reader address, not in total. Monitor the usage after you go live.

  • Session tokens, for signed-in apps

    With bearerSession(verify), your page sends its own session token. verify checks it, and returns a user ID, or null to refuse it. Keep projectIds, because without it the proxy cannot run its startup check, and only warns:

    import { bearerSession, createChatProxy } from '@squirro/neo-sdk/server';

const { handler } = createChatProxy({
  squirroUrl: process.env.SQUIRRO_API_URL,
  refreshToken: process.env.SQUIRRO_REFRESH_TOKEN,
  projectIds: process.env.PROJECT_ID.split(','),
  allowSharedAgentTools: process.env.ALLOW_SHARED_AGENT_TOOLS.split(','),
  // The other options of the template, unchanged.
  authorize: bearerSession(async (token) => {
    const session = await mySessions.verify(token);
    return session ? session.userId : null;
  }),
});

Serve the PDF Viewer#

PDF documents open in the Nutrient viewer, which comes with the SDK, so there is nothing to install. Your proxy serves the files of the viewer, under /api/chat/assets/nutrient/ with the templates. That route is public and does not run your authorize. The files are the same ones that anyone can download from npm.

To turn the route off, pass serveNutrientAssets: false to createChatProxy. Do that if you do not show PDF documents, or if you host the viewer files somewhere else and set pdfViewer.assetsUrl on the page to that location.

Squirro supplies the Nutrient license key for your deployment, so you never need a key of your own. To request it, see the Obtaining a License Key section of the PDF Viewer page. Set it on the proxy as NUTRIENT_LICENSE_KEY.

The proxy hands the key to the browser, where anyone can read it. The Nutrient domain lock is what protects it, so the key must be issued for the domains that you embed the widgets on. On any other domain, the viewer shows an evaluation watermark.

Without NUTRIENT_LICENSE_KEY, the proxy uses the license key of the PDF viewer of your Squirro instance, and reports it once through onWarning. That key is licensed for the domain of your Squirro instance, so the viewer still shows the watermark on any domain that its license does not cover. To keep the proxy from using it, pass pdfViewer: { instanceLicenseKey: false } to createChatProxy. Without any key, the viewer shows the watermark.

Without a license key, the viewer also sends usage metrics from the browser of the reader to Nutrient, at https://dam.our.services.nutrient-powered.io/proto/metrics. Blocking that request does not affect the viewer. Do not count on a license key to stop that call.

If your site has a Content Security Policy, the viewer needs a few sources of its own. For more information, see the Content Security Policy page.

Contents
This page
View as Markdown