Container Reference#

Volumes#

The following volumes are used by the Squirro container images to persist data.

Docker Compose Permanent Volume Configuration#

In an AWS based deployment it is recommended to use EFS for the shared data volume. The relevant bits are the target of /mnt/squirro and the uid and gid of 44040.

volumes:
    - squirro_shared_data:/mnt/squirro:uid=44040,gid=44040

Docker Compose Temporary Data Volume Configuration#

The temporary data volume is used to store temporary data that is not required to be persisted. Because the containers’ main filesystem is read-only, this volume is critical for the operation of the container.

- /tmp:uid=44040,gid=44040
- /run:uid=44040,gid=44040

Security#

The Squirro container images are designed to be secure by default. The following security features are implemented:

  • The container is scanned for vulnerabilities using Trivy.

  • The container is based on Amazon Linux 2023 which is a secure and hardened base image.

  • The container runs as a non-privileged user with user id 44040.

  • The container is configured to run in read-only mode.

  • The container is configured to run with a minimal set of capabilities.

  • The container is configured to drop all capabilities.

  • Secrets are not stored in the container image.

  • Environment variables are used to configure the container.

  • Secrets are provided as environment variables or secret files.

Example docker configuration#

For reference here is an example docker-compose file with the security related configuration. We recommend to always use the config file generation feature on the Squirro Container Registry to generate the docker-compose file.

services:
    squirro:
        restart: always
        image: containers.squirro.com/squirro/minimal:amd64-unstable-latest
        ports:
            - "8080:8080"
            - "8443:8443"
        secrets:
            - redis_password
            - db_password
            - elasticsearch_password
            - flask_secret_key
            - client_id
            - client_secret
        environment:
            # Elasticsearch
            - SQ_ES_INDEX_SERVERS=http://elasticsearch:9200
            - SQ_ES_AUTH_USER=elastic
            - SQ_ES_AUTH_PASSWORD=secretsfile:/run/secrets/elasticsearch_password
            # Redis
            - SQ_REDIS_QUEUE_PASSWORD=secretsfile:/run/secrets/redis_password
            - SQ_REDIS_CACHE_PASSWORD=secretsfile:/run/secrets/redis_password
            # Database
            - SQ_SQL_PASSWORD=secretsfile:/run/secrets/db_password
            # Frontend
            - SQ_FLASK_SECRET_KEY=secretsfile:/run/secrets/flask_secret_key
            # User Service Client Secrets
            - SQ_CLIENT_ID=secretsfile:/run/secrets/client_id
            - SQ_CLIENT_SECRET=secretsfile:/run/secrets/client_secret
            # Log Level (DEBUG, INFO, WARNING, ERROR, CRITICAL)
            - SQ_LOG_LEVEL=INFO
        user: "44040:44040"
        security_opt:
            - no-new-privileges:true
        read_only: true
        tmpfs:
            - /tmp:uid=44040,gid=44040
            - /run:uid=44040,gid=44040
        volumes:
            - squirro_shared_data:/mnt/squirro:uid=44040,gid=44040
        deploy:
        resources:
            limits:
                cpus: "4"
                memory: 10G
            reservations:
                cpus: "4.0"
                memory: 10G
        networks:
            - default
        depends_on:
            db:
                condition: service_healthy
            redis-queue:
                condition: service_started
            redis-cache:
                condition: service_started
            elasticsearch:
                condition: service_started
    db:
        ...
    redis-queue:
        ...
    redis-cache:
        ...
    elasticsearch:
        ...
volumes:
    squirro_shared_data:
networks:
    default:
        driver: bridge
secrets:
    mirror_user:
        file: ./secrets/mirror_user
    mirror_password:
        file: ./secrets/mirror_password
    redis_password:
        file: ./secrets/redis_password
    db_password:
        file: ./secrets/db_password
    elasticsearch_password:
        file: ./secrets/elasticsearch_password
    flask_secret_key:
        file: ./secrets/flask_secret_key
    client_id:
        file: ./secrets/client_id
    client_secret:
        file: ./secrets/client_secret

Environment Variables#

The following variables can be set to control the behavior of the Squirro container images. At the time of writing, this applies only to the squirro-minimal image.

The plan is that, where applicable, the same variables will be used in all Squirro future container images. (e.g. dedicated user service with same Logging, Database and Redis variables and secrets)

SSL / TLS Configuration#

Variable

Type

Default

Description

SQ_CA_FILE

file path

Provide a custom CA file to be used by the container. This is useful if you have a custom CA file that is not part of the default CA trust store.

The provided file will be prepended to the default CA trust store of Amazon Linux 2023. Hence, this is only needed if the container needs to trust additional CA certificates / self-signed certificates.

Elasticsearch Configuration#

Variable

Type

Default

Description

SQ_ES_INDEX_SERVERS

string

http://elasticsearch:9200

The URL to the Elasticsearch server. This can be a single URL or a comma separated list of URLs. e.g. http://elasticsearch:9200,http://elasticsearch2:9200

SQ_ES_AUTH_USER

string

elastic

The username to authenticate with the Elasticsearch server.

SQ_ES_AUTH_PASSWORD

string

The password to authenticate with the Elasticsearch server. Recommened to use a secret file for this value. e.g secretsfile:/run/secrets/elasticsearch_password

Database Configuration#

We currently only support PostgreSQL as the database backend for squirro-minimal. Support for MariadB/MySQL will be added in the near future.

Variable

Type

Default

Description

SQ_SQL_PROTOCOL

string

postgresql

Which database protocol to use. Currently only postgresql is supported. Equates to the sqlalchemy dialect.

SQ_SQL_HOST

string

db

The hostname of the database server. This can be the hostname of the database server or the IP address, for AWS RDS, this is the endpoint, e.g mydbinstance.123456789012.us-east-1.rds.amazonaws.com,

SQ_SQL_PORT

integer

5432

The TCP port of the database server.

SQ_SQL_USER

string

postgres

Username to connect to the database server.

SQ_SQL_PASSWORD

string

Password to connect to the database server. Recommened to use a secret file for this value. e.g secretsfile:/run/secrets/db_password

SQ_SQL_CONFIGURATION_DATABASE, SQ_SQL_DATASOURCE_DATABASE, SQ_SQL_EMAILSENDER_DATABASE, SQ_SQL_FILTERING_DATABASE, SQ_SQL_FINGERPRINT_DATABASE, SQ_SQL_MACHINELEARNING_DATABASE, SQ_SQL_NOTES_DATABASE, SQ_SQL_PLUMBER_DATABASE, SQ_SQL_SCHEDULER_DATABASE, SQ_SQL_STUDIO_DATABASE, SQ_SQL_TOPIC_DATABASE, SQ_SQL_USER_DATABASE

string

configuration, datasource, emailsender, filtering, fingerprint, machinelearning, notes, plumber, scheduler, studio, topic, squser

The name of the database to use for each service. You are responsible to create the databases.

SQ_SQL_CONFIGURATION_SERVICE_URI, SQ_SQL_DATASOURCE_SERVICE_URI, SQ_SQL_EMAILSENDER_SERVICE_URI, SQ_SQL_FILTERING_SERVICE_URI, SQ_SQL_FINGERPRINT_SERVICE_URI, SQ_SQL_MACHINELEARNING_SERVICE_URI, SQ_SQL_NOTES_SERVICE_URI, SQ_SQL_PLUMBER_SERVICE_URI, SQ_SQL_SCHEDULER_SERVICE_URI, SQ_SQL_STUDIO_SERVICE_URI, SQ_SQL_TOPIC_SERVICE_URI, SQ_SQL_USER_SERVICE_URI

string

For custom deployment, you can provide the full URI to the database server. e.g. postgresql://user:password@hostname:port/database?customparam=value

Redis Configuration#

Squirro requires two instances of Redis, one for the cache and one for persistent storage and queues.

Redis Queue

Variable

Type

Default

Description

SQ_REDIS_QUEUE_HOST

string

redis-queue

The hostname of the Redis server. For AWS ElastiCache, this is the endpoint. e.g. myredisinstance.123456789012.us-east-1.cache.amazonaws.com

SQ_REDIS_QUEUE_PORT

integer

6379

The TCP port of the Redis server.

SQ_REDIS_QUEUE_SSL

boolean

false

Enable SSL for the connection to the Redis server.

SQ_REDIS_QUEUE_SSL_VERIFY

boolean

false

Verify the SSL certificate of the Redis server.

Redis Queue

Variable

Type

Default

Description

SQ_REDIS_CACHE_HOST

string

redis-cache

The hostname of the Redis server. For AWS ElastiCache, this is the endpoint. e.g. myredisinstance.123456789012.us-east-1.cache.amazonaws.com

SQ_REDIS_CACHE_PORT

integer

6380

The TCP port of the Redis server.

SQ_REDIS_CACHE_SSL

boolean

false

Enable SSL for the connection to the Redis server.

SQ_REDIS_CACHE_SSL_VERIFY

boolean

false

Verify the SSL certificate of the Redis server.

Secrets and Client IDs#

Squirro requires a few secrets to be provided as environment variables or secret files.

Variable

Type

Default

Description

SQ_FLASK_SECRET_KEY

string or secretsfile

The secret key to use for the Flask application. This is used to sign cookies and other sensitive data. Recommened to use a secret file for this value. e.g secretsfile:/run/secrets/flask_secret_key

SQ_CLIENT_ID

string or secretsfile

The client id to use for various Service Clients. Recommened to use a secret file for this value. e.g secretsfile:/run/secrets/client_id

SQ_CLIENT_SECRET

string or secretsfile

The client secret to use for various Service Clients. Recommened to use a secret file for this value. e.g secretsfile:/run/secrets/client_id

Logging Configuration#

Variable

Type

Default

Description

SQ_LOG_LEVEL

string

INFO

Log Level (DEBUG, INFO, WARNING, ERROR, CRITICAL)