Container Reference

Container Reference#

Volumes#

The following volumes are used by the Squirro container images to persist data.

Docker Compose Permanent Volume Configuration#

In an AWS based deployment it is recommended to use EFS for the shared data volume. The relevant bits are the target of /mnt/squirro and the uid and gid of 44040.

volumes:
    - squirro_shared_data:/mnt/squirro:uid=44040,gid=44040

Docker Compose Temporary Data Volume Configuration#

The temporary data volume is used to store temporary data that is not required to be persisted. Because the containers’ main filesystem is read-only, this volume is critical for the operation of the container.

- /tmp:uid=44040,gid=44040
- /run:uid=44040,gid=44040

Security#

The Squirro container images are designed to be secure by default. The following security features are implemented:

The container is scanned for vulnerabilities using Trivy.
The container is based on Amazon Linux 2023 which is a secure and hardened base image.
The container runs as a non-privileged user with user id 44040.
The container is configured to run in read-only mode.
The container is configured to run with a minimal set of capabilities.
The container is configured to drop all capabilities.
Secrets are not stored in the container image.
Environment variables are used to configure the container.
Secrets are provided as environment variables or secret files.

Example docker configuration#

For reference here is an example docker-compose file with the security related configuration. We recommend to always use the config file generation feature on the Squirro Container Registry to generate the docker-compose file.

services:
    squirro:
        restart: always
        image: containers.squirro.com/squirro/minimal:amd64-unstable-latest
        ports:
            - "8080:8080"
            - "8443:8443"
        secrets:
            - redis_password
            - db_password
            - elasticsearch_password
            - flask_secret_key
            - client_id
            - client_secret
        environment:
            # Elasticsearch
            - SQ_ES_INDEX_SERVERS=http://elasticsearch:9200
            - SQ_ES_AUTH_USER=elastic
            - SQ_ES_AUTH_PASSWORD=secretsfile:/run/secrets/elasticsearch_password
            # Redis
            - SQ_REDIS_QUEUE_PASSWORD=secretsfile:/run/secrets/redis_password
            - SQ_REDIS_CACHE_PASSWORD=secretsfile:/run/secrets/redis_password
            # Database
            - SQ_SQL_PASSWORD=secretsfile:/run/secrets/db_password
            # Frontend
            - SQ_FLASK_SECRET_KEY=secretsfile:/run/secrets/flask_secret_key
            # User Service Client Secrets
            - SQ_CLIENT_ID=secretsfile:/run/secrets/client_id
            - SQ_CLIENT_SECRET=secretsfile:/run/secrets/client_secret
            # Log Level (DEBUG, INFO, WARNING, ERROR, CRITICAL)
            - SQ_LOG_LEVEL=INFO
        user: "44040:44040"
        security_opt:
            - no-new-privileges:true
        read_only: true
        tmpfs:
            - /tmp:uid=44040,gid=44040
            - /run:uid=44040,gid=44040
        volumes:
            - squirro_shared_data:/mnt/squirro:uid=44040,gid=44040
        deploy:
        resources:
            limits:
                cpus: "4"
                memory: 10G
            reservations:
                cpus: "4.0"
                memory: 10G
        networks:
            - default
        depends_on:
            db:
                condition: service_healthy
            redis-queue:
                condition: service_started
            redis-cache:
                condition: service_started
            elasticsearch:
                condition: service_started
    db:
        ...
    redis-queue:
        ...
    redis-cache:
        ...
    elasticsearch:
        ...
volumes:
    squirro_shared_data:
networks:
    default:
        driver: bridge
secrets:
    mirror_user:
        file: ./secrets/mirror_user
    mirror_password:
        file: ./secrets/mirror_password
    redis_password:
        file: ./secrets/redis_password
    db_password:
        file: ./secrets/db_password
    elasticsearch_password:
        file: ./secrets/elasticsearch_password
    flask_secret_key:
        file: ./secrets/flask_secret_key
    client_id:
        file: ./secrets/client_id
    client_secret:
        file: ./secrets/client_secret

Environment Variables#

The following variables can be set to control the behavior of the Squirro container images. At the time of writing, this applies only to the squirro-minimal image.

The plan is that, where applicable, the same variables will be used in all Squirro future container images. (e.g. dedicated user service with same Logging, Database and Redis variables and secrets)

SSL / TLS Configuration#

Variable

Type

Default

Description

SQ_CA_FILE

file path

Provide a custom CA file to be used by the container. This is useful if you have a custom CA file that is not part of the default CA trust store.

The provided file will be prepended to the default CA trust store of Amazon Linux 2023. Hence, this is only needed if the container needs to trust additional CA certificates / self-signed certificates.

Elasticsearch Configuration#

Variable	Type	Default	Description
SQ_ES_INDEX_SERVERS	string	http://elasticsearch:9200	The URL to the Elasticsearch server. This can be a single URL or a comma separated list of URLs. e.g. http://elasticsearch:9200,http://elasticsearch2:9200
SQ_ES_AUTH_USER	string	elastic	The username to authenticate with the Elasticsearch server.
SQ_ES_AUTH_PASSWORD	string		The password to authenticate with the Elasticsearch server. Recommened to use a secret file for this value. e.g secretsfile:/run/secrets/elasticsearch_password

Database Configuration#

We currently only support PostgreSQL as the database backend for squirro-minimal. Support for MariadB/MySQL will be added in the near future.

Variable	Type	Default	Description
SQ_SQL_PROTOCOL	string	postgresql	Which database protocol to use. Currently only postgresql is supported. Equates to the sqlalchemy dialect.
SQ_SQL_HOST	string	db	The hostname of the database server. This can be the hostname of the database server or the IP address, for AWS RDS, this is the endpoint, e.g mydbinstance.123456789012.us-east-1.rds.amazonaws.com,
SQ_SQL_PORT	integer	5432	The TCP port of the database server.
SQ_SQL_USER	string	postgres	Username to connect to the database server.
SQ_SQL_PASSWORD	string		Password to connect to the database server. Recommened to use a secret file for this value. e.g secretsfile:/run/secrets/db_password
SQ_SQL_CONFIGURATION_DATABASE, SQ_SQL_DATASOURCE_DATABASE, SQ_SQL_EMAILSENDER_DATABASE, SQ_SQL_FILTERING_DATABASE, SQ_SQL_FINGERPRINT_DATABASE, SQ_SQL_MACHINELEARNING_DATABASE, SQ_SQL_NOTES_DATABASE, SQ_SQL_PLUMBER_DATABASE, SQ_SQL_SCHEDULER_DATABASE, SQ_SQL_STUDIO_DATABASE, SQ_SQL_TOPIC_DATABASE, SQ_SQL_USER_DATABASE	string	configuration, datasource, emailsender, filtering, fingerprint, machinelearning, notes, plumber, scheduler, studio, topic, squser	The name of the database to use for each service. You are responsible to create the databases.
SQ_SQL_CONFIGURATION_SERVICE_URI, SQ_SQL_DATASOURCE_SERVICE_URI, SQ_SQL_EMAILSENDER_SERVICE_URI, SQ_SQL_FILTERING_SERVICE_URI, SQ_SQL_FINGERPRINT_SERVICE_URI, SQ_SQL_MACHINELEARNING_SERVICE_URI, SQ_SQL_NOTES_SERVICE_URI, SQ_SQL_PLUMBER_SERVICE_URI, SQ_SQL_SCHEDULER_SERVICE_URI, SQ_SQL_STUDIO_SERVICE_URI, SQ_SQL_TOPIC_SERVICE_URI, SQ_SQL_USER_SERVICE_URI	string		For custom deployment, you can provide the full URI to the database server. e.g. postgresql://user:password@hostname:port/database?customparam=value

Redis Configuration#

Squirro requires two instances of Redis, one for the cache and one for persistent storage and queues.

Redis Queue

Variable	Type	Default	Description
SQ_REDIS_QUEUE_HOST	string	redis-queue	The hostname of the Redis server. For AWS ElastiCache, this is the endpoint. e.g. myredisinstance.123456789012.us-east-1.cache.amazonaws.com
SQ_REDIS_QUEUE_PORT	integer	6379	The TCP port of the Redis server.
SQ_REDIS_QUEUE_SSL	boolean	false	Enable SSL for the connection to the Redis server.
SQ_REDIS_QUEUE_SSL_VERIFY	boolean	false	Verify the SSL certificate of the Redis server.