Model-as-a-Service#

This page provides an overview of creating, customizing, and prototyping machine learning (ML) models as part of Squirro’s Model-as-a-Service (MaaS).

Overview#

Model-as-a-Service (MaaS) allows Squirro customers to import custom ML models and speed up the prototyping phase for ML projects in Squirro.

Installation#

To use Maas, you must install the required packages from the Squirro mirror on your target Squirro server as follows:

yum install squirro-miniforge
yum install squirro-python38-mlflow

Prerequisites#

Before uploading a model, you must first create one.

For more information on MLFlow Models, see the official MLFlow Model site or an Example MLFlow Model.

Note: As an alternative to MaaS, you can create no-code models using the Squirro AI Studio. For a complete tutorial, see AI Studio.

Creating an MLFlow Model#

You can create an MLFlow Model in one of two ways:

train a MLFlow model on your local machine or on your exploration server
wrap an existing (pre-trained) model into the structure of a MLFlow Model and run it locally

Either way, MLFlow stores the (trained) model in the MLFlow base folder (mlruns/0/) with a unique hash (<HASH>) after executing the run-command.

The simplest structure of the MLFlow Model is as follows:

├── artifacts
│   └── model
│       ├── conda.yaml
│       ├── MLmodel
│       ├── python_model.pkl
│       └── requirements.txt
└── meta.yaml

MLFlow documentation#

See the (external) links below for information on creating MLFlow models.

Built-in Model Flavors (to write your own model)
Train a MLFlow Model
Packaging Training Code for MLFlow Model

Data Structure#

To use the MLFlow Model later in the context of a Squirro ML Workflow you need to stick to a specific data structure:

the input must be a pandas dataframe with an id and named feature fields as columns.
the output must be a pandas dataframe with an id and result fields as columns.

For more information about pandas, see the official pandas website.

Example:

input DataFrame

    id                         text
id0  this is a example sentence.
id1                 hello world.
id2             random sentence.
id3               test sentence.

output DataFrame

    id   class
id0  class1
id1  class0
id2  class0
id3  class1

Uploading a Model#

There are two ways to upload a MLFlow Model to Squirro:

Via squirro_asset CLI Reference (large models >500MB (exact number is under revision) can cause nginx issues → then use scp):
- go into the MLFlow base folder
- send the (trained) model via squirro_asset
```
squirro_asset -vvv mlflow_models upload -t $TOKEN -c $CLUSTER -f mlruns/0/<HASH>/
```

Via scp:

go into the MLFlow base folder

verify that the destination directory exists (on the Squirro server)

<BASE_DIR>=/var/lib/squirro/topic/assets/mlflow_models # default path
mkdir -p <BASE_DIR>/mlruns/0

compress the directory with the (trained) model (wherever you have trained your model)
```
cd mlruns/0/ && tar -czvf trained_model.tar.gz <HASH>/
```
send it to the MLFlow base folder on the Squirro server
```
scp trained_model.zip <SQUIRRO_SERVER_URL>:/tmp/
```

ssh into the Squirro server and unzip the sent file

cd <BASE_DIR>/mlruns/0 &&
mv /tmp/trained_model.tar.gz <BASE_DIR>/mlruns/0/  #create the dirs if not existing
tar -xzvf trained_model.tar.gz

adjust artifact_uri in the meta.yaml with the new path of the MLFlow Model (file:///<BASE_DIR>/mlruns/0/<HASH>/artifacts)
```
sed -i '/artifact_uri/c\artifact_uri: file:///<BASE_DIR>/mlruns/0/<HASH>/artifacts' <HASH>/meta.yaml
```

Starting of Service#

To start a Model-as-a-Service follow the steps below:

make sure you are in the MLFlow base folder on the Squirro server
activate the Squirro environment
```
squirro_activate3
```
serve the model identified by the <HASH> as a service listening to the chosen port <PORT>
```
mlflow models serve -m runs:/<HASH>/model -p <PORT>
```
- use nohup or screen when starting the service so the MaaS does not stop when you terminate your ssh session

Note

there is no service orchestration provided at this stage
keep an eye on memory and storage consumption. Then among others:
- a started model service loads the model in memory and keeps it there
- there is a new conda environment created for every new model which has a different conda.yaml file
on-premise customers need to manually package their conda environment. This can be done as explained here.

Using MaaS#

To use your model, you must first create a ML Workflow.

Your ML Workflow can then be used as an inference ML Job scheduled in an interval or as a published model in the enrich pipeline.

For more information on publishing models, see How To Publish ML Models Using the Squirro Client.

Example Workflows#

Below are document-level and sentence-level examples of ML Workflows.

Document Level

{
    "dataset": {
        "infer": {
            "count": 10,
            "query_string": "language:en"
        }
    },
    "pipeline": [
        {
            "fields": [
                "body"
            ],
            "step": "loader",
            "type": "squirro_query"
        },
        {
            "fields": [
                "body"
            ],
            "step": "filter",
            "type": "empty"
        },
        {
            "input_mapping": {
                "body":"text"
            },
            "output_mapping": {
                "class":"keywords.prediction"
            },
            "process_endpoint": "http://localhost:<PORT>/invocations",
            "name": "mlflow_maas",
            "step": "mlflow_maas",
            "type": "mlflow_maas"
        },
        {
            "fields": [
                "keywords.prediction"
            ],
            "step": "saver",
            "type": "squirro_item"
        }
    ]
}

Sentence Level (With Entity Generation)

{
    "dataset": {
        "infer": {
            "count": 10,
            "query_string": "language:en"
        }
    },
    "pipeline": [
        {
            "fields": [
                "body"
            ],
            "step": "loader",
            "type": "squirro_query"
        },
        {
            "fields": [
                "body"
            ],
            "step": "filter",
            "type": "empty"
        },
        {
            "input_fields": [
                "body"
            ],
            "output_fields": [
                "extract_sentences"
            ],
            "step": "tokenizer",
            "type": "sentences_nltk"
        },
        {
            "fields": [
                "extract_sentences"
            ],
            "step": "filter",
            "type": "doc_split"
        },
        {
            "input_mapping": {
                "extract_sentences":"text"
            },
            "output_mapping": {
                "class":"prediction"
            },
            "process_endpoint": "http://localhost:<PORT>/invocations",
            "name": "mlflow_maas",
            "step": "mlflow_maas",
            "type": "mlflow_maas"
        },
        {
            "fields": [
                "extract_sentences",
                "prediction"
            ],
            "step": "filter",
            "type": "doc_join"
        },
        {
            "entity_name_field": "Catalyst",
            "entity_type": "Catalyst",
            "excluded_values": [],
            "extract_field": "extract_sentences",
            "format_values": false,
            "global_property_field_map": {},
            "modes": [
                "process"
            ],
            "property_field_map": {
                "Catalyst": [
                    "prediction"
                ]
            },
            "required_properties": [
                "Catalyst"
            ],
            "source_field": "body",
            "step": "filter",
            "type": "squirro_entity"
        },
        {
            "fields": [
                "entities"
            ],
            "step": "saver",
            "type": "squirro_item"
        }
    ]
}

Model-as-a-Service

Contents