# How To Publish ML Models Using the Squirro Client Besides publishing models directly from the AI Studio or importing published models through project import, you can also use the _SquirroClient_ to publish ML models to the [Data Processing Pipeline](../data-processing/index.md#data-processing). This guide explains - How to publish a new ML model by submitting a complete ML workflow configuration. - How to publish an already existing ML workflow as a model to the Squirro pipeline. Reference: To learn more about the _SquirroClient_, see [SquirroClient (Python SDK)](../api/squirro_client/index.md#squirro-client). ## Publish a New ML Model Using Python, connect and authenticate with the `SquirroClient`: ```python from squirro_client import SquirroClient cluster = '' project_id='' token = '' client = SquirroClient(None, None, cluster=cluster) client.authenticate(refresh_token=token) ``` Define the workflow configuration as shown below: ```python config = {'dataset': {}, 'pipeline': [{'fields': ['body'], 'step': 'loader', 'type': 'squirro_query'}, {'fields': ['body'], 'mark_as_skipped': True, 'step': 'filter', 'type': 'empty'}, {'cleaning': {'approx.': 'approx', 'etc.': 'etc', 'i.e.': 'ie'}, 'input_fields': ['body'], 'output_fields': ['extract_sentences'], 'rules': ['**', '...', '…', ': '], 'step': 'tokenizer', 'type': 'sentences_nltk'}, {'fields': ['extract_sentences'], 'step': 'filter', 'type': 'doc_split'}, {'input_fields': ['extract_sentences'], 'output_fields': ['extract_sentences'], 'step': 'tokenizer', 'type': 'html'}, {'fields': ['extract_sentences'], 'step': 'filter', 'type': 'doc_split'}, {'input_fields': ['extract_sentences'], 'output_fields': ['sentences_normalized'], 'step': 'normalizer', 'type': 'html'}, {'fields': ['sentences_normalized'], 'mark_as_skipped': True, 'step': 'filter', 'type': 'regex', 'whitelist_regexes': ['^.{20,}$']}, {'blacklist_terms': [], 'fields': ['sentences_normalized'], 'matching_label': 'tax_rate1', 'name': './models/ais/proximity', 'non_matching_label': 'not_tax_rate1_tax_rate2', 'output_field': 'prediction_tax_rate1', 'step': 'filter', 'type': 'proximity', 'whitelist_terms': ['tax rate of~1|','tax rate~2|']}, {'blacklist_terms': [], 'fields': ['sentences_normalized'], 'matching_label': 'tax_rate2', 'name': './models/ais/proximity', 'non_matching_label': 'not_tax_rate1_tax_rate2', 'output_field': 'prediction_tax_rate2', 'step': 'filter', 'type': 'proximity', 'whitelist_terms': ['tax rate~4|']}, {'delimiter': ',', 'input_fields': ['prediction_tax_rate1', 'prediction_tax_rate2'], 'output_field': 'prediction', 'step': 'filter', 'type': 'merge'}, {'input_field': 'prediction', 'output_field': 'prediction', 'step': 'filter', 'type': 'split'}, {'fields': ['sentences_normalized', 'prediction'], 'step': 'filter', 'type': 'doc_join'}, {'entity_name_field': 'Catalyst', 'entity_type': 'Catalyst', 'excluded_values': ['not_tax_rate1_tax_rate2'], 'extract_field': 'sentences_normalized', 'format_values': False, 'global_property_field_map': {}, 'modes': ['process'], 'property_field_map': {'Catalyst': ['prediction']}, 'required_properties': ['Catalyst'], 'source_field': 'body', 'step': 'filter', 'type': 'squirro_entity'} ] } ``` Publish the model using `client.ml_publish_model`. Below is an example of how the command might look like for the above workflow `config`: ```python client.ml_publish_model(project_id,\ published_as='Proximity Model Tax Rate',\ description='Proximity Model for Tax Rate v1',\ external_model=True,\ global_id='',\ location='',\ labels=['tax_rate1','tax_rate2','not_tax_rate1_tax_rate2'],\ tagging_level='sentence',\ workflow_name='[PUB] prox config import',\ workflow_config=config) ``` ## Publish an Existing ML Workflow To publish an existing ML Workflow, retrieve its ID from _ML Workflows_ under the _AI STUDIO_ tab: [](https://s3.amazonaws.com/download.squirro.net/docs/migrated-attachments/2682290819/2682814501.png) Publish the model using `client.ml_publish_model` and submitting the `workflow_id`: ```python client.ml_publish_model(project_id,\ published_as='Proximity Model Tax Rate',\ description='Proximity Model for Tax Rate v1',\ external_model=True,\ global_id='',\ location='',\ labels=['tax_rate1','tax_rate2','not_tax_rate1_tax_rate2'],\ tagging_level='sentence',\ workflow_id='VLGRAEbLRZ2v5Uq_MPt77w') ``` ## Notes For document-level tagging you must provide the `keywords.prediction` field in the `output_fields` to store the predictions in a keyword. See the example below: ```python { "dataset": { "infer": { "count": 10000, "query_string": "language:en" } }, "pipeline": [ { "fields": [ "body" ], "step": "loader", "type": "squirro_query" }, { "fields": [ "body" ], "step": "filter", "type": "empty" }, { "input_fields": [ "body" ], "output_fields": [ "clean_body" ], "step": "normalizer", "type": "html" }, { "input_fields": [ "clean_body" ], "output_fields": [ "extract_sentences" ], "step": "tokenizer", "type": "sentences_nltk" }, { "fields": [ "extract_sentences" ], "step": "filter", "type": "doc_split" }, { "input_fields": [ "extract_sentences" ], "label_field": "", "output_field": "prediction", "step": "classifier", "type": "vadersentiment" }, { "fields": [ "extract_sentences", "prediction" ], "step": "filter", "type": "doc_join" }, { "input_fields": [ "prediction" ], "output_fields": [ "keywords.prediction" ], "step": "filter", "type": "vote" }, { "fields": [ "keywords.prediction" ], "step": "saver", "type": "squirro_item" } ] } ``` You can then assign the keyword in which the predictions are stored from the _Label_ dropdown in the [Pipeline Editor](../data-processing/pipeline-editor.md#data-processing-pipeline-editor) when editing the published model step, as shown below: [](https://s3.amazonaws.com/download.squirro.net/docs/migrated-attachments/2682290819/2682749031.png)