SquirroNERGroundtruthLoader

SquirroNERGroundtruthLoader#

class SquirroNERGroundtruthLoader(config)#

Bases: SquirroQueryLoader

Loads items from a Squirro instance using the SquirroClient and transforms the items into Document items.

Note - The fields client_id, client_secret, cluster, token and project_id do not need to be set if it is used inside the Squirro machinelearning service

Parameters:

  • batch_size (int, 1000) – size of Squirro query batch

  • client_id (str, None) – Squirro client id

  • client_secret (str, None) – Squirro client secret

  • cluster (str) – Squirro cluster

  • token (str) – Squirro token

  • project_id (str) – id of Squirro project

  • ner_groundtruth_id (str) – id of the Squirro Ground Truth

Example

{
    "step": "loader",
    "type": "squirro_ner_groundtruth",
    "batch_size": 100,
    "cluster": "CLUSTER",
    "token": "TOKEN",
    "project_id": "PROJECT_ID",
    "fields": ["body"],
    "ner_groundtruth_id": "NER_GROUNDTRUTH_ID",
}

Methods Summary

process(_)

Process a query and yield documents.

Methods Documentation

process(_)#

Process a query and yield documents.

Arguments

dataset (dict): Dictionary containing query strings

Yields:

Document – A document from a query

Contents