SquirroQueryLoader

class squirro.lib.nlp.steps.loaders.SquirroQueryLoader(config)

Bases: squirro.lib.nlp.steps.loaders.Loader

Loads items from a Squirro query and converts them to libNLP Document.

Parameters
  • type (str) – squirro_query

  • batch_size (int, 1000) – size of Squirro query batch

  • project_id (str) – id of Squirro project

  • cluster (str) – Squirro cluster

  • token (str) – Squirro token

  • client_id (str, None) – Squirro client id

  • client_secret (str, None) – Squirro client secret

  • merge_sub_items (bool, False) – Load content from sub_items and return as merged field

Methods Summary

process(dataset)

Process a query and yield documents.

Methods Documentation

process(dataset)

Process a query and yield documents.

Arguments

dataset (dict): Dictionary containing query strings

Yields

Document – A document from a query