SklearnTFIDFEmbedder

class squirro.lib.nlp.steps.embedders.SklearnTFIDFEmbedder(config)

Bases: squirro.lib.nlp.steps.embedders.Embedder

Sklearn TFIDF-Vectorizer See https://scikit-learn.org/stable/modules/generated/sklearn.feature_extraction.text.TfidfVectorizer.html

Parameters
  • type (str) – sklearn_tfidf

  • model_kwargs (dict, {}) – Keyword arguments to pass on to sklearn TfidfVectorizer

Methods Summary

load()

Load a step

process_batch(batch)

Process a batch of documents.

save()

Save a step

train(docs)

Train on a step of a set of documents

Methods Documentation

load()

Load a step

process_batch(batch)

Process a batch of documents. If not defined will default to using self.process_doc for each document in the batch.

Parameters

batch (list(Document)) – List of documents

Returns

List of processed documents

Return type

list(Document)

save()

Save a step

train(docs)

Train on a step of a set of documents

Parameters

docs (generator(Document)) – Generator of documents

Returns

Generator of processed documents

Return type

generator(Document)