StopwordsNormalizer

class squirro.lib.nlp.steps.normalizers.StopwordsNormalizer(config)

Bases: squirro.lib.nlp.steps.normalizers.Normalizer

The stopwords Normalizer strips stopwords from the text

Input - all input fields need to be of type str.

Output - all output fields are filled with data of type str.

Parameters
  • type (str) – stopwords

  • stopwords (list, []) – List of stopwords

  • stopwords_file (str, None) – Path to the file with list of stopwords

Example

{
    "step": "normalizer",
    "type": "stopwords",
    "input_fields": ["body"],
    "output_fields": ["normalized_body"],
    "stopwords": ["the","stopwords"]
}

Methods Summary

load()

Load a step

process_doc(doc)

Process a document

save()

Save a step

Methods Documentation

load()

Load a step

process_doc(doc)

Process a document

Parameters

doc (Document) – Document

Returns

Processed document

Return type

Document

save()

Save a step