EmptyFilter

class squirro.lib.nlp.steps.filters.EmptyFilter(config)

Bases: squirro.lib.nlp.steps.filters.Filter

The empty Filter removes a Document or marks a Document to skip based on empty fields

Input - No specific input type necessary as long as it returns True/False for if not field_name.

Output - No specific output format due to removal or marking to skip of the whole Document

Parameters
  • type (str) – empty

  • field (str, None) – Field to filter

  • fields (list, None) – Fields to filter

  • replacement_value (object, None) – Value with which to replace an empty field

  • skip (bool, True) – Whether or not to skip a doc with an empty field

Example

{
    "step": "filter",
    "type": "empty",
    "fields":["body"]
}

Methods Summary

process_doc(doc)

Process a document

Methods Documentation

process_doc(doc)

Process a document

Parameters

doc (Document) – Document

Returns

Processed document

Return type

Document