EmptyFilter#

class squirro.lib.nlp.steps.filters.EmptyFilter(config)#

The empty Filter removes a Document or marks a Document to skip based on empty fields

Input - No specific input type necessary as long as it returns True/False for if not field_name.

Output - No specific output format due to removal or marking to skip of the whole Document

Parameters

type (str) – empty
field (str, None) – Field to filter
fields (list, None) – Fields to filter
replacement_value (object, None) – Value with which to replace an empty field
skip (bool, True) – Whether or not to skip a doc with an empty field

Example

{
    "step": "filter",
    "type": "empty",
    "fields":["body"]
}

Methods Summary

Process a document

Methods Documentation

process_doc(doc)#

Process a document