EmptyFilter

EmptyFilter#

class EmptyFilter(config)#

Bases: Filter

The empty Filter removes a Document or marks a Document to skip based on empty fields

Input - No specific input type necessary as long as it returns True/False for if not field_name.

Output - No specific output format due to removal or marking to skip of the whole Document

Parameters:

  • type (str) – empty

  • field (str, None) – Field to filter, this option is here to support older scripts, as removing it would break them

  • fields (list, None) – Fields to filter

  • replacement_value (object, None) – Value with which to replace an empty field

  • skip (bool, True) – Whether or not to skip a doc with an empty field

Example

{
    "step": "filter",
    "type": "empty",
    "fields":["body"]
}

Methods Summary

process_doc(doc)

Process a document

Methods Documentation

process_doc(doc)#

Process a document

Parameters:

doc (Document) – Document

Returns:

Processed document

Return type:

Document

Contents