DocJoinFilter#
- class DocJoinFilter(config)#
Bases:
Filter
The doc_join
Filter
joins multipleDocument
into oneDocument
via id or a specified field.Note - This step assumes the fields arrive in a pre-sorted state, so that when the value of the join_field (if specified) changes, a new
Document
is created. Fields from subsequent Documents with the same join_field will be appended until the join_field changes again.Input - No specific input fields type required.
Output - The output field type is going to be of type
list
[ input field type ].- Parameters
Example
{ "step": "filter", "type": "doc_join", "fields":["sentences"] }
Methods Summary
process
(docs)Process a set of documents
train
(docs)Train on a step of a set of documents
Methods Documentation
- process(docs)#
Process a set of documents