make_tokenizer#

squirro.lib.nlp.steps.tokenizers.make_tokenizer(config)#

Tokenizer factory

Parameters

config (dict) – step config

Returns

created Tokenizer object

Return type

Tokenizer