LanguageDetection

class squirro.lib.nlp.apps.query_processing.LanguageDetection(config)

Bases: squirro.lib.nlp.steps.batched_step.BatchedStep

Perform language detection on document.

Parameters
  • step (str, "app") – app

  • type (str, "query_processing") – query_processing

  • name (str, "lang_detection") – lang_detection

  • input_field (str,"user_terms_str") – query

  • output_field (str,"language") – detected language as ISO code

  • fallback_language (str, "en") – Default language to use

  • path (str, ".") – path

Attributes Summary

MODEL_CACHE_KEY

Methods Summary

process_doc(doc)

Process a document

Attributes Documentation

MODEL_CACHE_KEY = 'fasttext_language_detector'

Methods Documentation

process_doc(doc)

Process a document

Parameters

doc (Document) – Document

Returns

Processed document

Return type

Document