CSVLoader

class squirro.lib.nlp.steps.loaders.CSVLoader(config)

Bases: squirro.lib.nlp.steps.loaders.Loader

The CSV Loader loads CSV files and transforms the data into Document items.

Parameters
  • type (str) – csv

  • delimiter (str, ',') – delimiter between fields

  • encoding (str, 'utf-8') – given file’s encoding

  • quotechar (str, '"') – character used around values to quote them

  • replace_new_lines (bool, True) – indicates if replacement of new lines characters is applied

Example

{
    "step": "loader",
    "type": "csv",
    "fields": [ "text", "label" ]
}

Methods Summary

process(directory)

Process a directory of csv files and yield documents.

Methods Documentation

process(directory)

Process a directory of csv files and yield documents.

Parameters

directory (str) – path to directory with *.csv files

Returns

Generator of Documents read from CSV

Return type

generator(Document)