CSVLoader

CSVLoader#

class CSVLoader(config)#

Bases: Loader

The CSV Loader loads CSV files and transforms the data into Document items.

Parameters:

  • type (str) – csv

  • delimiter (str, ',') – delimiter between fields

  • encoding (str, 'utf-8') – given file’s encoding

  • quotechar (str, '"') – character used around values to quote them

  • replace_new_lines (bool, True) – indicates if replacement of new lines characters is applied

Example

{
    "step": "loader",
    "type": "csv",
    "fields": [ "text", "label" ]
}

Methods Summary

process(directory)

Process a directory of csv files and yield documents.

Methods Documentation

process(directory)#

Process a directory of csv files and yield documents.

Parameters:

directory (str) – path to directory with *.csv files

Returns:

Generator of Documents read from CSV

Return type:

generator(Document)

Contents