<!-- Source: https://docs.squirro.com/en/latest/technical/data-processing/index.html -->
# Data Processing Pipeline

This section outlines the various ways data is enriched, related, classified, indexed, and otherwise processed as part of the data processing pipeline.

Throughout these docs, you’ll see the data processing pipeline referred to simply as **the pipeline**.

## Overview

For an overview of the data processing pipeline architecture, sections, and processing, see the [Pipeline Overview](pipeline.md#data-processing-pipeline-overview).

## Pipeline Steps

The data processing pipeline is a collection of built-in steps, with the option to create custom steps.

For information on built-in steps, see [Built-In Steps](pipeline-steps/index.md#data-processing-pipeline-steps).

For information on custom steps, see [Custom Steps](custom/index.md#pipeline-steps-custom).

## Pipeline Editor

The Pipeline Editor allows you to configure pipeline steps directly within the Squirro user interface.

To learn more, see [Pipeline Editor](pipeline-editor.md#data-processing-pipeline-editor).

## Pipeline Reruns

There are situations where the pipeline will need to be rerun. This includes rerunning pipeline workflows, individual pipeline steps, and specific enrichments.

See [Pipeline Reruns](pipeline-rerun.md#data-processing-pipeline-rerun) for more information.

## Pipeline Prioritization

Squirro supports prioritization for certain data sources, which can be configured within the user interface.

To learn more, see [Pipeline Prioritization](priorities.md#data-processing-pipeline-priorities).

## Content Streamers

In Squirro, a content streamer is a component that provides the functionality for managing the `inputstream`.

Content streamers are used by any Squirro service (e.g., `api.provider`, `ingester`) that needs to interact with the `inputstream`.

For more information on this topic, see [Content Streamers](content-streamers.md#data-processing-content-streamers).

## Indexing Common Document Formats

Squirro’s pipeline supports the processing of complex document types, such as common office formats like Word or Excel files or PDFs.

See [Indexing Common Formats](common-formats.md#data-processing-common-formats) for instructions on setting up the pipeline to achieve this.

## Reset Project

If you need to reset your project, you can do so from the user interface.

See [Reset Project](reset.md#data-processing-reset-project) for more information.

## Processing Errors

Some steps in the pipeline may abort with an error. In those cases, the item is tagged with a Processing Error keyword.

For information on these errors, see [Processing Errors](errors.md#data-procssing-errors).
