Enrich Pipeline Steps

There are ten enrich steps available in the Squirro data processing pipeline.

This includes the following steps:

• Unshorten Link

• Duplicate Detection (also known as deduplication)

• Content Augmentation

• Content Extraction

• PDF OCR

• Noise Removal

• Near Duplicate Detection

• Thumbnail Extraction

• Language Detection

• PDF Conversion