Architecture Diagrams#

The Squirro platform primarily generates insights from unstructured data but also processes structured data. Squirro uses structured data made available to Squirro through the data import APIs. The data is persisted on the data storage layer and can be consumed with the data access APIs.

At the most fundamental level this can be split into:

  • Gather - collect data from different systems

  • Understand - derive information and insights from this data

  • Act - provide to the users in an easy-to-understand and actionable way

image1

Processing Sections#

Zooming in one level, the architecture diagram reveals the more fine-grained sections into which the processing can be split. This also shows how the Squirro AI Studio can serve as a foundation for some of these steps.

image2

These steps are split into three different areas of the product:

  • Load - this is the data loading step. The work here is done through built-in as well as custom connectors.

  • Enrich … Automate - this is handled in the pipeline. Every item yielded by data loading flows through this pipeline and will eventually get indexed.

  • Visualize - data is presented to the end users in Dashboards.

Detailed Overview#

Zooming in further, the architecture diagram below shows the individual pipeline steps within each section. The diagram also shows data sources and connectors (at left) and data integrations (at right), which connect to Squirro via APIs.

Ingester Service and Data Processing Pipeline