Trend Detection#
This page discusses the Squirro Trend Detection functionality. For a step-by-step walkthrough of trend detection, see data-processing-trend-how.
User Interface#
Trend Detection is exposed in the user interface in two areas: creating of a trend detection alert and visualizing through the Trend Detection widget.
The user interface does not allow managing or deleting the trend detection entities yet.
API#
Trend Detection is fully accessible through the API.
The Python SDK also contains methods for every Trend Detection API endpoint. See squirro_client.topic.TrendDetectionMixin for details.
Configuration#
A number of configuration options can be provided when creating a trend detection entity. The more complex of those options are explained here.
Aggregation Field#
This optional configuration allows analysis on any numerical facet in a project.
Reference: For an example of how to use this, follow Scenario 2 in the data-processing-trend-how.
Aggregation Interval#
This determines the time buckets into which the data is fitted. An anomaly is detected as soon as a time bucket’s value is bigger than the predicted trend for this bucket. The bigger the time bucket, the less impact one individual data point has.
Example: Take as an example the analysis of the number of incidents in the system and alerting on that. If the interval is set to daily, a small spike at 9am followed by less incidents than usual around noon would not trigger an anomaly, because the data evens out over the whole day. But using an hour or minute interval would probably pick up that spike and alert on it.
As a result, the interval to use in a project depends largely on how quickly you need to find anomalies and how much potential noise you are willing to accept.
In the API, the format of the interval is <number><unit> - for example 1d for a daily interval. The number can be left away, so day is the same as specifying 1day.
The following units are allowed:
Units |
Meaning |
Example |
m |
Minutes |
5m |
h |
Hours |
3h |
d |
Days |
1d |
w |
Weeks |
4w |
month |
Months |
1month |
quarter |
Quarters |
2quarter |
year |
Years |
1year |
Aggregation Method#
The aggregation method is used together with the aggregation interval. When more than one data point falls within a time bucket, those points are summarized using the aggregation method.
Valid aggregation methods are:
avg: Default, takes the average of all the data points in a bucket.
sum: Summarizes all data points in a time bucket.
max: Uses the maximum value for the interval.
min: Chooses the minimum value per time bucket.