Skip to main content
Version: 2.4.7

Search Aggregation

There are several situations where specific data is not too relevant but needs lifting to a higher level. That is where aggregation comes into play. Aggregation allows grouping entities by one or more properties and then do math operations on that group.

For example, the following search will select and count compute instances that are older than 3 years:

> search is(instance) and age > 3y | count
‚Äčtotal matched: 21
‚Äčtotal unmatched: 0

The count command is actually a special type of aggregation.

Aggregation Functions‚Äč

Aggregations can utilize the following functions:

  • sum
  • min
  • max
  • avg

Function arguments can be variable names (e.g., min(path.to.prop)), static values (e.g., sum(1)), or even calculations using simple expressions (min(path.to.prop * 3 + 2)).

Each grouping function can have an as <name> clause to give the function result a specific name: <function>(..) as <name>. If this as <name> clause is omitted, a name is derived from the function name and property path.

Example: min(memory), sum(1) as count, avg(instance_cores) as average_cores.

The aggregate command tells Resoto to aggregate the search results based on the defined criteria. Each result of the search is then passed to the defined aggregation function(s).

The above example using the count could also be rewritten with aggregate like so:

> search is(instance) and age > 3y | aggregate sum(1) as count
‚Äčcount: 21

Every element is counted as 1, so sum(1) is the number of elements.

It is also possible to define multiple aggregation functions. Let's count both instances and cores:

> search is(instance) and age > 3y | aggregate
sum(1) as count,
sum(instance_cores) as cores
‚Äčcount: 21
‚Äčcores: 66

We could even compute the average, minimum, and maximum number of available cores:

> search is(instance) and age > 3y | aggregate
sum(1) as count,
sum(instance_cores) as cores,
min(instance_cores) as min_cores,
max(instance_cores) as max_cores,
avg(instance_cores) as avg_cores
‚Äčcount: 21
‚Äčcores: 66
‚Äčmin_cores: 1
‚Äčmax_cores: 4
‚Äčavg_cores: 3.14

Aggregation Groups‚Äč

The real power of aggregations is in defining groups and applying functions on those groups.

A group is defined using a property path. The value of this path is looked up in every document. All documents with the same value form a group.

For example, instances can be grouped by status:

> search is(instance) and age > 3y | aggregate instance_status: sum(1) as count
‚Äčgroup:
‚Äč instance_status: running
‚Äčcount: 1
‚Äč---
‚Äčgroup:
‚Äč instance_status: stopped
‚Äčcount: 15
‚Äč---
‚Äčgroup:
‚Äč instance_status: terminated
‚Äčcount: 5

Grouping variable can be named using as. By default, the last part of the path is used as the variable name.

Additional aggregation functions also get applied on each group:

> search is(instance) and age > 3y | aggregate instance_status as status:
sum(1) as count,
sum(instance_cores) as cores,
min(instance_cores) as min_cores,
max(instance_cores) as max_cores,
avg(instance_cores) as avg_cores
‚Äčgroup:
‚Äč status: running
‚Äčcount: 1
‚Äčcores: 4
‚Äčmin_cores: 4
‚Äčmax_cores: 4
‚Äčavg_cores: 4
‚Äč---
‚Äč group:
‚Äčstatus: stopped
‚Äčcount: 15
‚Äčcores: 51
‚Äčmin_cores: 1
‚Äčmax_cores: 4
‚Äčavg_cores: 3.4
‚Äč---
‚Äč group:
‚Äčstatus: terminated
‚Äčcount: 5
‚Äčcores: 11
‚Äčmin_cores: 1
‚Äčmax_cores: 4
‚Äčavg_cores: 2.2

Groups can also be defined using multiple grouping variables:

> search is(instance) and age > 3y | aggregate
instance_status as status, instance_type as type:
sum(1) as count,
sum(instance_cores) as cores
‚Äčgroup:
‚Äč status: running
‚Äč type: n1-standard-4
‚Äčcount: 1
‚Äčcores: 4
‚Äč---
‚Äčgroup:
‚Äč status: stopped
‚Äč type: m5.xlarge
‚Äčcount: 12
‚Äčcores: 48
‚Äč---
‚Äčgroup:
‚Äč status: stopped
‚Äč type: t2.micro
‚Äčcount: 3
‚Äčcores: 3
‚Äč---
‚Äčgroup:
‚Äč status: terminated
‚Äč type: n1-standard-2
‚Äčcount: 5
‚Äčcores: 11