Aggregation rules offer the ability to automatically combine datapoints from multiple input metrics into a smaller set of output metrics according to defined rules. This is useful if you need to use hundreds or thousands of input metrics in a dashboard or graph query, but still want a fast answer. Aggregation rules allow you to effectively pre-compute the most expensive parts of some queries.
Aggregation rules perform the same job as the carbon_aggregator service if you’re self-hosting the open source Graphite tools.
Hosted Graphite’s advanced data views feature is also supported on top of the output metrics from an aggregation rule, so you can get the :sum or an arbitrary percentile across all the metrics matching an aggregation rule
NOTE: The output metric names will be automatically prefixed with “aggregates.” even if you do not specify it in the “output” field
Why are aggregates useful?
If you have a render query using a wildcard to match a large number of metrics, the result of the render query can, in some cases, be very slow to return. Using aggregations reduces the number of metrics which need to be retrieved and can reduce the time spent rendering graphs.
For example, if you have a number of metrics which match the pattern servers.*.requests.<response_code>, and you want to render servers.*.requests.200, the overhead in matching all nodes which match the * in the second node, can be skipped if you set up an aggregation rule with the following pattern/output combination:
{
"pattern": "servers.*.requests.<response_code>",
"output": "aggregates.servers.all.requests.<response_code>"
}
The effect of this config would be as follows: Datapoints coming in for metrics named like this:
servers.server1.requests.200
servers.server2.requests.200
Would be copied and combined as soon as the datapoints are received and a new metric created:
Datapoints coming in for metrics named like:
servers.server1.requests.404
servers.server2.requests.404
Would be copied and combined as soon as the datapoints are received and a new metric created:
Now, instead of defining a graph requesting servers.*.requests.200 which could be slow if it matches hundreds or thousands of metrics, you can define a graph requesting aggregates.servers.all.requests.200. That means you’ll be querying just one metric, which will be much faster.
You can also use the <<rest>> pattern as described in the carbon-aggregator docs which allows you to match across nodes. For example the pattern/output config might look like:
{
"pattern": "servers.*.<<rest>>",
"output": "aggregates.servers.all.<rest>"
}
This would combine all metrics seen with a starting servers.* into the same metric under aggregates.servers.all.<rest>.
So if we saw some metrics which looked like:
servers.server1.requests.200
servers.server2.requests.200
servers.server3.requests.200
servers.server4.requests.200
They would be combined into a new metric:
Similarly, the following:
servers.server1.auth.v2.success
servers.server2.auth.v2.success
servers.server3.auth.v2.success
servers.server4.auth.v2.success
Would be matched by the same pattern and would combine into the new metric:
POST /v1/aggregates/
Request format:
{
"pattern": "string",
"output": "string"
}
Parameters:
Responses:
Curl Example:
curl -H "Content-Type: application/json" -X POST -d \
'{"pattern": "servers.*.requests.<response_code>", "output": "servers.all.requests.<response_code>"}' \
"https://YOUR-API-KEY@api.hostedgraphite.com/v1/aggregates/"Creates an aggregation with the pattern: servers.*.requests.<response_code> and the output servers.all.requests.<response_code>. Metrics which look like servers.server-0001.requests.200 and servers.server-0002.requests.200 would appear in your account as aggregates.servers.all.requests.200. Metrics which look like servers.server-0001.requests.404 and servers.server-0002.requests.404 would appear in your account as aggregates.servers.all.requests.404.
GET /v1/aggregates/
Responses:
GET /v1/aggregates/<aggregation_id>/
Responses:
Curl Example:
curl "https://YOUR-API-KEY@api.hostedgraphite.com/v1/aggregates/<optional_id>/"Lists your aggregations or individual aggregation if optional ID is passed.
DELETE /v1/aggregates/<aggregate_id>/
Responses:
Curl Example:
curl -X DELETE "https://YOUR-API-KEY@api.hostedgraphite.com/v1/aggregates/<aggregate_id>/"Delete aggregation rule with passed id.
To edit an aggregation rule, simply delete the old rule and create a new one. Aggregation rules have no state, so it is safe to delete and re-create them.