rrdtool on Andrés Álvarez

Working With RRDTool

Tue, 02 May 2017 00:00:00 +0000

Zenoss makes a lot of use of RRDTool for storing monitored performance data and generating graphs. In this post I want to go into more detail on the components of RRDTool and how works.

Round Robin Databases

RRDTool is a program that works with Round Robin databases (hence RRD). This type of database is more geared and limited towards storing time series data.

Moreover, rrdtool also doesn't store the data handed to it. Instead, it first re-samples the data and then stores this re-sampled data. Since rrdtool deals with time series data, it knows that certain data is meant to be stored every certain period of time (i.e 5 minutes). This newly re-sampled data will be the same as the original data, but the stored numbers will be different.

This means that if for example, we were to store some data every 5 minutes, a data sample that arrives at the 03 minute mark cannot go into the database, therefore it has to be re-adjusted into the correct slots. While rrdtool makes this adjustment, it will also change the value, so that this new value still represents the space in the curve.

In rrdtool, each database table is stored in a separate .rrd file. This means that one file represents one table.

We can create a database using the following command:

rrdtool create ex5.rrd --step=5 DS:input:GAUGE:10:0:1000000 RRA:LAST:0.5:1:603

Data Re-sampling: Step

One important parameter here is the step parameter. This means that rrdtool will re-sample all the data that is fed into the database to a 300 second interval. Still, data can be fed as quickly as desired (i.e a lot of data in one second), however the data will be re-sampled to 300 seconds on storage. This means that it only stores data every 300 seconds. Because of this re-sampling, rrdtool will not accept data from the past.

Fixing Zenoss Device Network Interface Graphs

Fri, 13 Jan 2017 00:00:00 +0000

When testing the monitoring of the OpenStack compute node devices in Zenoss (using OpenStack Infrastructure ZenPack), I noticed that I could not get any graphs for the network interfaces of the device, although we could still get and model all the available interfaces:

This was indeed strange because I could perfectly get the graphs for other devices. After a lot of head scratching and prying around the code and interface, I finally found the reason, which I explain below.

Using RRDTool in Zenoss

Tue, 22 Nov 2016 00:00:00 +0000

rrdtool is an awesome high performance data logging and graphing tool for time series data. Zenoss Core uses RRDTool to collect, monitor, and graph peformance data for devices.

However Zenoss Core comes with built-in helper wrappers around RRDTool that makes using it within Zenoss much easier. These source code files can be found in $ZENHOME/Products/ZenRRD/.

RRDUtil

Located in $ZENHOME/Products/ZenRRD/RRDUtil.py, this Python module contains many wrapper methods around the rrdtool library. These methods can help us write to, create new, and read .rrd files using Python.

We can easily import this module into our code, using the following import statement:

from Products.ZenRRD.RRDUtil import RRDUtil

Monitoring CPU Utilization in Zenoss

Fri, 04 Nov 2016 00:00:00 +0000

While taking a look at the CPU Utilization graphs offered Zenoss Core's Linux Monitor ZenPack (v1.2.1), I noticed that the percentage values for Idle were ridiculously high:

This made sense since this particular device contains 16 cores. However, this then means that the monitoring template isn't really taking this into consideration, and instead just spits out the total value from all cores.

Zenoss Monitoring Template Data Points

Thu, 03 Nov 2016 00:00:00 +0000

Content from this post is mostly obtained from Zenoss Core Administration guide.

In Zenoss Core monitoring templates, data sources can return data for one or more performance metrics. Each metric retrieved by a data source is represented by a data point.

When creating a data point, there are some important fields that we can define for our data point:

Name: Displays the name you entered in the Add a New DataPoint dialog.
RRD Type: Specify the RRD data source type to use for storing data for this data point. (Zenoss Core uses RRDTool to store performance data.) Available options are:
- COUNTER: Saves the rate of change of the value over a step period. This assumes that the value is always increasing (the difference between the current and the previous value is greater than 0). Traffic counters on a router are an ideal candidate for using COUNTER.
- GAUGE: Does not save the rate of change, but saves the actual value. There are no divisions or calculations. To see memory consumption in a server, for example, you might want to select this value.
- DERIVE: Same as COUNTER, but additionally allows negative values. If you want to see the rate of change in free disk space on your server, for example, then you might want to select this value.
- ABSOLUTE: Saves the rate of change, but assumes that the previous value is set to 0. The difference between the current and the previous value is always equal to the current value. Thus, ABSOLUTE stores the current value, divided by the step interval.
Create Command: Enter an RRD expression used to create the database for this data point. If you do not enter a value, then the system uses a default applicable to most situations. For details about the rrdcreate command, go here.
RRD Minimum: Enter a value. Any value received that is less than this number is ignored.
RRD Maximum: Enter a value. Any value received that is greater than this number is ignored.

Data Point Aliases

Performance reports pull information from various data points that represent a metric. The report itself knows which data points it requires, and which modifications are needed, if any, to put the data in its proper units and format.

The addition of a data point requires changing the report.

To allow for more flexibility in changes, some reports use data point aliases. Data point aliases group data points so they can be more easily used for reporting. In addition, if the data points return data in different units, then the plugin can normalize that data into a common unit.