A comprehensive set of turnkey infrastructure integrations

Including dozens of AWS and Azure services, web, database, network, containers, orchestrations like Docker and Kubernetes, and more.


Complete visibility into the health and performance of applications and their underlying infrastructure

Quickly pinpoint the root cause of performance issues across the stack, down to a poor-performing line of code


Custom metrics and analytics

Analyze custom infrastructure, application, and business metrics

View Custom Metrics Monitoring Info
Powerful API that makes it easy to collect and create any custom metric

Achieve ultimate visibility and enhanced troubleshooting with synthetic and real user monitoring


Free APM Software

Catch bugs early on, and gain full visibility and insights into the applications you’re developing

View Product Info
Free, full-function APM tool for testing and troubleshooting application performance before moving into production

Dev Edition includes five traces per minute, 100 metrics, three hosts, and six containers


Log Management and Analytics powered by SolarWinds Loggly

Integrated, cost-effective, hosted, and scalable full-stack, multi-source log management

View Log Management and Analytics Info
Collect, search, and analyze log data in addition to your metrics and traces to quickly pinpoint application performance problems

Reduce mean time to resolution (MTTR) by quickly jumping from a trace or host view into the relevant logs to accelerate troubleshooting


Digital Experience Monitoring Powered by SolarWinds Pingdom

Make your websites faster and more reliable with easy-to-use web performance and digital experience monitoring

View Digital Experience Monitoring Info
Add client-side web application performance monitoring. Provide maximum observability by adding the user’s perspective.

Achieve ultimate visibility and enhanced troubleshooting with synthetic and real user monitoring


Graphios is the fifth installment to our Collector Highlight Series.  We started the series with an overview of the patterns employed by the various data collections tools available, with additional posts including StatsD, collectd, and Heka.

Note: This article was originally published on Librato, which has since been merged with SolarWinds®AppOptics. Check out what’s possible with AppOptics APM tools.

What Is It?

Graphios is the easiest way to connect Nagios to systems like Librato and Graphite. If you need to emit metric data from Nagios to one of the newer graphing tools like InfluxDB or OpenTSDB, Graphios (via StatsD) is the only option other than coding up something yourself.

Graphios works as a glue layer between Nagios and various time-series databases. It uses the host_perfdata and service_perfdata hooks (defined in your nagios.cfg) to capture metrics in your perfdata log, then formats and sends them to systems like Librato, StatsD, and collectd. Graphios uses a modular backend system to support various metrics storage systems via a series of plugins.

How Does It Work?

Although its initial configuration may seem complex, Graphios is conceptually a very simple special-purpose log parser. It runs as a daemon, waking up on a configurable interval, checking for new performance data logs exported by Nagios, and processing them.

Graphios has a modular back-end model which allows it to write to multiple metrics systems. When Graphios finds a new performance data file, it parses metrics out of it, computes appropriate metric names for the enabled back-end plugins, and then emits the metrics to each back-end metrics system as required.

How Do I Install It?

Graphios is a Python program, so the easiest way to install it is with pip:

pip install graphios

It’s also pretty easy to install Graphios manually.  First, get the most recent
version from git with:

git clone https://github.com/shawn-sterling/graphios.git

Then copy the various files to the appropriate locations:

mkdir -p /etc/graphios
cp graphios*.py /usr/local/bin
cp graphios.cfg /etc/graphios/

Configuration Requirements:

To get Graphios up and running, you’ll need to manually configure three things:

  • The Nagios config files that deal with host and service checks
  • The nagios.cfg
  • The graphios.cfg file

If you installed with pip, the setup.py script will attempt to detect and automatically add a commented-out configuration to your nagios.cfg. The setup script does a pretty good job of creating this configuration on all but the most bespoke Nagios setups. All you should need to do is uncomment the new syntax and restart Nagios.  Given the configuration flexibility of the Nagios however, it’s possible that you’ll need to manually intervene and modify the nagios.cfg yourself.

Designing and Configuring a Metric Name Schema

Nagios is a stand-alone monolithic system in that it assumes its check-command output will never be exported—that no system but Nagios will ever need to process it. So Nagios services generally have fairly simple names like PING, or LOAD.  Because all of the necessary context is inside the Nagios UI, it should be obvious to the operator what those names refer to.

Graphing systems like Graphite, however, are not monolithic; they’re designed to work alongside other monitoring systems and data collectors.  Therefore, they necessarily assume that all data is sourced externally (everything comes from some other monitoring system), and as a result they use dot-delineated, hierarchical metric names like Nagios.dc4.dbserver12.LOAD. In Graphite, a name like LOAD doesn’t tell the operator anything about what system the metric refers to, much less how it was collected.

To be an effective glue layer, Graphios gives you a simple, transparent means to translate Nagios’ simple, monolithic service names into context-rich hierarchical names that can be used by systems like Librato and Graphite. Specifically, Graphios can read metric prefixes and suffixes out of your Nagios service and host definitions using custom attributes. For example, a typical Nagios service description, excluding the minutiae normally packed into upper-level templates, looks like this:

define service{
   use                  generic-service
   host_name            box1,box2,box3
   service_description  SSH
   check_command        check_ssh

The output of the check_ssh plugin looks like this:

SSH OK - OpenSSH_5.9p1 Debian-5ubuntu1 (protocol 2.0) | 

Everything after the pipe is performance data; these are the metrics Graphios exports. In this case, we have a single metric called “time”, which measures the response time of the ssh port (in this case, the ssh port responded to the check_ssh plugin in 0.009549 seconds). Graphios automatically prefixes the metric name with the host name, so without doing anything at all, our metric name becomes:


As we’ve already observed above, box1.time isn’t a particularly meaningful metric name, so we can tell Graphios to put some additional context in front of this metric name by inserting a _graphiteprefix custom attribute into the service definition like so:

define service{
   use                     generic-service
   host_name               box1,box2,box3
   service_description     SSH
   check_command           check_ssh
   _graphiteprefix         nagios.dc1

Graphios will now prepend this prefix to the metric name, making it:


This is a little bit better, but we can insert some additional context about the service between the host name and the metric name using a _graphitepostfix custom attribute in our service configuration like so:

define service{
   use                  generic-service
   host_name            box1,box2,box3
   service_description  SSH
   check_command        check_ssh
   _graphiteprefix      nagios.dc1
   _graphitepostfix     sshd.rt

Now, Graphios will insert ssh.rt between the host and metric name, making it:


At this time, we have a pretty decent metric name for use with systems like Graphite and StatsD.

Librato and the Source Dimension Exception

In Librato, metrics are first-class citizens which is a very nice feature that enables the use of dynamic dashboards and templates. Graphios correctly supports the Librato source dimension by submitting host-names like box1 to Librato as the source dimension, rather than hard-coding them into the metric name. So, continuing with our example, the metric name submitted to Librato therefore becomes:


Any graph displaying this metric would then automatically depict one line for each source (three lines total, one each for box1, box2, and box3).

Configuring the Nagios Perfdata Hooks

Next, we need to configure Nagios to export performance data to a log file in a format that Graphios can understand. If you installed Graphios using pip, check the bottom of your nagios.cfg file for a block of configuration that begins with:


If you aren’t already using Nagios perfdata hooks for something else (i.e.,  if you’re currently running Nagios configuration contains: process_performance_data=0), you can simply uncomment this configuration block, and restart Nagios. If you’re already using Nagios perfdata hooks for something like PNP4Nagios, or one of the other RRDTool-based graphing systems, chances are you can safely run both Graphios and your current toolset at the same time. Refer to the Graphios documentation for instructions on how to set this up.

If you didn’t install with pip, or if you don’t see a block of Graphios config at the bottom of your nagios.cfg, then you’ll need to manually enable performance data, and configure it for Graphios.  Refer to the Graphios documentation for instructions on how to set this up.

Once you’ve configured Nagios to emit performance data, restart the Nagios daemon and verify that it’s writing a log file to the Graphios spool directory (named by the service_perfdata_file attribute in your nagios.cfg) with a name like service-perfdata.1418637947. The file should contain lines that look like this:


Configuring Graphios

Graphios installs its config file in /etc/graphios/graphios.cfg by default. This file is very well commented and, for the most part, self-explanatory. There is a global configuration section, and one section for each backend plugin that Graphios can write to. Plugins are generally enabled by setting their enable line to True and configuring the required attributes for the plugin. Here, for example, is a working configuration for Librato:

librato_email = dave@librato.com
librato_token = ecb79ff8a82areallylonggreatbigstringtokenything6b8cb77e8b5bb98934cf7

The whitelist attribute bears mentioning since, without it, Graphios emits all performance data measured by Nagios to Librato, which could become expensive. As configured above, only metrics matching the regular expressions: “load”, “rta”, and “swap” will be emitted to Librato. Here’s a working configuration for StatsD:

statsd_servers =

You may enable multiple backend plugins (Librato AND StatsD), and even multiple comma-separated instances of the same backend plugin (four different StatsD servers, and a carbon server), and Graphios will happily parse out and emit your Nagios metrics to each back-end system. At this point, you can run Graphios from the command line and see if everything works as expected:

graphios.py --verbose


Graphios ships with init scripts for Debian and RPM-based systems. These were installed automatically if you installed with pip on a compatible system. Check the Graphios documentation for more information on starting and running Graphios.

Hints, Tradeoffs, and Gotchas

If Graphios successfully sends all of the metrics in a file to every back-end, it deletes the log file. Otherwise, it keeps the performance data log, and tries again the next time it wakes up. This is generally good operationally-robust behavior, but it’s possible to get into the situation where some number of performance-data logs are never deleted and begin to bog down Graphios by forcing it to reprocess the same files over and over.

You can safely manually delete these files from the spool directory using, for example,

cd /var/nagios/spool/graphios && find . -mmin 60

or by using the nerf attribute in the graphios.cfg to label certain back-ends as non-critical.

For example, if you set nerf_statsd=true in the graphios.cfg, and Graphios fails to send some data to Statsd, Graphios will delete the performance data file anyway, and that data will simply never be sent. This is often an acceptable trade-off for performance metrics sourced from Nagios, which are usually collected on the order of minutes anyway.

Get Started!

If you’re running Nagios today, and you’re still trapped in the RRDTool era, you owe it to yourself to install Graphios and experience the future of scalable metrics analysis systems like Graphite, InfluxDB, and OpenTSDB.

Related articles

© 2024 SolarWinds Worldwide, LLC. All rights reserved.