Monitor server resources with checks

Sensu checks are commands (or scripts) the Sensu agent executes that output data and produce an exit code to indicate a state. Sensu checks use the same specification as Nagios, so you can use Nagios check plugins with Sensu.

You can use checks to monitor server resources, services, and application health (for example, to check whether Nginx is running) and collect and analyze metrics (for example, to learn how much disk space you have left).

This guide will help you monitor server resources (specifically, CPU usage) by configuring a check named check-cpu with a subscription named system to target all entities that are subscribed to the system subscription. To use this guide, you’ll need to install a Sensu backend and have at least one Sensu agent running on Linux.

Register dynamic runtime assets

To power the check, you’ll use the Sensu CPU Checks dynamic runtime asset. The Sensu CPU Checks asset includes the check-cpu.rb plugin, which your check will rely on.

The Sensu assets packaged from Sensu CPU Checks are built against the Sensu Ruby runtime environment, so you also need to add the Sensu Ruby Runtime dynamic runtime asset. The Sensu Ruby Runtime asset delivers the Ruby executable and supporting libraries the check will need to run the check-cpu.rb plugin.

Use sensuctl asset add to register the Sensu CPU Checks dynamic runtime asset, sensu-plugins/sensu-plugins-cpu-checks:4.1.0:

sensuctl asset add sensu-plugins/sensu-plugins-cpu-checks:4.1.0 -r cpu-checks-plugins
fetching bonsai asset: sensu-plugins/sensu-plugins-cpu-checks:4.1.0
added asset: sensu-plugins/sensu-plugins-cpu-checks:4.1.0

You have successfully added the Sensu asset resource, but the asset will not get downloaded until
it's invoked by another Sensu resource (ex. check). To add this runtime asset to the appropriate
resource, populate the "runtime_assets" field with ["cpu-checks-plugins"].

This example uses the -r (rename) flag to specify a shorter name for the dynamic runtime asset: cpu-checks-plugins.

You can also download the dynamic runtime asset definition for Debian or Alpine from Bonsai and register the asset with sensuctl create --file filename.yml.

Then, use the following sensuctl example to register the Sensu Ruby Runtime dynamic runtime asset, sensu/sensu-ruby-runtime:0.0.10:

sensuctl asset add sensu/sensu-ruby-runtime:0.0.10 -r sensu-ruby-runtime
fetching bonsai asset: sensu/sensu-ruby-runtime:0.0.10
added asset: sensu/sensu-ruby-runtime:0.0.10

You have successfully added the Sensu asset resource, but the asset will not get downloaded until
it's invoked by another Sensu resource (ex. check). To add this runtime asset to the appropriate
resource, populate the "runtime_assets" field with ["sensu-ruby-runtime"].

You can also download the dynamic runtime asset definition from Bonsai and register the asset using sensuctl create --file filename.yml.

Use sensuctl to confirm that both the cpu-checks-plugins and sensu-ruby-runtime dynamic runtime assets are ready to use:

sensuctl asset list
          Name                                                URL                                       Hash    
────────────────────────── ─────────────────────────────────────────────────────────────────────────── ───────── 
 cpu-checks-plugins   //assets.bonsai.sensu.io/.../sensu-plugins-cpu-checks_4.1.0_centos_linux_amd64.tar.gz          518e7c1  
 sensu-ruby-runtime         //assets.bonsai.sensu.io/.../sensu-ruby-runtime_0.0.10_ruby-2.4.4_centos_linux_amd64.tar.gz     338b88b

NOTE: Sensu does not download and install dynamic runtime asset builds onto the system until they are needed for command execution. Read the asset reference for more information about dynamic runtime asset builds.

Create a check

Now that the dynamic runtime assets are registered, create a check named check-cpu that runs the command check-cpu.rb -w 75 -c 90 with the cpu-checks-plugins and sensu-ruby-runtime dynamic runtime assets at an interval of 60 seconds for all entities subscribed to the system subscription. This check generates a warning event (-w) when CPU usage reaches 75% and a critical alert (-c) at 90%.

sensuctl check create check-cpu \
--command 'check-cpu.rb -w 75 -c 90' \
--interval 60 \
--subscriptions system \
--runtime-assets cpu-checks-plugins,sensu-ruby-runtime

Configure the subscription

To run the check, you’ll need a Sensu agent with the subscription system. After you install an agent, open /etc/sensu/agent.yml and add the system subscription so the subscription configuration looks like this:

subscriptions:
  - system

Then, restart the agent:

sudo service sensu-agent restart

Validate the check

Use sensuctl to confirm that Sensu is monitoring CPU usage using the check-cpu, returning an OK status (0). It might take a few moments after you create the check for the check to be scheduled on the entity and the event to return to Sensu backend.

sensuctl event list
    Entity        Check                                                                    Output                                                                   Status   Silenced             Timestamp            
────────────── ─────────── ─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── ──────── ────────── ─────────────────────────────── 
 sensu-centos   check-cpu   CheckCPU TOTAL OK: total=0.2 user=0.0 nice=0.0 system=0.2 idle=99.8 iowait=0.0 irq=0.0 softirq=0.0 steal=0.0 guest=0.0 guest_nice=0.0        0   false      2019-04-23 16:42:28 +0000 UTC

Next steps

Now that you know how to run a check to monitor CPU usage, read these resources to learn more: