Agent reference

Example Sensu agent configuration file (download)

The Sensu agent is a lightweight client that runs on the infrastructure components you want to monitor. Agents register with the Sensu backend as monitoring entities with type: "agent". Agent entities are responsible for creating check and metrics events to send to the backend event pipeline.

The Sensu agent is available for Linux, macOS, and Windows. For Windows operating systems, the Sensu agent uses cmd.exe for the execution environment. For all other operating systems, the Sensu agent uses the Bourne shell (sh).

Read the installation guide to install the agent.

Agent authentication

The Sensu agent authenticates to the Sensu backend via WebSocket transport by either built-in basic (username and password) or mutual transport layer security (mTLS) authentication.

Username and password authentication

The default mechanism for agent authentication is built-in basic authentication with username and password. The Sensu agent uses username and password authentication unless mTLS authentication has been explicitly configured.

For username and password authentication, sensu-agent joins the username and password with a colon and encodes them as a Base64 value. Sensu provides the encoded string as the value of the Authorization HTTP header — for example, Authorization: Basic YWdlbnQ6UEBzc3cwcmQh — to authenticate to the Sensu backend.

When using username and password authentication, sensu-agent also sends the following HTTP headers in requests to the backend:

• Sensu-User: the username in plaintext
• Sensu-AgentName: the agent’s configured name in plaintext
• Sensu-Subscriptions: the agent’s subscriptions in a comma-separated plaintext list
• Sensu-Namespace: the agent’s configured namespace in plaintext

mTLS authentication

When mTLS is configured for both the Sensu agent and backend, the agent uses mTLS authentication instead of the default username and password authentication.

Sensu backends that are configured for mTLS authentication will no longer accept agent authentication via username and password. Agents that are configured to use mTLS authentication cannot authenticate with the backend unless the backend is configured for mTLS.

To configure the agent and backend for mTLS authentication:

• In the backend configuration, specify valid certificate and key files as values for the agent-auth-cert-file and agent-auth-key-file parameters (e.g. backend-1.pem and backend-1-key.pem, respectively).
• In the agent configuration, specify valid certificate and key files as values for the cert-file and key-file parameters (e.g. agent.pem and agent-key.pem, respectively).

The agent and backend will compare the provided certificates with the trusted CA certificate either in the system trust store or specified explicitly as the agent-auth-trusted-ca-file in the backend configuration and trusted-ca-file in the agent configuration.

When using mTLS authentication, sensu-agent sends the following HTTP headers in requests to the backend:

• Sensu-AgentName: the agent’s configured name in plaintext
• Sensu-Subscriptions: the agent’s subscriptions in a comma-separated, plaintext list
• Sensu-Namespace: the agent’s configured namespace in plaintext

If the Sensu agent is configured for mTLS authentication, it will not send the Authorization HTTP header.

Certificate revocation check

The Sensu backend checks certificate revocation list (CRL) and Online Certificate Status Protocol (OCSP) endpoints for agent mTLS, etcd client, and etcd peer connections whose remote sides present X.509 certificates that provide CRL and OCSP revocation information.

Communication between the agent and backend

The Sensu agent uses WebSocket (ws) protocol to send and receive JSON messages with the Sensu backend. For optimal network throughput, agents will attempt to negotiate the use of Protobuf serialization when communicating with a Sensu backend that supports it. This communication is via clear text by default.

Follow Secure Sensu to configure the backend and agent for WebSocket Secure (wss) encrypted communication.

Create observability events using service checks

Sensu uses the publish/subscribe pattern of communication, which allows automated registration and deregistration of ephemeral systems. At the core of this model are Sensu subscriptions: a list of roles and responsibilities assigned to the system (for example, a webserver or database). These subscriptions determine which monitoring checks the agent will execute. For an agent to execute a service check, you must specify the same subscription in the agent configuration and the check definition.

After receiving a check request from the Sensu backend, the agent:

1. Applies any tokens that match attribute values in the check definition.
2. Fetches dynamic runtime assets and stores them in its local cache. By default, agents cache dynamic runtime asset data at /var/cache/sensu/sensu-agent (C:\ProgramData\sensu\cache\sensu-agent on Windows systems) or as specified by the the cache-dir flag.
3. Executes the check command.
4. Executes any hooks specified by the check based on the exit status.
5. Creates an event that contains information about the applicable entity, check, and metric.

Read the subscriptions reference for more information.

Proxy entities

Sensu proxy entities allow Sensu to monitor external resources on systems or devices where a Sensu agent cannot be installed (such a network switch). The Sensu backend stores proxy entity definitions (unlike agent entities, which the agent stores). When the backend requests a check that includes a proxy_entity_name, the agent includes the provided entity information in the observation data in events in place of the agent entity data. read the entity reference and Monitor external resources for more information about monitoring proxy entities.

Create observability events using the agent API

The Sensu agent API allows external sources to send monitoring data to Sensu without requiring the external sources to know anything about Sensu’s internal implementation. The agent API listens on the address and port specified by the API configuration flags. Only unsecured HTTP (no HTTPS) is supported at this time. Any requests for unknown endpoints result in an HTTP 404 Not Found response.

/events (POST)

The /events API provides HTTP POST access to publish observability events to the Sensu backend pipeline via the agent API. The agent places events created via the /events POST endpoint into a queue stored on disk. In case of a loss of connection with the backend or agent shutdown, the agent preserves queued event data. When the connection is reestablished, the agent sends the queued events to the backend.

The /events API uses a configurable burst limit and rate limit for relaying events to the backend. Read API configuration flags to configure the events-burst-limit and events-rate-limit flags.

Example POST request to events API

The following example submits an HTTP POST request to the /events API. The request creates event for a check named check-mysql-status with the output could not connect to mysql and a status of 1 (warning). The agent responds with an HTTP 202 Accepted response to indicate that the event has been added to the queue to be sent to the backend.

The event will be handled according to an email handler definition.

curl -X POST \
-H 'Content-Type: application/json' \
-d '{
  "check": {
    "metadata": {
      "name": "check-mysql-status"
    },
    "handlers": ["email"],
    "status": 1,
    "output": "could not connect to mysql"
  }
}' \
http://127.0.0.1:3031/events

HTTP/1.1 202 Accepted

Detect silent failures

You can use the Sensu agent API in combination with the check time-to-live (TTL) attribute to detect silent failures. This creates what’s commonly referred to as a “dead man’s switch”.

With check TTLs, Sensu can set an expectation that a Sensu agent will publish additional events for a check within the period of time specified by the TTL attribute. If a Sensu agent fails to publish an event before the check TTL expires, the Sensu backend creates an event with a status of 1 (warning) to indicate the expected event was not received. For more information about check TTLs, read the checks reference.

You can use the Sensu agent API to enable tasks that run outside of Sensu’s check scheduling to emit events. Using the check TTL attribute, these events create a dead man’s switch: if the task fails for any reason, the lack of an “all clear” event from the task will notify operators of a silent failure (which might otherwise be missed). If an external source sends a Sensu event with a check TTL to the Sensu agent API, Sensu expects another event from the same external source before the TTL expires.

In this example, external event input via the Sensu agent API uses a check TTL to create a dead man’s switch for MySQL backups. Assume that a MySQL backup script runs periodically, and you expect the job to take a little less than 7 hours to complete.

• If the job completes successfully, you want a record of it, but you don’t need to receive an alert.
• If the job fails or continues running longer than the expected 7 hours, you do need to receive an alert.

This script sends an event that tells the Sensu backend to expect an additional event with the same name within 7 hours of the first event:

curl -X POST \
-H 'Content-Type: application/json' \
-d '{
  "check": {
    "metadata": {
      "name": "mysql-backup-job"
    },
    "status": 0,
    "output": "mysql backup initiated",
    "ttl": 25200
  }
}' \
http://127.0.0.1:3031/events

With this initial event submitted to the agent API, you recorded in the Sensu backend that your script started. You also configured the dead man’s switch so that you’ll receive an alert if the job fails or runs for too long. Although it is possible for your script to handle errors gracefully and emit additional observability events, this approach allows you to worry less about handling every possible error case. A lack of additional events before the 7-hour period elapses results in an alert.

If your backup script runs successfully, you can send an additional event without the TTL attribute, which removes the dead man’s switch:

curl -X POST \
-H 'Content-Type: application/json' \
-d '{
  "check": {
    "metadata": {
      "name": "mysql-backup-job"
    },
    "status": 0,
    "output": "mysql backup ran successfully!"
  }
}' \
http://127.0.0.1:3031/events

When you omit the TTL attribute from this event, you also remove the dead man’s switch being monitored by the Sensu backend. This effectively sounds the “all clear” for this iteration of the task.

API specification

{
  "check": {
    "metadata": {
      "name": "check-mysql-status"
    },
    "status": 1,
    "output": "could not connect to mysql"
  }
}

/healthz (GET)

The /healthz API provides HTTP GET access to the status of the Sensu agent via the agent API.

Example

In the following example, an HTTP GET request is submitted to the /healthz API:

curl http://127.0.0.1:3031/healthz

The request results in a healthy response:

ok

API specification

Create observability events using the StatsD listener

Sensu agents include a listener to send StatsD metrics to the event pipeline. By default, Sensu agents listen on UDP socket 8125 for messages that follow the StatsD line protocol and send metric events for handling by the Sensu backend.

For example, you can use the Netcat utility to send metrics to the StatsD listener:

echo 'abc.def.g:10|c' | nc -w1 -u localhost 8125

Sensu does not store metrics received through the StatsD listener, so it’s important to configure event handlers.

StatsD line protocol

The Sensu StatsD listener accepts messages formatted according to the StatsD line protocol:

<metricname>:<value>|<type>

For more information, read the StatsD documentation.

Configure the StatsD listener

To configure the StatsD listener, specify the statsd-event-handlers configuration flag in the agent configuration, and start the agent. For example, to start an agent that sends StatsD metrics to InfluxDB, run:

sensu-agent --statsd-event-handlers influx-db

Use the StatsD configuration flags to change the default settings for the StatsD listener address, port, and flush interval. For example, to start an agent with a customized address and flush interval, run:

sensu-agent --statsd-event-handlers influx-db --statsd-flush-interval 1 --statsd-metrics-host 123.4.5.11 --statsd-metrics-port 8125

Create observability events using the agent TCP and UDP sockets

Sensu agents listen for external monitoring data using TCP and UDP sockets. The agent sockets accept JSON event data and pass events to the Sensu backend event pipeline for processing. The TCP and UDP sockets listen on the address and port specified by the socket configuration flags.

Use the TCP socket

This example demonstrates external monitoring data input via the Sensu agent TCP socket. The example uses Bash’s built-in /dev/tcp file to communicate with the Sensu agent socket:

echo '{"name": "check-mysql-status", "status": 1, "output": "error!"}' > /dev/tcp/localhost/3030

You can also use the Netcat utility to send monitoring data to the agent socket:

echo '{"name": "check-mysql-status", "status": 1, "output": "error!"}' | nc localhost 3030

Use the UDP socket

This example demonstrates external monitoring data input via the Sensu agent UDP socket. The example uses Bash’s built-in /dev/udp file to communicate with the Sensu agent socket:

echo '{"name": "check-mysql-status", "status": 1, "output": "error!"}' > /dev/udp/127.0.0.1/3030

You can also use the Netcat utility to send monitoring data to the agent socket:

echo '{"name": "check-mysql-status", "status": 1, "output": "error!"}' | nc -u -v 127.0.0.1 3030

Socket event format

The agent TCP and UDP sockets use a special event data format designed for backward compatibility with Sensu Core 1.x check results. Attributes specified in socket events appear in the resulting event data passed to the Sensu backend.

Example socket input: Minimum required attributes

{
  "name": "check-mysql-status",
  "status": 1,
  "output": "error!"
}

Example socket input: All attributes

{
  "name": "check-http",
  "status": 1,
  "output": "404",
  "source": "sensu-docs-site",
  "executed": 1550013435,
  "duration": 1.903135228,
  "handlers": ["slack", "influxdb"]
}

Socket event specification

{
  "name": "check-mysql-status"
}

{
  "status": 0
}

{
  "output": "CheckHttp OK: 200, 78572 bytes"
}

{
  "source": "sensu-docs-site"
}

{
  "client": "sensu-docs-site"
}

{
  "executed": 1458934742
}

{
  "duration": 1.903135228
}

{
  "command": "http-check --url https://sensu.io"
}

{
  "interval": 60
}

{
  "handlers": ["slack", "influxdb"]
}

Keepalive monitoring

Sensu keepalives are the heartbeat mechanism used to ensure that all registered agents are operational and able to reach the Sensu backend. Sensu agents publish keepalive events containing entity configuration data to the Sensu backend according to the interval specified by the keepalive-interval flag.

If a Sensu agent fails to send keepalive events over the period specified by the keepalive-critical-timeout flag, the Sensu backend creates a keepalive critical alert in the Sensu web UI. The keepalive-critical-timeout is set to 0 (disabled) by default to help ensure that it will not interfere with your keepalive-warning-timeout setting.

If a Sensu agent fails to send keepalive events over the period specified by the keepalive-warning-timeout flag, the Sensu backend creates a keepalive warning alert in the Sensu web UI. The value you specify for keepalive-warning-timeout must be lower than the value you specify for keepalive-critical-timeout.

You can use keepalives to identify unhealthy systems and network partitions, send notifications, and trigger auto-remediation, among other useful actions. In addition, the agent maps keepalive-critical-timeout and keepalive-warning-timeout values to certain event check attributes, so you can create time-based event filters to reduce alert fatigue for agent keepliave events.

Handle keepalive events

You can use a keepalive handler to connect keepalive events to your monitoring workflows. Sensu looks for an event handler named keepalive and automatically uses it to process keepalive events.

Suppose you want to receive Slack notifications for keepalive alerts, and you already have a Slack handler set up to process events. To process keepalive events using the Slack pipeline, create a handler set named keepalive and add the slack handler to the handlers array. The resulting keepalive handler set configuration looks like this:

---
type: Handler
api_version: core/v2
metadata:
  name: keepalive
  namespace: default
spec:
  handlers:
  - slack
  type: set

{
  "type": "Handler",
  "api_version": "core/v2",
  "metadata" : {
    "name": "keepalive",
    "namespace": "default"
  },
  "spec": {
    "type": "set",
    "handlers": [
      "slack"
    ]
  }
}

You can also use the keepalive-handlers flag to send keepalive events to any handler you have configured. If you do not specify a keepalive handler with the keepalive-handlers flag, the Sensu backend will use the default keepalive handler and create an event in sensuctl and the Sensu web UI.

Connection failure

Although connection failure may be due to different kinds of socket errors (such as unexpectedly closed connections and TLS handshake failures), the Sensu agent generally keeps retrying connections to each URL in the backend-url list until it is successfully connected to a backend URL or you stop the process.

When you start up a Sensu agent configured with multiple backend-url values, the agent shuffles the backend-url list and attempts to connect to the first URL in the shuffled list.

If the agent cannot establish a WebSocket connection with the first URL within the number of seconds specified for the backend-handshake-timeout, the agent abandons the connection attempt and tries the next URL in the shuffled list.

When the agent establishes a WebSocket connection with a backend URL within the backend-handshake-timeout period, the agent sends a heartbeat message to the backend at the specified backend-heartbeat-interval. For every heartbeat the agent sends, the agent expects the connected backend to send a heartbeat response within the number of seconds specified for the backend-heartbeat-timeout. If the connected backend does not respond within the backend-heartbeat-timeout period, the agent closes the connection and attempts to connect to the next backend URL in the shuffled list.

The agent iterates through the shuffled backend-url list until it successfully establishes a WebSocket connection with a backend, returning to the first URL if it fails to connect with the last URL in the list.

Service management

Start the service

Use the sensu-agent tool to start the agent and apply configuration flags.

sensu-agent start --subscriptions disk-checks --log-level debug

sensu-agent start --help

sudo service sensu-agent start

sensu-agent service install

sensu-agent service install --config-file 'C:\\monitoring\\sensu\\config\\agent.yml' --log-file 'C:\\monitoring\\sensu\\log\\sensu-agent.log'

Stop the service

To stop the agent service using a service manager:

sudo service sensu-agent stop

sc.exe stop SensuAgent

Restart the service

You must restart the agent to implement any configuration updates.

To restart the agent using a service manager:

sudo service sensu-agent restart

sc.exe start SensuAgent

Enable on boot

To enable the agent to start on system boot:

sudo systemctl enable sensu-agent

sudo systemctl disable sensu-agent

Get service status

To view the status of the agent service using a service manager:

service sensu-agent status

sc.exe query SensuAgent

Get service version

There are two ways to get the current agent version: the sensu-agent tool and the agent version API.

To get the version of the current sensu-agent tool:

sensu-agent version

To get the version of the running sensu-agent service:

curl http://127.0.0.1:3031/version

Uninstall the service

sensu-agent service uninstall

Get help

The sensu-agent tool provides general and command-specific help flags.

To view sensu-agent commands, run:

sensu-agent help

To list options for a specific command (in this case, sensu-agent start), run:

sensu-agent start --help

Registration, endpoint management, and service discovery

Sensu agents automatically discover and register infrastructure components and the services running on them. When an agent process stops, the Sensu backend can automatically create and process a deregistration event.

In practice, agent registration happens when a Sensu backend processes an agent keepalive event for an agent that is not already registered in the Sensu agent registry (based on the configured agent name). The Sensu backend stores this agent registry, and it is accessible via sensuctl entity list.

All Sensu agent data provided in keepalive events gets stored in the agent registry and used to add context to Sensu events and detect Sensu agents in an unhealthy state.

Registration events

If a Sensu event handler named registration is configured, the Sensu backend creates and processes an event for agent registration, applying any configured filters and mutators before executing the configured handler.

You can use registration events to execute one-time handlers for new Sensu agents. For example, you can use registration event handlers to update external configuration management databases (CMDBs) such as ServiceNow.

The handlers reference includes an example registration event handler.

Deregistration events

As with registration events, the Sensu backend can create and process a deregistration event when the Sensu agent process stops. You can use deregistration events to trigger a handler that updates external CMDBs or performs an action to update ephemeral infrastructures. To enable deregistration events, use the deregister flag, and specify the event handler using the deregistration-handler flag. You can specify a deregistration handler per agent using the deregistration-handler agent flag or by setting a default for all agents using the deregistration-handler backend configuration flag.

Cluster

Agents can connect to a Sensu cluster by specifying any Sensu backend URL in the cluster in the backend-url configuration flag. For more information about clustering, read Backend datastore configuration flags and Run a Sensu cluster.

Synchronize time

System clocks between agents and the backend should be synchronized to a central NTP server. If system time is out-of-sync, it may cause issues with keepalive, metric, and check alerts.

Configuration via flags

The agent loads configuration upon startup, so you must restart the agent for any configuration updates to take effect.

sensu-agent start --help

start the sensu agent

Usage:
  sensu-agent start [flags]

Flags:
      --agent-managed-entity                manage this entity via the agent
      --allow-list string                   path to agent execution allow list configuration file
      --annotations stringToString          entity annotations map (default [])
      --api-host string                     address to bind the Sensu client HTTP API to (default "127.0.0.1")
      --api-port int                        port the Sensu client HTTP API listens on (default 3031)
      --assets-burst-limit int              asset fetch burst limit (default 100)
      --assets-rate-limit float             maximum number of assets fetched per second
      --backend-handshake-timeout int       number of seconds the agent should wait when negotiating a new WebSocket connection (default 15)
      --backend-heartbeat-interval int      interval at which the agent should send heartbeats to the backend (default 30)
      --backend-heartbeat-timeout int       number of seconds the agent should wait for a response to a hearbeat (default 45)
      --backend-url strings                 comma-delimited list of ws/wss URLs of Sensu backend servers. This flag can also be invoked multiple times (default [ws://127.0.0.1:8081])
      --cache-dir string                    path to store cached data (default "/var/cache/sensu/sensu-agent")
      --cert-file string                    certificate for TLS authentication
  -c, --config-file string                  path to sensu-agent config file (default "/etc/sensu/agent.yml")
      --deregister                          ephemeral agent
      --deregistration-handler string       deregistration handler that should process the entity deregistration event
      --detect-cloud-provider               enable cloud provider detection
      --disable-api                         disable the Agent HTTP API
      --disable-assets                      disable check assets on this agent
      --disable-sockets                     disable the Agent TCP and UDP event sockets
      --discover-processes                  indicates whether process discovery should be enabled
      --events-burst-limit int              /events api burst limit (default 10)
      --events-rate-limit float             maximum number of events transmitted to the backend through the /events api
  -h, --help                                help for start
      --insecure-skip-tls-verify            skip TLS verification (not recommended!)
      --keepalive-critical-timeout uint32   number of seconds until agent is considered dead by backend to create a critical event
      --keepalive-handlers strings          comma-delimited list of keepalive handlers for this entity. This flag can also be invoked multiple times
      --keepalive-interval int              number of seconds to send between keepalive events (default 20)
      --keepalive-warning-timeout uint32    number of seconds until agent is considered dead by backend to create a warning event (default 120)
      --key-file string                     key for TLS authentication
      --labels stringToString               entity labels map (default [])
      --log-level string                    logging level [panic, fatal, error, warn, info, debug] (default "info")
      --name string                         agent name (defaults to hostname) (default "my_hostname")
      --namespace string                    agent namespace (default "default")
      --password string                     agent password (default "P@ssw0rd!")
      --redact strings                      comma-delimited list of fields to redact, overwrites the default fields. This flag can also be invoked multiple times (default [password,passwd,pass,api_key,api_token,access_key,secret_key,private_key,secret])
      --require-fips                        indicates whether fips support should be required in openssl
      --require-openssl                     indicates whether openssl should be required instead of go's built-in crypto
      --retry-max                           maximum amount of time to wait before retrying an agent connection to the backend
      --retry-min                           minimum amount of time to wait before retrying an agent connection to the backend
      --retry-multiplier                    value multiplied with the current retry delay to produce a longer retry delay (bounded by --retry-max)
      --socket-host string                  address to bind the Sensu client socket to (default "127.0.0.1")
      --socket-port int                     port the Sensu client socket listens on (default 3030)
      --statsd-disable                      disables the statsd listener and metrics server
      --statsd-event-handlers strings       comma-delimited list of event handlers for statsd metrics. This flag can also be invoked multiple times
      --statsd-flush-interval int           number of seconds between statsd flush (default 10)
      --statsd-metrics-host string          address used for the statsd metrics server (default "127.0.0.1")
      --statsd-metrics-port int             port used for the statsd metrics server (default 8125)
      --subscriptions strings               comma-delimited list of agent subscriptions. This flag can also be invoked multiple times
      --trusted-ca-file string              TLS CA certificate bundle in PEM format
      --user string                         agent user (default "agent")

General configuration flags

sensu-agent start --agent-managed-entity

agent-managed-entity: true

sensu-agent start --allow-list /etc/sensu/check-allow-list.yaml

allow-list: /etc/sensu/check-allow-list.yaml

sensu-agent start --annotations sensu.io/plugins/slack/config/webhook-url=https://hooks.slack.com/services/T00000000/B00000000/XXXXXXXXXXXXXXXXXXXXXXXX
sensu-agent start --annotations example-key="example value" --annotations example-key2="example value"

annotations:
  sensu.io/plugins/slack/config/webhook-url: "https://hooks.slack.com/services/T00000000/B00000000/XXXXXXXXXXXXXXXXXXXXXXXX"

sensu-agent start --assets-burst-limit 100

assets-burst-limit: 100

sensu-agent start --assets-rate-limit 1.39

assets-rate-limit: 1.39

sensu-agent start --backend-handshake-timeout 20

backend-handshake-timeout: 20

sensu-agent start --backend-heartbeat-interval 45

backend-heartbeat-interval: 45

sensu-agent start --backend-heartbeat-timeout 60

backend-heartbeat-timeout: 60

sensu-agent start --backend-url ws://127.0.0.1:8081
sensu-agent start --backend-url ws://127.0.0.1:8081 --backend-url ws://127.0.0.1:8082

sensu-agent start --backend-url wss://127.0.0.1:8081
sensu-agent start --backend-url wss://127.0.0.1:8081 --backend-url wss://127.0.0.1:8082

backend-url:
  - "ws://127.0.0.1:8081"
  - "ws://127.0.0.1:8082"

backend-url:
  - "wss://127.0.0.1:8081"
  - "wss://127.0.0.1:8082"

sensu-agent start --cache-dir /cache/sensu-agent

cache-dir: "/cache/sensu-agent"

sensu-agent start --config-file /sensu/agent.yml
sensu-agent start -c /sensu/agent.yml

sensu-agent start --disable-assets

disable-assets: true

sensu-agent start --discover-processes

discover-processes: true

sensu-agent start --labels proxy_type=website
sensu-agent start --labels example_key1="example value" example_key2="example value"

labels:
  proxy_type: website

sensu-agent start --log-level debug

log-level: debug

sensu-agent start --name agent-01

name: "agent-01"

sensu-agent start --retry-max 120s

retry-max: 120s

sensu-agent start --retry-min 1s

retry-min: 1s

sensu-agent start --retry-multiplier 2.0

retry-multiplier: 2.0

sensu-agent start --subscriptions disk-checks,process-checks
sensu-agent start --subscriptions disk-checks --subscriptions process-checks

subscriptions:
  - disk-checks
  - process-checks

API configuration flags

sensu-agent start --api-host 127.0.0.1

api-host: "127.0.0.1"

sensu-agent start --api-port 3031

api-port: 3031

sensu-agent start --disable-api

disable-api: true

sensu-agent start --events-burst-limit 20

events-burst-limit: 20

sensu-agent start --events-rate-limit 20.0

events-rate-limit: 20.0

Ephemeral agent configuration flags

sensu-agent start --deregister

deregister: true

sensu-agent start --deregistration-handler deregister

deregistration-handler: deregister

sensu-agent start --detect-cloud-provider false

detect-cloud-provider: false

Keepalive configuration flags

sensu-agent start --keepalive-critical-timeout 300

keepalive-critical-timeout: 300

sensu-agent start --keepalive-handlers slack,email

keepalive-handlers:
- slack
- email

sensu-agent start --keepalive-interval 30

keepalive-interval: 30

sensu-agent start --keepalive-warning-timeout 300

keepalive-warning-timeout: 300

Security configuration flags

sensu-agent start --cert-file /path/to/tls/agent.pem

cert-file: "/path/to/tls/agent.pem"

sensu-agent start --insecure-skip-tls-verify

insecure-skip-tls-verify: true

sensu-agent start --key-file /path/to/tls/agent-key.pem

key-file: "/path/to/tls/agent-key.pem"

sensu-agent start --namespace ops

namespace: ops

sensu-agent start --password secure-password

password: secure-password

sensu-agent start --redact secret,ec2_access_key

redact:
  - secret
  - ec2_access_key

sensu-agent start --require-fips

require-fips: true

sensu-agent start --require-openssl

require-openssl: true

sensu-agent start --trusted-ca-file /path/to/tls/ca.pem

trusted-ca-file: "/path/to/tls/ca.pem"

sensu-agent start --user agent-01

user: "agent-01"

Socket configuration flags

sensu-agent start --disable-sockets

disable-sockets: true

sensu-agent start --socket-host 127.0.0.1

socket-host: "127.0.0.1"

sensu-agent start --socket-port 3030

socket-port: 3030

StatsD configuration flags

sensu-agent start --statsd-disable

statsd-disable: true

sensu-agent start --statsd-event-handlers influxdb,opentsdb
sensu-agent start --statsd-event-handlers influxdb --statsd-event-handlers opentsdb

statsd-event-handlers:
  - influxdb
  - opentsdb

sensu-agent start --statsd-flush-interval 30

statsd-flush-interval: 30

sensu-agent start --statsd-metrics-host 127.0.0.1

statsd-metrics-host: "127.0.0.1"

sensu-agent start --statsd-metrics-port 8125

statsd-metrics-port: 8125

Allow list configuration commands

The allow list includes check and hook commands the agent can execute. Use the allow-list flag to specify the path to the yaml or json file that contains your allow list.

Use these commands to build your allow list configuration file.

args:
- foo

{
  "args": ["foo"]
}

enable_env: true

{
  "enable_env": true
}

exec: "/usr/local/bin/check_memory.sh"

{
  "exec": "/usr/local/bin/check_memory.sh"
}

sha512: 4f926bf4328...

{
  "sha512": "4f926bf4328..."
}

Example allow list configuration file

- exec: /usr/local/bin/check_memory.sh
  args:
  - ""
  sha512: 736ac120323772543fd3a08ee54afdd54d214e58c280707b63ce652424313ef9084ca5b247d226aa09be8f831034ff4991bfb95553291c8b3dc32cad034b4706
  enable_env: true
  foo: bar
- exec: /usr/local/bin/show_process_table.sh
  args:
  - ""
  sha512: 28d61f303136b16d20742268a896bde194cc99342e02cdffc1c2186f81c5adc53f8550635156bebeed7d87a0c19a7d4b7a690f1a337cc4737e240b62b827f78a
- exec: echo-asset.sh
  args:
  - "foo"
  sha512: cce3d16e5881ba829f271df778f9014f7c3659917f7acfd7a60a91bfcabb472eea72f9781194d310388ba046c21790364ad0308a5a897cde50022195ba90924b

[
  {
    "exec": "/usr/local/bin/check_memory.sh",
    "args": [
      ""
    ],
    "sha512": "736ac120323772543fd3a08ee54afdd54d214e58c280707b63ce652424313ef9084ca5b247d226aa09be8f831034ff4991bfb95553291c8b3dc32cad034b4706",
    "enable_env": true,
    "foo": "bar"
  },
  {
    "exec": "/usr/local/bin/show_process_table.sh",
    "args": [
      ""
    ],
    "sha512": "28d61f303136b16d20742268a896bde194cc99342e02cdffc1c2186f81c5adc53f8550635156bebeed7d87a0c19a7d4b7a690f1a337cc4737e240b62b827f78a"
  },
  {
    "exec": "echo-asset.sh",
    "args": [
      "foo"
    ],
    "sha512": "cce3d16e5881ba829f271df778f9014f7c3659917f7acfd7a60a91bfcabb472eea72f9781194d310388ba046c21790364ad0308a5a897cde50022195ba90924b"
  }
]

Configuration via environment variables

Instead of using configuration flags, you can use environment variables to configure your Sensu agent. Each agent configuration flag has an associated environment variable. You can also create your own environment variables, as long as you name them correctly and save them in the correct place. Here’s how.

1. Create the files from which the sensu-agent service configured by our supported packages will read environment variables: /etc/default/sensu-agent for Debian/Ubuntu systems or /etc/sysconfig/sensu-agent for RHEL/CentOS systems.

   sudo touch /etc/default/sensu-agent

   sudo touch /etc/sysconfig/sensu-agent

2. Make sure the environment variable is named correctly. All environment variables that control Sensu agent configuration begin with SENSU_.

   To rename a configuration flag you wish to specify as an environment variable, prepend SENSU_, convert dashes to underscores, and capitalize all letters. For example, the environment variable for the flag api-host is SENSU_API_HOST.

   For a custom environment variable, you do not have to prepend SENSU. For example, TEST_VAR_1 is a valid custom environment variable name.

3. Add the environment variable to the environment file (/etc/default/sensu-agent for Debian/Ubuntu systems or /etc/sysconfig/sensu-agent for RHEL/CentOS systems).

   In this example, the api-host flag is configured as an environment variable and set to "0.0.0.0":

   echo 'SENSU_API_HOST="0.0.0.0"' | sudo tee -a /etc/default/sensu-agent

   echo 'SENSU_API_HOST="0.0.0.0"' | sudo tee -a /etc/sysconfig/sensu-agent

4. Restart the sensu-agent service so these settings can take effect.

   sudo systemctl restart sensu-agent

   sudo systemctl restart sensu-agent

Format for label and annotation environment variables

To use labels and annotations as environment variables in your check and plugin configurations, you must use a specific format when you create the environment variables.

For example, to create the labels "region": "us-east-1" and "type": "website" as an environment variable:

echo 'SENSU_LABELS='{"region": "us-east-1", "type": "website"}'' | sudo tee -a /etc/default/sensu-agent

echo 'SENSU_LABELS='{"region": "us-east-1", "type": "website"}'' | sudo tee -a /etc/sysconfig/sensu-agent

To create the annotations "maintainer": "Team A" and "webhook-url": "https://hooks.slack.com/services/T0000/B00000/XXXXX" as an environment variable:

echo 'SENSU_ANNOTATIONS='{"maintainer": "Team A", "webhook-url": "https://hooks.slack.com/services/T0000/B00000/XXXXX"}'' | sudo tee -a /etc/default/sensu-agent

echo 'SENSU_ANNOTATIONS='{"maintainer": "Team A", "webhook-url": "https://hooks.slack.com/services/T0000/B00000/XXXXX"}'' | sudo tee -a /etc/sysconfig/sensu-agent

Use environment variables with the Sensu agent

Any environment variables you create in /etc/default/sensu-agent (Debian/Ubuntu) or /etc/sysconfig/sensu-agent (RHEL/CentOS) will be available to check and hook commands executed by the Sensu agent. This includes your checks and plugins.

For example, if you create a custom environment variable TEST_VARIABLE in your sensu-agent file, it will be available to use in your check and hook configurations as $TEST_VARIABLE.

The following check example demonstrates how to use a TEST_GITHUB_TOKEN environment variable (set to the token value in the sensu-agent file) in the check command to run a script that pings the GitHub API:

---
type: CheckConfig
api_version: core/v2
metadata:
  name: ping-github-api
  namespace: default
spec:
  command: ping-github-api.sh $TEST_GITHUB_TOKEN
  handlers:
  - slack
  interval: 10
  publish: true
  subscriptions:
  - system

{
  "type": "CheckConfig",
  "api_version": "core/v2",
  "metadata": {
    "name": "ping-github-api",
    "namespace": "default"
  },
  "spec": {
    "command": "ping-github-api.sh $TEST_GITHUB_TOKEN",
    "handlers": [
      "slack"
    ],
    "interval": 10,
    "publish": true,
    "subscriptions": [
      "system"
    ]
  }
}

Use environment variables to specify an HTTP proxy for agent use

If an HTTP proxy is required to access the internet in your compute environment, you may need to configure the Sensu agent to successfully download dynamic runtime assets or execute commands that depend on internet access.

For Sensu agents that require a proxy server, define HTTP_PROXY and HTTPS_PROXY environment variables in your sensu-agent file.

HTTP_PROXY="http://YOUR_PROXY_SERVER:PORT"
HTTPS_PROXY="http://YOUR_PROXY_SERVER:PORT"

You can use the same proxy server URL for HTTP_PROXY and HTTPS_PROXY. The proxy server URL you specify for HTTPS_PROXY does not need to use https://.

After you add the HTTP_PROXY and HTTPS_PROXY environment variables and restart sensu-agent, they will be available to check and hook commands executed by the Sensu agent. You can then use HTTP_PROXY and HTTPS_PROXY to add dynamic runtime assets, run checks, and complete other tasks that typically require an internet connection for your unconnected entities.

Create overrides

Sensu has default settings and limits for certain configuration attributes, like the default log level. Depending on your environment and preferences, you may want to create overrides for these Sensu-specific defaults and limits.

You can create overrides in several ways:

• Command line configuration flag arguments for sensu-agent start.
• Environment variables in /etc/default/sensu-agent (Debian/Ubuntu) or /etc/sysconfig/sensu-agent (RHEL/CentOS).
• Configuration settings in the agent.yml config file.

Sensu applies the following precedence to override settings:

1. Arguments passed to the Sensu agent via command line configuration flags.
2. Environment variables in /etc/default/sensu-agent (Debian/Ubuntu) or /etc/sysconfig/sensu-agent (RHEL/CentOS).
3. Configuration in the agent.yml config file.

For example, if you create overrides using all three methods, the command line configuration flag values will take precedence over the values you specify in /etc/default/sensu-agent or /etc/sysconfig/sensu-agent or the agent.yml config file.

Example override: Log level

The default log level for the Sensu agent is warn.

To override the default and automatically apply a different log level for the agent, add the --log-level command line configuration flag when you start the Sensu agent. For example, to specify debug as the log level:

sensu-agent start --log-level debug

To configure an environment variable for the desired agent log level:

echo 'SENSU_LOG_LEVEL=debug' | sudo tee -a /etc/default/sensu-agent

echo 'SENSU_LOG_LEVEL=debug' | sudo tee -a /etc/sysconfig/sensu-agent

To configure the desired log level in the config file, add this line to agent.yml:

log-level: debug