Secure Sensu

As with any piece of software, it is critical to minimize any attack surface the software exposes. Sensu is no different.

This reference describes the components you need to secure to make Sensu production-ready, including etcd peer communication, the Sensu API and web UI, and Sensu agent-to-server communication. It also describes agent mutual transport layer security (mTLS) authentication, which is required for secrets management.

Before you can secure Sensu, you must generate the certificates you will need. After you generate certificates, follow this reference to secure Sensu for production.

Secure etcd peer communication

To properly secure etcd communication, replace the default parameter values in your backend store configuration in /etc/sensu/backend.yml as follows:

1. Replace the placeholder with the path to your certificate and key for the etcd-cert-file and etcd-key-file to secure client communication:

   etcd-cert-file: "/etc/sensu/tls/backend-1.pem"
   etcd-key-file: "/etc/sensu/tls/backend-1-key.pem"

2. Replace the placeholder with the path to your certificate and key for the etcd-peer-cert-file and etcd-peer-key-file to secure cluster communication:

   etcd-peer-cert-file: "/etc/sensu/tls/backend-1.pem"
   etcd-peer-key-file: "/etc/sensu/tls/backend-1-key.pem"

3. Replace the placeholder with the path to your ca.pem certificate for the etcd-trusted-ca-file and etcd-peer-trusted-ca-file to secure communication with the etcd client server and between etcd cluster members:

   etcd-trusted-ca-file: "/etc/sensu/tls/ca.pem"
   etcd-peer-trusted-ca-file: "/etc/sensu/tls/ca.pem"

4. Add non-default values for etcd-listen-client-urls, etcd-listen-peer-urls, and etcd-initial-advertise-peer-urls:

   etcd-listen-client-urls: "https://localhost:2379"
   etcd-listen-peer-urls: "https://localhost:2380"
   etcd-initial-advertise-peer-urls: "https://localhost:2380"

   NOTE: If you are securing a cluster, use your backend node IP address instead of localhost in the non-default values for etcd-listen-client-urls, etcd-listen-peer-urls, and etcd-initial-advertise-peer-urls.

5. Set etcd-client-cert-auth and etcd-peer-client-cert-auth to true to ensure that etcd only allows connections from clients and peers that present a valid, trusted certificate:

   etcd-client-cert-auth: "true"
   etcd-peer-client-cert-auth: "true"

   Because etcd does not require authentication by default, you must set the etcd-client-cert-auth and etcd-peer-client-cert-auth parameters to true to secure Sensu’s embedded etcd datastore against unauthorized access.

Secure the Sensu agent API, HTTP API, and web UI

The Sensu Go agent API, HTTP API, and web UI use a common stanza in /etc/sensu/backend.yml to provide the certificate, key, and CA file needed to provide secure communication.

Configure the following backend secure sockets layer (SSL) attributes in /etc/sensu/backend.yml:

1. Replace the placeholders with the paths to your CA root, backend certificate, and backend key files for the trusted-ca-file, cert-file, and key-file parameters:

   trusted-ca-file: "/etc/sensu/tls/ca.pem"
   cert-file: "/etc/sensu/tls/backend-1.pem"
   key-file: "/etc/sensu/tls/backend-1-key.pem"

2. Set the insecure-skip-tls-verify parameter to false:

   insecure-skip-tls-verify: false

3. When you provide these cert-file and key-file parameters, the agent WebSocket API and HTTP API will serve requests over SSL/TLS (https). For this reason, you must also specify https:// schema for the api-url parameter for backend API configuration:

   api-url: "https://localhost:8080"

After you restart the sensu-backend service, the parameters will load and you will able to access the web UI at https://localhost:3000. Configuring these attributes will also ensure that agents can communicate securely.

Specify a separate web UI certificate and key

You can use the same certificates and keys to secure etcd, the HTTP API, and the web UI. However, if you prefer, you can use a separate certificate and key for the web UI (for example, a commercially purchased certificate and key).

To do this, add the dashboard-cert-file and dashboard-key-file parameters for backend SSL configuration in /etc/sensu/backend.yml:

dashboard-cert-file: "/etc/sensu/tls/separate-webui-cert.pem"
dashboard-key-file: "/etc/sensu/tls/separate-webui-key.pem"

Secure Sensu agent-to-server communication

By default, an agent uses the insecure ws:// transport. Here’s an example for agent configuration in /etc/sensu/agent.yml:

backend-url:
  - "ws://127.0.0.1:8081"

To use WebSocket over SSL/TLS (wss), change the backend-url value to the wss:// schema in /etc/sensu/agent.yml:

backend-url:
  - "wss://127.0.0.1:8081"

The agent will connect to Sensu backends over wss. Remember, if you change the configuration to wss, plaintext communication will not be possible.

You can also provide a trusted CA root certificate file as part of the agent configuration (named ca.pem in the example in Generate certificates). If you will start the agent via sensu-agent start, pass the --trusted-ca-file flag with the start command. Otherwise, include the trusted-ca-file parameter in the agent configuration in /etc/sensu/agent.yml:

trusted-ca-file: "/etc/sensu/tls/ca.pem"

Optional: Configure Sensu agent mTLS authentication

By default, Sensu agents require username and password authentication to communicate with Sensu backends. For Sensu’s default user credentials and details about configuring Sensu role-based access control (RBAC), see the RBAC reference.

Alternatively, Sensu agents can use mTLS for authenticating to the backend websocket transport. When agent mTLS authentication is enabled, agents do not need to send password credentials to backends when they connect. To use secrets management, Sensu agents must be secured with mTLS. In addition, when using mTLS authentication, agents do not require an explicit user in Sensu. Sensu agents default to authenticating as the agent user and using permissions granted to the system:agents group by the system:agents cluster role and cluster role binding.

You can still bind agents to a specific user when the system:agents group is problematic. For this use case, create a user that matches the Common Name (CN) of the agent’s certificate.

To view a certificate’s CN with openssl, run:

openssl x509 -in client.pem -text -noout

The response should be similar to this example:

Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number:
            37:57:7b:04:1d:67:63:7b:ff:ae:39:19:5b:55:57:80:41:3c:ec:ff
        Signature Algorithm: sha256WithRSAEncryption
        Issuer: CN = CA
        Validity
            Not Before: Sep 26 18:58:00 2019 GMT
            Not After : Sep 24 18:58:00 2024 GMT
        Subject: CN = client
...

The Subject: field indicates the certificate’s CN is client, so to bind the agent to a particular user in Sensu, create a user called client.

To enable agent mTLS authentication:

1. Create and distribute a new Certificate Authority (CA) root certificate and new agent and backend certificates and keys according to the Generate certificates guide.

2. Add the following parameters and values to the backend configuration:

   agent-auth-cert-file: "/etc/sensu/tls/backend-1.pem"
   agent-auth-key-file: "/etc/sensu/tls/backend-1-key.pem"
   agent-auth-trusted-ca-file: "/etc/sensu/tls/ca.pem"

3. Add the following parameters and values to the agent configuration:

   cert-file: "/etc/sensu/tls/agent.pem"
   key-file: "/etc/sensu/tls/agent-key.pem"
   trusted-ca-file: "/etc/sensu/tls/ca.pem"

You can use use certificates for authentication that are distinct from other communication channels used by Sensu, like etcd or the API. However, deployments can also use the same certificates and keys for etcd peer and client communication, the HTTP API, and agent authentication without issues.

Certificate revocation check

The Sensu backend checks certificate revocation list (CRL) and Online Certificate Status Protocol (OCSP) endpoints for agent mTLS, etcd client, and etcd peer connections whose remote sides present X.509 certificates that provide CRL and OCSP revocation information.

Next step: Run a Sensu cluster

Well done! Your Sensu installation should now be secured with TLS. The last step before you deploy Sensu is to set up a Sensu cluster.