Secure Sensu
As with any piece of software, it is critical to minimize any attack surface the software exposes. Sensu is no different.
This reference describes the components you need to secure to make Sensu production-ready, including etcd peer communication, the Sensu API and web UI, and Sensu agent-to-server communication. It also describes agent mutual transport layer security (mTLS) authentication, which is required for secrets management.
Before you can secure Sensu, you must generate the certificates you will need. After you generate certificates, follow this reference to secure Sensu for production.
Secure etcd peer communication
WARNING: You must update the default configuration for Sensu’s embedded etcd with an explicit, non-default configuration to secure etcd communication in transit. If you do not properly configure secure etcd communication, your Sensu configuration will be vulnerable to unauthorized manipulation via etcd client connections.
To properly secure etcd communication, replace the default parameter values in your backend store configuration in /etc/sensu/backend.yml
as follows:
-
Replace the placeholder with the path to your certificate and key for the
etcd-cert-file
andetcd-key-file
to secure client communication:etcd-cert-file: "/path/to/your/cert" etcd-key-file: "/path/to/your/key"
-
Replace the placeholder with the path to your certificate and key for the
etcd-peer-cert-file
andetcd-peer-key-file
to secure cluster communication:etcd-peer-cert-file: "/path/to/your/peer/cert" etcd-peer-key-file: "/path/to/your/peer/key"
-
Replace the placeholder with the path to your certificate and key for the
etcd-trusted-ca-file
andetcd-peer-trusted-ca-file
to secure communication with the etcd client server and between etcd cluster members:etcd-trusted-ca-file: "/path/to/your/ca/file" etcd-peer-trusted-ca-file: "/path/to/your/peer/ca/file"
-
Add non-default values for
etcd-listen-client-urls
,etcd-listen-peer-urls
, andetcd-initial-advertise-client-urls
:etcd-listen-client-urls: "https://localhost:2379" etcd-listen-peer-urls: "https://localhost:2380" etcd-initial-advertise-peer-urls: "https://localhost:2380"
-
Set
etcd-client-cert-auth
andetcd-peer-client-cert-auth
totrue
to ensure that etcd only allows connections from clients and peers that present a valid, trusted certificate:etcd-client-cert-auth: "true" etcd-peer-client-cert-auth: "true"
Because etcd does not require authentication by default, you must set the
etcd-client-cert-auth
andetcd-peer-client-cert-auth
parameters totrue
to secure Sensu’s embedded etcd datastore against unauthorized access.
NOTE: The Sensu backend reference includes more information about each etcd store parameter.
Secure the Sensu API and web UI
The Sensu Go Agent API, HTTP API, and web UI use a common stanza in /etc/sensu/backend.yml
to provide the certificate, key, and CA file needed to provide secure communication.
NOTE: By changing these parameters, the server will communicate using transport layer security (TLS) and expect agents that connect to it to use the WebSocket secure protocol. For communication to continue, you must complete the configuration in this section and in the Sensu agent-to-server communication section.
Configure the following backend secure sockets layer (SSL) attributes in /etc/sensu/backend.yml
:
-
Replace the placeholders with the paths to your certificate and key files for the
cert-file
,key-file
, andtrusted-ca-file
parameters:cert-file: "/path/to/ssl/cert.pem" key-file: "/path/to/ssl/key.pem" trusted-ca-file: "/path/to/trusted-certificate-authorities.pem" insecure-skip-tls-verify: false
-
Set the
insecure-skip-tls-verify
parameter tofalse
:insecure-skip-tls-verify: false
-
When you provide these cert-file and key-file parameters, the agent WebSocket API and HTTP API will serve requests over SSL/TLS (https). For this reason, you must also specify
https://
schema for theapi-url
parameter for backend API configuration:api-url: "https://localhost:8080"
After you restart the sensu-backend
service, the parameters will load and you will able to access the web UI at https://localhost:3000.
Configuring these attributes will also ensure that agents can communicate securely.
NOTE: The Sensu backend reference includes more information about each API and web UI security configuration parameter.
Specify separate web UI certificate and key
You can specify a certificate and key for the web UI separately from the API.
To do this, add the dashboard-cert-file
and dashboard-key-file
parameters for backend SSL configuration in /etc/sensu/backend.yml
:
dashboard-cert-file: "/path/to/ssl/cert.pem"
dashboard-key-file: "/path/to/ssl/key.pem"
NOTE: The Sensu backend reference includes more information about the and
web UI configuration parameter.
Secure Sensu agent-to-server communication
NOTE: If you change the agent configuration to communicate via WebSocket Secure protocol, the agent will no longer communicate over a plaintext connection. For communication to continue, you must complete the steps in this section and secure the Sensu API and web UI.
By default, an agent uses the insecure ws://
transport.
Here’s an example for agent configuration in /etc/sensu/agent.yml
:
backend-url:
- "ws://127.0.0.1:8081"
To use WebSocket over SSL/TLS (wss), change the backend-url
value to the wss://
schema in /etc/sensu/agent.yml
:
backend-url:
- "wss://127.0.0.1:8081"
The agent will connect to Sensu backends over wss. Remember, if you change the configuration to wss, plaintext communication will not be possible.
You can also provide a trusted CA as part of the agent configuration.
If you will start the agent via sensu-agent start
, pass the --trusted-ca-file
flag with the start command.
Otherwise, include the trusted-ca-file
parameter in the agent configuration in /etc/sensu/agent.yml
:
trusted-ca-file: "/path/to/trusted-certificate-authorities.pem"
NOTE: If you are creating a Sensu cluster, every cluster member needs to be present in the configuration. See Run a Sensu cluster for more information about how to configure agents for a clustered configuration.
Configure Sensu agent mTLS authentication
COMMERCIAL FEATURE: Access client mutual transport layer security (mTLS) authentication in the packaged Sensu Go distribution. For more information, see Get started with commercial features.
By default, Sensu agents require username and password authentication to communicate with Sensu backends. For Sensu’s default user credentials and details about configuring Sensu role-based access control (RBAC), see the RBAC reference.
Alternatively, Sensu agents can use mTLS for authenticating to the backend websocket transport.
When agent mTLS authentication is enabled, agents do not need to send password credentials to backends when they connect.
To use secrets management, Sensu agents must be secured with mTLS.
In addition, when using mTLS authentication, agents do not require an explicit user in Sensu.
Sensu agents default to authenticating as the agent
user and using permissions granted to the system:agents
group by the system:agents
cluster role and cluster role binding.
You can still bind agents to a specific user when the system:agents
group is problematic.
For this use case, create a user that matches the Common Name (CN) of the agent’s certificate.
NOTE: Sensu agents need to be able to create events in the agent’s namespace.
To ensure that agents with incorrect CN fields can’t access the backend, remove the default system:agents
group.
To view a certificate’s CN with openssl, run:
openssl x509 -in client.pem -text -noout
The response should be similar to this example:
Certificate:
Data:
Version: 3 (0x2)
Serial Number:
37:57:7b:04:1d:67:63:7b:ff:ae:39:19:5b:55:57:80:41:3c:ec:ff
Signature Algorithm: sha256WithRSAEncryption
Issuer: CN = CA
Validity
Not Before: Sep 26 18:58:00 2019 GMT
Not After : Sep 24 18:58:00 2024 GMT
Subject: CN = client
...
The Subject:
field indicates the certificate’s CN is client
, so to bind the agent to a particular user in Sensu, create a user called client
.
To enable agent mTLS authentication:
-
Create and distribute new certificates and keys according to the Generate certificates guide.
-
With the TLS certificate and key are in place, update the agent configuration using
cert-file
andkey-file
security configuration flags. -
After you create backend and agent certificates, modify the backend configuration:
agent-auth-cert-file: "/path/to/backend-1.pem" agent-auth-key-file: "/path/to/backend-1-key.pem" agent-auth-trusted-ca-file: "/path/to/ca.pem"
-
Modify the agent configuration:
cert-file: "/path/to/agent.pem" key-file: "/path/to/agent-key.pem" trusted-ca-file: "/path/to/ca.pem"
You can use use certificates for authentication that are distinct from other communication channels used by Sensu, like etcd or the API. However, deployments can also use the same certificates and keys for etcd peer and client communication, the HTTP API, and agent authentication without issues.
Certificate revocation check
The Sensu backend checks certificate revocation list (CRL) and Online Certificate Status Protocol (OCSP) endpoints for agent mTLS, etcd client, and etcd peer connections whose remote sides present X.509 certificates that provide CRL and OCSP revocation information.
Next step: Run a Sensu cluster
Well done! Your Sensu installation should now be secured with TLS. The last step before you deploy Sensu is to set up a Sensu cluster.