Events reference
An event is a generic container used by Sensu to provide context to checks and metrics.
The context, called observation data or event data, contains information about the originating entity and the corresponding check or metric result.
An event must contain a status or metrics.
In certain cases, an event can contain both a status and metrics.
These generic containers allow Sensu to handle different types of events in the pipeline.
Because events are polymorphic in nature, it is important to never assume their contents (or lack of content).
Sensu events contain:
entity
scope (required)
- Information about the source of the event, including any attributes defined in the entity specification
check
scope (optional if the metrics
scope is present)
metrics
scope (optional if the check
scope is present)
timestamp
- Time that the event occurred in seconds since the Unix epoch
event_id
- Universally unique identifier (UUID) for the event
Example status-only event
The following example shows the complete resource definition for a status-only event:
type: Event
api_version: core/v2
metadata:
namespace: default
spec:
check:
check_hooks: null
command: check-cpu.rb -w 75 -c 90
duration: 5.058211427
env_vars: null
executed: 1617050501
handlers: []
high_flap_threshold: 0
history:
- executed: 1617050261
status: 0
- executed: 1617050321
status: 0
- executed: 1617050381
status: 0
- executed: 1617050441
status: 0
- executed: 1617050501
status: 0
interval: 60
is_silenced: false
issued: 1617050501
last_ok: 1617050501
low_flap_threshold: 0
metadata:
name: check_cpu
namespace: default
occurrences: 5
occurrences_watermark: 5
output: |
CheckCPU TOTAL OK: total=0.41 user=0.2 nice=0.0 system=0.2 idle=99.59 iowait=0.0 irq=0.0 softirq=0.0 steal=0.0 guest=0.0 guest_nice=0.0
output_metric_format: ""
output_metric_handlers: null
proxy_entity_name: ""
publish: true
round_robin: false
runtime_assets:
- cpu-checks-plugins
- sensu-ruby-runtime
scheduler: memory
secrets: null
state: passing
status: 0
stdin: false
subdue: null
subscriptions:
- system
timeout: 0
total_state_change: 0
ttl: 0
entity:
deregister: false
deregistration: {}
entity_class: agent
last_seen: 1617050501
metadata:
name: sensu-centos
namespace: default
redact:
- password
- passwd
- pass
- api_key
- api_token
- access_key
- secret_key
- private_key
- secret
sensu_agent_version: 6.1.0
subscriptions:
- linux
- entity:sensu-centos
system:
arch: amd64
cloud_provider: ""
hostname: sensu-centos
libc_type: glibc
network:
interfaces:
- addresses:
- 127.0.0.1/8
- ::1/128
name: lo
- addresses:
- 10.0.2.15/24
- fe80::a268:dcce:3be:1c73/64
mac: 08:00:27:8b:c9:3f
name: eth0
- addresses:
- 172.28.128.45/24
- fe80::a00:27ff:feb2:dc46/64
mac: 08:00:27:b2:dc:46
name: eth1
os: linux
platform: centos
platform_family: rhel
platform_version: 7.5.1804
processes: null
vm_role: guest
vm_system: vbox
user: agent
id: 3c3e68f6-6db7-40d3-9b84-4d61817ae559
timestamp: 1617050507
{
"type": "Event",
"api_version": "core/v2",
"metadata": {
"namespace": "default"
},
"spec": {
"check": {
"check_hooks": null,
"command": "check-cpu.rb -w 75 -c 90",
"duration": 5.058211427,
"env_vars": null,
"executed": 1617050501,
"handlers": [],
"high_flap_threshold": 0,
"history": [
{
"executed": 1617050261,
"status": 0
},
{
"executed": 1617050321,
"status": 0
},
{
"executed": 1617050381,
"status": 0
},
{
"executed": 1617050441,
"status": 0
},
{
"executed": 1617050501,
"status": 0
}
],
"interval": 60,
"is_silenced": false,
"issued": 1617050501,
"last_ok": 1617050501,
"low_flap_threshold": 0,
"metadata": {
"name": "check_cpu",
"namespace": "default"
},
"occurrences": 5,
"occurrences_watermark": 5,
"output": "CheckCPU TOTAL OK: total=0.41 user=0.2 nice=0.0 system=0.2 idle=99.59 iowait=0.0 irq=0.0 softirq=0.0 steal=0.0 guest=0.0 guest_nice=0.0\n",
"output_metric_format": "",
"output_metric_handlers": null,
"proxy_entity_name": "",
"publish": true,
"round_robin": false,
"runtime_assets": [
"cpu-checks-plugins",
"sensu-ruby-runtime"
],
"scheduler": "memory",
"secrets": null,
"state": "passing",
"status": 0,
"stdin": false,
"subdue": null,
"subscriptions": [
"system"
],
"timeout": 0,
"total_state_change": 0,
"ttl": 0
},
"entity": {
"deregister": false,
"deregistration": {},
"entity_class": "agent",
"last_seen": 1617050501,
"metadata": {
"name": "sensu-centos",
"namespace": "default"
},
"redact": [
"password",
"passwd",
"pass",
"api_key",
"api_token",
"access_key",
"secret_key",
"private_key",
"secret"
],
"sensu_agent_version": "6.1.0",
"subscriptions": [
"linux",
"entity:sensu-centos"
],
"system": {
"arch": "amd64",
"cloud_provider": "",
"hostname": "sensu-centos",
"libc_type": "glibc",
"network": {
"interfaces": [
{
"addresses": [
"127.0.0.1/8",
":1/128"
],
"name": "lo"
},
{
"addresses": [
"10.0.2.15/24",
"fe80::a268:dcce:3be:1c73/64"
],
"mac": "08:00:27:8b:c9:3f",
"name": "eth0"
},
{
"addresses": [
"172.28.128.45/24",
"fe80::a00:27ff:feb2:dc46/64"
],
"mac": "08:00:27:b2:dc:46",
"name": "eth1"
}
]
},
"os": "linux",
"platform": "centos",
"platform_family": "rhel",
"platform_version": "7.5.1804",
"processes": null,
"vm_role": "guest",
"vm_system": "vbox"
},
"user": "agent"
},
"id": "3c3e68f6-6db7-40d3-9b84-4d61817ae559",
"timestamp": 1617050507
}
}
Example status-only event from the Sensu API
Sensu sends events to the backend in json
format, without the outer-level spec
wrapper or type
and api_version
attributes that are included in the wrapped-json
format.
This is the format that events are in when Sensu sends them to handlers:
{
"check": {
"command": "check-cpu.rb -w 75 -c 90",
"handlers": [],
"high_flap_threshold": 0,
"interval": 60,
"low_flap_threshold": 0,
"publish": true,
"runtime_assets": [
"cpu-checks-plugins",
"sensu-ruby-runtime"
],
"subscriptions": [
"system"
],
"proxy_entity_name": "",
"check_hooks": null,
"stdin": false,
"subdue": null,
"ttl": 0,
"timeout": 0,
"round_robin": false,
"duration": 5.058211427,
"executed": 1617050501,
"history": [
{
"status": 0,
"executed": 1617050261
},
{
"status": 0,
"executed": 1617050321
},
{
"status": 0,
"executed": 1617050381
},
{
"status": 0,
"executed": 1617050441
},
{
"status": 0,
"executed": 1617050501
}
],
"issued": 1617050501,
"output": "CheckCPU TOTAL OK: total=0.4 user=0.2 nice=0.0 system=0.2 idle=99.6 iowait=0.0 irq=0.0 softirq=0.0 steal=0.0 guest=0.0 guest_nice=0.0\n",
"state": "passing",
"status": 0,
"total_state_change": 0,
"last_ok": 1617050501,
"occurrences": 5,
"occurrences_watermark": 5,
"output_metric_format": "",
"output_metric_handlers": null,
"env_vars": null,
"metadata": {
"name": "check_cpu",
"namespace": "default"
},
"secrets": null,
"is_silenced": false,
"scheduler": "memory"
},
"entity": {
"entity_class": "agent",
"system": {
"hostname": "sensu-centos",
"os": "linux",
"platform": "centos",
"platform_family": "rhel",
"platform_version": "7.5.1804",
"network": {
"interfaces": [
{
"name": "lo",
"addresses": [
"127.0.0.1/8",
"::1/128"
]
},
{
"name": "eth0",
"mac": "08:00:27:8b:c9:3f",
"addresses": [
"10.0.2.15/24",
"fe80::a268:dcce:3be:1c73/64"
]
},
{
"name": "eth1",
"mac": "08:00:27:b2:dc:46",
"addresses": [
"172.28.128.45/24",
"fe80::a00:27ff:feb2:dc46/64"
]
}
]
},
"arch": "amd64",
"libc_type": "glibc",
"vm_system": "vbox",
"vm_role": "guest",
"cloud_provider": "",
"processes": null
},
"subscriptions": [
"linux",
"entity:sensu-centos"
],
"last_seen": 1617049781,
"deregister": false,
"deregistration": {},
"user": "agent",
"redact": [
"password",
"passwd",
"pass",
"api_key",
"api_token",
"access_key",
"secret_key",
"private_key",
"secret"
],
"metadata": {
"name": "sensu-centos",
"namespace": "default"
},
"sensu_agent_version": "6.1.0"
},
"id": "3c3e68f6-6db7-40d3-9b84-4d61817ae559",
"metadata": {
"namespace": "default"
},
"timestamp": 1617050507
}
Example metrics-only event
This example shows the complete resource definition for a metrics-only event:
type: Event
api_version: core/v2
metadata:
namespace: default
spec:
entity:
deregister: false
deregistration: {}
entity_class: agent
last_seen: 1552495139
metadata:
name: sensu-go-sandbox
namespace: default
redact:
- password
- passwd
- pass
- api_key
- api_token
- access_key
- secret_key
- private_key
- secret
subscriptions:
- entity:sensu-go-sandbox
system:
arch: amd64
hostname: sensu-go-sandbox
network:
interfaces:
- addresses:
- 127.0.0.1/8
- ::1/128
name: lo
- addresses:
- 10.0.2.15/24
- fe80::5a94:f67a:1bfc:a579/64
mac: 08:00:27:8b:c9:3f
name: eth0
os: linux
platform: centos
platform_family: rhel
platform_version: 7.5.1804
processes: null
user: agent
metrics:
handlers:
- influx-db
points:
- name: sensu-go-sandbox.curl_timings.time_total
tags: []
timestamp: 1552506033
value: 0.005
- name: sensu-go-sandbox.curl_timings.time_namelookup
tags: []
timestamp: 1552506033
value: 0.004
timestamp: 1552506033
event_id: 47ea07cd-1e50-4897-9e6d-09cd39ec5180
{
"type": "Event",
"api_version": "core/v2",
"metadata": {
"namespace": "default"
},
"spec": {
"entity": {
"deregister": false,
"deregistration": {},
"entity_class": "agent",
"last_seen": 1552495139,
"metadata": {
"name": "sensu-go-sandbox",
"namespace": "default"
},
"redact": [
"password",
"passwd",
"pass",
"api_key",
"api_token",
"access_key",
"secret_key",
"private_key",
"secret"
],
"subscriptions": [
"entity:sensu-go-sandbox"
],
"system": {
"arch": "amd64",
"hostname": "sensu-go-sandbox",
"network": {
"interfaces": [
{
"addresses": [
"127.0.0.1/8",
"::1/128"
],
"name": "lo"
},
{
"addresses": [
"10.0.2.15/24",
"fe80::5a94:f67a:1bfc:a579/64"
],
"mac": "08:00:27:8b:c9:3f",
"name": "eth0"
}
]
},
"os": "linux",
"platform": "centos",
"platform_family": "rhel",
"platform_version": "7.5.1804",
"processes": null
},
"user": "agent"
},
"metrics": {
"handlers": [
"influx-db"
],
"points": [
{
"name": "sensu-go-sandbox.curl_timings.time_total",
"tags": [],
"timestamp": 1552506033,
"value": 0.005
},
{
"name": "sensu-go-sandbox.curl_timings.time_namelookup",
"tags": [],
"timestamp": 1552506033,
"value": 0.004
}
]
},
"timestamp": 1552506033,
"event_id": "47ea07cd-1e50-4897-9e6d-09cd39ec5180"
}
}
Example status and metrics event
The following example resource definition for a status and metrics event contains both a check and metrics:
type: Event
api_version: core/v2
metadata:
namespace: default
spec:
check:
check_hooks: null
command: /opt/sensu-plugins-ruby/embedded/bin/metrics-curl.rb -u "http://localhost"
duration: 0.060790838
env_vars: null
executed: 1552506033
handlers: []
high_flap_threshold: 0
history:
- executed: 1552505833
status: 0
- executed: 1552505843
status: 0
interval: 10
is_silenced: false
issued: 1552506033
last_ok: 1552506033
low_flap_threshold: 0
metadata:
name: curl_timings
namespace: default
occurrences: 1
occurrences_watermark: 1
output: |-
sensu-go-sandbox.curl_timings.time_total 0.005 1552506033
sensu-go-sandbox.curl_timings.time_namelookup 0.004
output_metric_format: graphite_plaintext
output_metric_handlers:
- influx-db
proxy_entity_name: ""
publish: true
round_robin: false
runtime_assets: []
state: passing
status: 0
stdin: false
subdue: null
subscriptions:
- entity:sensu-go-sandbox
timeout: 0
total_state_change: 0
ttl: 0
entity:
deregister: false
deregistration: {}
entity_class: agent
last_seen: 1552495139
metadata:
name: sensu-go-sandbox
namespace: default
redact:
- password
- passwd
- pass
- api_key
- api_token
- access_key
- secret_key
- private_key
- secret
subscriptions:
- entity:sensu-go-sandbox
system:
arch: amd64
hostname: sensu-go-sandbox
network:
interfaces:
- addresses:
- 127.0.0.1/8
- ::1/128
name: lo
- addresses:
- 10.0.2.15/24
- fe80::5a94:f67a:1bfc:a579/64
mac: 08:00:27:8b:c9:3f
name: eth0
os: linux
platform: centos
platform_family: rhel
platform_version: 7.5.1804
processes: null
user: agent
metrics:
handlers:
- influx-db
points:
- name: sensu-go-sandbox.curl_timings.time_total
tags: []
timestamp: 1552506033
value: 0.005
- name: sensu-go-sandbox.curl_timings.time_namelookup
tags: []
timestamp: 1552506033
value: 0.004
timestamp: 1552506033
event_id: 431a0085-96da-4521-863f-c38b480701e9
{
"type": "Event",
"api_version": "core/v2",
"metadata": {
"namespace": "default"
},
"spec": {
"check": {
"check_hooks": null,
"command": "/opt/sensu-plugins-ruby/embedded/bin/metrics-curl.rb -u \"http://localhost\"",
"duration": 0.060790838,
"env_vars": null,
"executed": 1552506033,
"handlers": [],
"high_flap_threshold": 0,
"history": [
{
"executed": 1552505833,
"status": 0
},
{
"executed": 1552505843,
"status": 0
}
],
"interval": 10,
"is_silenced": false,
"issued": 1552506033,
"last_ok": 1552506033,
"low_flap_threshold": 0,
"metadata": {
"name": "curl_timings",
"namespace": "default"
},
"occurrences": 1,
"occurrences_watermark": 1,
"output": "sensu-go-sandbox.curl_timings.time_total 0.005 1552506033\nsensu-go-sandbox.curl_timings.time_namelookup 0.004",
"output_metric_format": "graphite_plaintext",
"output_metric_handlers": [
"influx-db"
],
"proxy_entity_name": "",
"publish": true,
"round_robin": false,
"runtime_assets": [],
"state": "passing",
"status": 0,
"stdin": false,
"subdue": null,
"subscriptions": [
"entity:sensu-go-sandbox"
],
"timeout": 0,
"total_state_change": 0,
"ttl": 0
},
"entity": {
"deregister": false,
"deregistration": {},
"entity_class": "agent",
"last_seen": 1552495139,
"metadata": {
"name": "sensu-go-sandbox",
"namespace": "default"
},
"redact": [
"password",
"passwd",
"pass",
"api_key",
"api_token",
"access_key",
"secret_key",
"private_key",
"secret"
],
"subscriptions": [
"entity:sensu-go-sandbox"
],
"system": {
"arch": "amd64",
"hostname": "sensu-go-sandbox",
"network": {
"interfaces": [
{
"addresses": [
"127.0.0.1/8",
"::1/128"
],
"name": "lo"
},
{
"addresses": [
"10.0.2.15/24",
"fe80::5a94:f67a:1bfc:a579/64"
],
"mac": "08:00:27:8b:c9:3f",
"name": "eth0"
}
]
},
"os": "linux",
"platform": "centos",
"platform_family": "rhel",
"platform_version": "7.5.1804",
"processes": null
},
"user": "agent"
},
"metrics": {
"handlers": [
"influx-db"
],
"points": [
{
"name": "sensu-go-sandbox.curl_timings.time_total",
"tags": [],
"timestamp": 1552506033,
"value": 0.005
},
{
"name": "sensu-go-sandbox.curl_timings.time_namelookup",
"tags": [],
"timestamp": 1552506033,
"value": 0.004
}
]
},
"timestamp": 1552506033,
"event_id": "431a0085-96da-4521-863f-c38b480701e9"
}
}
Create events using the Sensu agent
The Sensu agent is a powerful event producer and monitoring automation tool.
You can use Sensu agents to produce events automatically using service checks and metric checks.
Sensu agents can also act as a collector for metrics throughout your infrastructure.
Create events using the events API
You can send events directly to the Sensu pipeline using the events API.
To create an event, send a JSON event definition to the events API PUT endpoint.
If you use the events API to create a new event referencing an entity that does not already exist, the sensu-backend will automatically create a proxy entity in the same namespace when the event is published.
NOTE: An agent cannot belong to, execute checks in, or create events in more than one namespace.
Manage events
You can manage events using the Sensu web UI, events API, and sensuctl command line tool.
View events
To list all events:
To show event details in the default output format (tabular):
sensuctl event info entity-name check-name
With both the list
and info
commands, you can specify an output format using the --format
flag:
sensuctl event info entity-name check-name --format yaml
sensuctl event info entity-name check-name --format wrapped-json
sensuctl event info entity-name check-name --format json
Delete events
To delete an event:
sensuctl event delete entity-name check-name
You can use the --skip-confirm
flag to skip the confirmation step:
sensuctl event delete entity-name check-name --skip-confirm
You should see a confirmation message upon success:
Resolve events
You can use sensuctl to change the status of an event to 0
(OK).
Events resolved by sensuctl include the output message Resolved manually by sensuctl
.
sensuctl event resolve entity-name check-name
You should see a confirmation message upon success:
Use event data
Observability data in events is a powerful tool for automating monitoring workflows.
For example, the state
attribute provides handlers with a high-level description of check status.
Filtering events based on this attribute can help reduce alert fatigue.
State attribute
The state
event attribute adds meaning to the check status:
passing
means the check status is 0
(OK).
failing
means the check status is non-zero (WARNING or CRITICAL).
flapping
indicates an unsteady state in which the check result status (determined based on per-check low and high flap thresholds attributes) is not settling on passing
or failing
according to the flap detection algorithm.
Flapping typically indicates intermittent problems with an entity, provided your low and high flap threshold settings are properly configured.
Although some teams choose to filter out flapping events to reduce unactionable alerts, we suggest sending flapping events to a designated handler for later review.
If you repeatedly observe events in flapping state, Sensu’s per-check flap threshold configuration allows you to adjust the sensitivity of the flap detection algorithm.
Flap detection algorithm
Sensu uses the same flap detection algorithm as Nagios.
Every time you run a check, Sensu records whether the status
value changed since the previous check.
Sensu stores the last 21 status
values and uses them to calculate the percent state change for the entity/check pair.
Then, Sensu’s algorithm applies a weight to these status changes: more recent changes have more value than older changes.
After calculating the weighted total percent state change, Sensu compares it with the low and high flap thresholds set in the check attributes.
- If the entity was not already flapping and the weighted total percent state change for the entity/check pair is greater than or equal to the
high_flap_threshold
setting, the entity has started flapping.
- If the entity was already flapping and the weighted total percent state change for the entity/check pair is less than the
low_flap_threshold
setting, the entity has stopped flapping.
Depending on the result of this comparison, Sensu will trigger the appropriate event filters based on check attributes like event.check.high_flap_threshold
and event.check.low_flap_threshold
.
Occurrences and occurrences watermark
The occurrences
and occurrences_watermark
event attributes give you context about recent events for a given entity and check.
You can use these attributes within event filters to fine-tune incident notifications and reduce alert fatigue.
Starting at 1
, the occurrences
attribute increments for events with the same status as the preceding event (OK, WARNING, CRITICAL, or UNKNOWN) and resets whenever the status changes.
You can use the occurrences
attribute to create a state-change-only filter or an interval filter.
The occurrences_watermark
attribute gives you useful information when looking at events that change status between non-OK (WARNING, CRITICAL, or UNKNOWN) and OK.
For these resolution events, the occurrences_watermark
attribute tells you the number of preceding events with a non-OK status.
Sensu resets occurrences_watermark
to 1
on the first non-OK event.
Within a sequence of only OK or only non-OK events, Sensu increments occurrences_watermark
when the occurrences
attribute is greater than the preceding occurrences_watermark
.
The following table shows the occurrences attributes for a series of example events:
event sequence |
occurrences |
occurrences_watermark |
1. OK event |
occurrences: 1 |
occurrences_watermark: 1 |
2. OK event |
occurrences: 2 |
occurrences_watermark: 2 |
3. WARNING event |
occurrences: 1 |
occurrences_watermark: 1 |
4. WARNING event |
occurrences: 2 |
occurrences_watermark: 2 |
5. WARNING event |
occurrences: 3 |
occurrences_watermark: 3 |
6. CRITICAL event |
occurrences: 1 |
occurrences_watermark: 3 |
7. CRITICAL event |
occurrences: 2 |
occurrences_watermark: 3 |
8. CRITICAL event |
occurrences: 3 |
occurrences_watermark: 3 |
9. CRITICAL event |
occurrences: 4 |
occurrences_watermark: 4 |
10. OK event |
occurrences: 1 |
occurrences_watermark: 4 |
11. CRITICAL event |
occurrences: 1 |
occurrences_watermark: 1 |
Events specification
Top-level attributes
type |
|
description |
Top-level attribute that specifies the sensuctl create resource type. Events should always be type Event . |
required |
Required for events in wrapped-json or yaml format for use with sensuctl create . |
type |
String |
example |
|
api_version |
|
description |
Top-level attribute that specifies the Sensu API group and version. For events in this version of Sensu, api_version should always be core/v2 . |
required |
Required for events in wrapped-json or yaml format for use with sensuctl create . |
type |
String |
example |
{
"api_version": "core/v2"
}
|
metadata |
|
description |
Top-level scope that contains the event namespace and created_by field. The metadata map is always at the top level of the check definition. This means that in wrapped-json and yaml formats, the metadata scope occurs outside the spec scope. See the metadata attributes for details. |
required |
Required for events in wrapped-json or yaml format for use with sensuctl create . |
type |
Map of key-value pairs |
example |
metadata:
namespace: default
created_by: admin
{
"metadata": {
"namespace": "default",
"created_by": "admin"
}
}
|
spec |
|
description |
Top-level map that includes the event spec attributes. |
required |
Required for events in wrapped-json or yaml format for use with sensuctl create . |
type |
Map of key-value pairs |
example |
spec:
check:
check_hooks:
command: /opt/sensu-plugins-ruby/embedded/bin/metrics-curl.rb -u "http://localhost"
duration: 0.060790838
env_vars:
executed: 1552506033
handlers: []
high_flap_threshold: 0
history:
- executed: 1552505833
status: 0
- executed: 1552505843
status: 0
interval: 10
is_silenced: true
issued: 1552506033
last_ok: 1552506033
low_flap_threshold: 0
metadata:
name: curl_timings
namespace: default
occurrences: 1
occurrences_watermark: 1
silenced:
- webserver:*
output: |-
sensu-go-sandbox.curl_timings.time_total 0.005 1552506033
sensu-go-sandbox.curl_timings.time_namelookup 0.004
output_metric_format: graphite_plaintext
output_metric_handlers:
- influx-db
proxy_entity_name: ''
publish: true
round_robin: false
runtime_assets: []
state: passing
status: 0
stdin: false
subdue:
subscriptions:
- entity:sensu-go-sandbox
timeout: 0
total_state_change: 0
ttl: 0
entity:
deregister: false
deregistration: {}
entity_class: agent
last_seen: 1552495139
metadata:
name: sensu-go-sandbox
namespace: default
redact:
- password
- passwd
- pass
- api_key
- api_token
- access_key
- secret_key
- private_key
- secret
subscriptions:
- entity:sensu-go-sandbox
system:
arch: amd64
hostname: sensu-go-sandbox
network:
interfaces:
- addresses:
- 127.0.0.1/8
- "::1/128"
name: lo
- addresses:
- 10.0.2.15/24
- fe80::5a94:f67a:1bfc:a579/64
mac: '08:00:27:8b:c9:3f'
name: eth0
os: linux
platform: centos
platform_family: rhel
platform_version: 7.5.1804
processes:
user: agent
metrics:
handlers:
- influx-db
points:
- name: sensu-go-sandbox.curl_timings.time_total
tags: []
timestamp: 1552506033
value: 0.005
- name: sensu-go-sandbox.curl_timings.time_namelookup
tags: []
timestamp: 1552506033
value: 0.004
timestamp: 1552506033
event_id: 431a0085-96da-4521-863f-c38b480701e9
{
"spec": {
"check": {
"check_hooks": null,
"command": "/opt/sensu-plugins-ruby/embedded/bin/metrics-curl.rb -u \"http://localhost\"",
"duration": 0.060790838,
"env_vars": null,
"executed": 1552506033,
"handlers": [],
"high_flap_threshold": 0,
"history": [
{
"executed": 1552505833,
"status": 0
},
{
"executed": 1552505843,
"status": 0
}
],
"interval": 10,
"is_silenced": true,
"issued": 1552506033,
"last_ok": 1552506033,
"low_flap_threshold": 0,
"metadata": {
"name": "curl_timings",
"namespace": "default"
},
"occurrences": 1,
"occurrences_watermark": 1,
"silenced": [
"webserver:*"
],
"output": "sensu-go-sandbox.curl_timings.time_total 0.005 1552506033\nsensu-go-sandbox.curl_timings.time_namelookup 0.004",
"output_metric_format": "graphite_plaintext",
"output_metric_handlers": [
"influx-db"
],
"proxy_entity_name": "",
"publish": true,
"round_robin": false,
"runtime_assets": [],
"state": "passing",
"status": 0,
"stdin": false,
"subdue": null,
"subscriptions": [
"entity:sensu-go-sandbox"
],
"timeout": 0,
"total_state_change": 0,
"ttl": 0
},
"entity": {
"deregister": false,
"deregistration": {},
"entity_class": "agent",
"last_seen": 1552495139,
"metadata": {
"name": "sensu-go-sandbox",
"namespace": "default"
},
"redact": [
"password",
"passwd",
"pass",
"api_key",
"api_token",
"access_key",
"secret_key",
"private_key",
"secret"
],
"subscriptions": [
"entity:sensu-go-sandbox"
],
"system": {
"arch": "amd64",
"hostname": "sensu-go-sandbox",
"network": {
"interfaces": [
{
"addresses": [
"127.0.0.1/8",
"::1/128"
],
"name": "lo"
},
{
"addresses": [
"10.0.2.15/24",
"fe80::5a94:f67a:1bfc:a579/64"
],
"mac": "08:00:27:8b:c9:3f",
"name": "eth0"
}
]
},
"os": "linux",
"platform": "centos",
"platform_family": "rhel",
"platform_version": "7.5.1804",
"processes": null
},
"user": "agent"
},
"metrics": {
"handlers": [
"influx-db"
],
"points": [
{
"name": "sensu-go-sandbox.curl_timings.time_total",
"tags": [],
"timestamp": 1552506033,
"value": 0.005
},
{
"name": "sensu-go-sandbox.curl_timings.time_namelookup",
"tags": [],
"timestamp": 1552506033,
"value": 0.004
}
]
},
"timestamp": 1552506033,
"event_id": "431a0085-96da-4521-863f-c38b480701e9"
}
}
|
namespace |
|
description |
Sensu RBAC namespace that this event belongs to. |
required |
false |
type |
String |
default |
default |
example |
{
"namespace": "production"
}
|
created_by |
|
description |
Username of the Sensu user who created the event or last updated the event. Sensu automatically populates the created_by field when the event is created or updated. |
required |
false |
type |
String |
example |
{
"created_by": "admin"
}
|
Spec attributes
timestamp |
|
description |
Time that the event occurred. In seconds since the Unix epoch. |
required |
false |
type |
Integer |
default |
Time that the event occurred |
example |
{
"timestamp": 1522099512
}
|
event_id |
|
description |
Universally unique identifier (UUID) for the event. |
required |
false |
type |
String |
example |
event_id: 431a0085-96da-4521-863f-c38b480701e9
{
"event_id": "431a0085-96da-4521-863f-c38b480701e9"
}
|
entity |
|
description |
Entity attributes from the originating entity (agent or proxy). If you use the events API to create a new event referencing an entity that does not already exist, the sensu-backend will automatically create a proxy entity when the event is published. |
type |
Map |
required |
true |
example |
entity:
deregister: false
deregistration: {}
entity_class: agent
last_seen: 1552495139
metadata:
name: sensu-go-sandbox
namespace: default
redact:
- password
- passwd
- pass
- api_key
- api_token
- access_key
- secret_key
- private_key
- secret
subscriptions:
- entity:sensu-go-sandbox
system:
arch: amd64
hostname: sensu-go-sandbox
network:
interfaces:
- addresses:
- 127.0.0.1/8
- "::1/128"
name: lo
- addresses:
- 10.0.2.15/24
- fe80::5a94:f67a:1bfc:a579/64
mac: '08:00:27:8b:c9:3f'
name: eth0
os: linux
platform: centos
platform_family: rhel
platform_version: 7.5.1804
user: agent
{
"entity": {
"deregister": false,
"deregistration": {},
"entity_class": "agent",
"last_seen": 1552495139,
"metadata": {
"name": "sensu-go-sandbox",
"namespace": "default"
},
"redact": [
"password",
"passwd",
"pass",
"api_key",
"api_token",
"access_key",
"secret_key",
"private_key",
"secret"
],
"subscriptions": [
"entity:sensu-go-sandbox"
],
"system": {
"arch": "amd64",
"hostname": "sensu-go-sandbox",
"network": {
"interfaces": [
{
"addresses": [
"127.0.0.1/8",
"::1/128"
],
"name": "lo"
},
{
"addresses": [
"10.0.2.15/24",
"fe80::5a94:f67a:1bfc:a579/64"
],
"mac": "08:00:27:8b:c9:3f",
"name": "eth0"
}
]
},
"os": "linux",
"platform": "centos",
"platform_family": "rhel",
"platform_version": "7.5.1804"
},
"user": "agent"
}
}
|
check |
|
description |
Check definition used to create the event and information about the status and history of the event. The check scope includes attributes described in the event specification and the check specification. |
type |
Map |
required |
true |
example |
check:
check_hooks:
command: /opt/sensu-plugins-ruby/embedded/bin/metrics-curl.rb -u "http://localhost"
duration: 0.060790838
env_vars:
executed: 1552506033
handlers: []
high_flap_threshold: 0
history:
- executed: 1552505833
status: 0
- executed: 1552505843
status: 0
interval: 10
is_silenced: true
issued: 1552506033
last_ok: 1552506033
low_flap_threshold: 0
metadata:
name: curl_timings
namespace: default
occurrences: 1
occurrences_watermark: 1
silenced:
- webserver:*
output: sensu-go-sandbox.curl_timings.time_total 0.005
output_metric_format: graphite_plaintext
output_metric_handlers:
- influx-db
proxy_entity_name: ''
publish: true
round_robin: false
runtime_assets: []
state: passing
status: 0
stdin: false
subdue:
subscriptions:
- entity:sensu-go-sandbox
timeout: 0
total_state_change: 0
ttl: 0
{
"check": {
"check_hooks": null,
"command": "/opt/sensu-plugins-ruby/embedded/bin/metrics-curl.rb -u \"http://localhost\"",
"duration": 0.060790838,
"env_vars": null,
"executed": 1552506033,
"handlers": [],
"high_flap_threshold": 0,
"history": [
{
"executed": 1552505833,
"status": 0
},
{
"executed": 1552505843,
"status": 0
}
],
"interval": 10,
"is_silenced": true,
"issued": 1552506033,
"last_ok": 1552506033,
"low_flap_threshold": 0,
"metadata": {
"name": "curl_timings",
"namespace": "default"
},
"occurrences": 1,
"occurrences_watermark": 1,
"silenced": [
"webserver:*"
],
"output": "sensu-go-sandbox.curl_timings.time_total 0.005",
"output_metric_format": "graphite_plaintext",
"output_metric_handlers": [
"influx-db"
],
"proxy_entity_name": "",
"publish": true,
"round_robin": false,
"runtime_assets": [],
"state": "passing",
"status": 0,
"stdin": false,
"subdue": null,
"subscriptions": [
"entity:sensu-go-sandbox"
],
"timeout": 0,
"total_state_change": 0,
"ttl": 0
}
}
|
metrics |
|
description |
Metrics collected by the entity in Sensu metric format. See the metrics attributes. |
type |
Map |
required |
false |
example |
metrics:
handlers:
- influx-db
points:
- name: sensu-go-sandbox.curl_timings.time_total
tags: []
timestamp: 1552506033
value: 0.005
- name: sensu-go-sandbox.curl_timings.time_namelookup
tags: []
timestamp: 1552506033
value: 0.004
{
"metrics": {
"handlers": [
"influx-db"
],
"points": [
{
"name": "sensu-go-sandbox.curl_timings.time_total",
"tags": [],
"timestamp": 1552506033,
"value": 0.005
},
{
"name": "sensu-go-sandbox.curl_timings.time_namelookup",
"tags": [],
"timestamp": 1552506033,
"value": 0.004
}
]
}
}
|
Check attributes
Sensu events include a check
scope that contains information about how the event was created, including any attributes defined in the check specification, and information about the event and its history, including the attributes defined below.
duration |
|
description |
Command execution time. In seconds. |
required |
false |
type |
Float |
example |
{
"duration": 1.903135228
}
|
executed |
|
description |
Time at which the check request was executed. In seconds since the Unix epoch. |
required |
false |
type |
Integer |
example |
{
"executed": 1522100915
}
|
history |
|
description |
Check status history for the last 21 check executions. See history attributes. |
required |
false |
type |
Array |
example |
history:
- executed: 1552505983
status: 0
- executed: 1552505993
status: 0
{
"history": [
{
"executed": 1552505983,
"status": 0
},
{
"executed": 1552505993,
"status": 0
}
]
}
|
issued |
|
description |
Time that the check request was issued. In seconds since the Unix epoch. |
required |
false |
type |
Integer |
example |
|
last_ok |
|
description |
Last time that the check returned an OK status (0 ). In seconds since the Unix epoch. |
required |
false |
type |
Integer |
example |
{
"last_ok": 1552506033
}
|
occurrences |
|
description |
Number of preceding events with the same status as the current event (OK, WARNING, CRITICAL, or UNKNOWN). Starting at 1 , the occurrences attribute increments for events with the same status as the preceding event and resets whenever the status changes. See Use event data for more information. |
required |
false |
type |
Integer greater than 0 |
example |
|
occurrences_watermark |
|
description |
For incident and resolution events, the number of preceding events with an OK status (for incident events) or non-OK status (for resolution events). The occurrences_watermark attribute gives you useful information when looking at events that change status between OK (0 )and non-OK (1 -WARNING, 2 -CRITICAL, or UNKNOWN).
Sensu resets occurrences_watermark to 1 whenever an event for a given entity and check transitions between OK and non-OK. Within a sequence of only OK or only non-OK events, Sensu increments occurrences_watermark only when the occurrences attribute is greater than the preceding occurrences_watermark . See Use event data for more information. |
required |
false |
type |
Integer greater than 0 |
example |
{
"occurrences_watermark": 1
}
|
is_silenced |
|
description |
If true , the event was silenced at the time of processing. Otherwise, false . If true , the event.Check definition will also list the silenced entries that match the event in the silenced array. |
required |
false |
type |
Boolean |
example |
{
"is_silenced": "true"
}
|
silenced |
|
description |
Array of silencing entries that match the event. The silenced attribute is only present for events if one or more silencing entries matched the event at time of processing. If the silenced attribute is not present in an event, the event was not silenced at the time of processing. |
required |
false |
type |
Array |
example |
{
"silenced": [
"webserver:*"
]
}
|
output |
|
description |
Output from the execution of the check command. |
required |
false |
type |
String |
example |
output: "sensu-go-sandbox.curl_timings.time_total 0.005
{
"output": "sensu-go-sandbox.curl_timings.time_total 0.005"
}
|
state |
|
description |
State of the check: passing (status 0 ), failing (status other than 0 ), or flapping . You can use the low_flap_threshold and high_flap_threshold check attributes to configure flapping state detection. |
required |
false |
type |
String |
example |
|
status |
|
description |
Exit status code produced by the check.0 indicates “OK”1 indicates “WARNING”2 indicates “CRITICAL” Exit status codes other than 0 , 1 , or 2 indicate an “UNKNOWN” or custom status. |
required |
false |
type |
Integer |
example |
|
total_state_change |
|
description |
Total state change percentage for the check’s history. |
required |
false |
type |
Integer |
example |
{
"total_state_change": 0
}
|
History attributes
executed |
|
description |
Time at which the check request was executed. In seconds since the Unix epoch. |
required |
false |
type |
Integer |
example |
{
"executed": 1522100915
}
|
status |
|
description |
Exit status code produced by the check.0 indicates “OK”1 indicates “WARNING”2 indicates “CRITICAL” Exit status codes other than 0 , 1 , or 2 indicate an “UNKNOWN” or custom status. |
required |
false |
type |
Integer |
example |
|
Metrics attributes
handlers |
|
description |
Array of Sensu handlers to use for events created by the check. Each array item must be a string. |
required |
false |
type |
Array |
example |
{
"handlers": [
"influx-db"
]
}
|
points |
|
description |
Metric data points, including a name, timestamp, value, and tags. See points attributes. |
required |
false |
type |
Array |
example |
points:
- name: sensu-go-sandbox.curl_timings.time_total
tags:
- name: response_time_in_ms
value: '101'
timestamp: 1552506033
value: 0.005
- name: sensu-go-sandbox.curl_timings.time_namelookup
tags:
- name: namelookup_time_in_ms
value: '57'
timestamp: 1552506033
value: 0.004
{
"points": [
{
"name": "sensu-go-sandbox.curl_timings.time_total",
"tags": [
{
"name": "response_time_in_ms",
"value": "101"
}
],
"timestamp": 1552506033,
"value": 0.005
},
{
"name": "sensu-go-sandbox.curl_timings.time_namelookup",
"tags": [
{
"name": "namelookup_time_in_ms",
"value": "57"
}
],
"timestamp": 1552506033,
"value": 0.004
}
]
}
|
Points attributes
name |
|
description |
Metric name in the format $entity.$check.$metric where $entity is the entity name, $check is the check name, and $metric is the metric name. |
required |
false |
type |
String |
example |
name: sensu-go-sandbox.curl_timings.time_total
{
"name": "sensu-go-sandbox.curl_timings.time_total"
}
|
tags |
|
description |
Optional tags to include with the metric. Each element of the array must be a hash that contains two key value pairs: the name of the tag and the value . Both values of the pairs must be strings. |
required |
false |
type |
Array |
example |
tags:
- name: response_time_in_ms
value: '101'
{
"tags": [
{
"name": "response_time_in_ms",
"value": "101"
}
]
}
|
timestamp |
|
description |
Time at which the metric was collected. In seconds since the Unix epoch. |
required |
false |
type |
Integer |
example |
{
"timestamp": 1552506033
}
|
value |
|
description |
Metric value. |
required |
false |
type |
Float |
example |
|