Netdata allows you to send custom notifications to any endpoint you choose.
To configure custom notifications, you will need to customize health_alarm_notify.conf. You can look at the other senders in /usr/libexec/netdata/plugins.d/alarm-notify.sh for examples of how to modify the custom_sender() function in health_alarm_notify.conf. Ensure you follow the instructions of changing any configuration file to persist your configuration.
As with other notifications, you will also need to define the recipient list in DEFAULT_RECIPIENT_CUSTOM and/or the role_recipients_custom array.
The following is a sample custom_sender function in health_alarm_notify.conf, to send an SMS via an imaginary HTTPS endpoint to the SMS gateway:
custom_sender() {
# example human readable SMS
local msg="${host} ${status_message}: ${alarm} ${raised_for}"
# limit it to 160 characters and encode it for use in a URL
urlencode "${msg:0:160}" >/dev/null; msg="${REPLY}"
# a space separated list of the recipients to send alarms to
to="${1}"
for phone in ${to}; do
httpcode=$(docurl -X POST \
--data-urlencode "From=XXX" \
--data-urlencode "To=${phone}" \
--data-urlencode "Body=${msg}" \
-u "${accountsid}:${accounttoken}" \
https://domain.website.com/)
if [ "${httpcode}" = "200" ]; then
info "sent custom notification ${msg} to ${phone}"
sent=$((sent + 1))
else
error "failed to send custom notification ${msg} to ${phone} with HTTP error code ${httpcode}."
fi
done
}
Variables available to the custom_sender:
${to_custom}the list of recipients for the alarm${host}the host generated this event${url_host}same as${host}but URL encoded${unique_id}the unique id of this event${alarm_id}the unique id of the alarm that generated this event${event_id}the incremental id of the event, for this alarm id${when}the timestamp this event occurred${name}the name of the alarm, as given in Netdata health.d entries${url_name}same as${name}but URL encoded${chart}the name of the chart (type.id)${url_chart}same as${chart}but URL encoded${family}the family of the chart${url_family}same as${family}but URL encoded${status}the current status : REMOVED, UNINITIALIZED, UNDEFINED, CLEAR, WARNING, CRITICAL${old_status}the previous status: REMOVED, UNINITIALIZED, UNDEFINED, CLEAR, WARNING, CRITICAL${value}the current value of the alarm${old_value}the previous value of the alarm${src}the line number and file the alarm has been configured${duration}the duration in seconds of the previous alarm state${duration_txt}same as${duration}for humans${non_clear_duration}the total duration in seconds this is/was non-clear${non_clear_duration_txt}same as${non_clear_duration}for humans${units}the units of the value${info}a short description of the alarm${value_string}friendly value (with units)${old_value_string}friendly old value (with units)${image}the URL of an image to represent the status of the alarm${color}a color in #AABBCC format for the alarm${goto_url}the URL the user can click to see the Netdata dashboard${calc_expression}the expression evaluated to provide the value for the alarm${calc_param_values}the value of the variables in the evaluated expression${total_warnings}the total number of alarms in WARNING state on the host${total_critical}the total number of alarms in CRITICAL state on the host
The following are more human friendly:
${alarm}like "name = value units"${status_message}like "needs attention", "recovered", "is critical"${severity}like "Escalated to CRITICAL", "Recovered from WARNING"${raised_for}like "(alarm was raised for 10 minutes)"