Data annotations (aka Chart markers)

[w33ble]

The ability to view annotations on top of visualizations would be highly useful.

The idea comes from my use of CopperEgg, where it would overlay my server ping times with alert events. @simianhacker also wanted this same functionality when plotting power usage, as he wanted to annotate the graphs with specific events (AC on/off, lights, etc).

[simianhacker]

+1000000

[simianhacker]

From Kibana 3

[w33ble]

Chris' annotation use case:

[dtheodorakis]

+1 That would be really great!

[ssoto]

👍
Grafana uses a specified elasticsearch index to store annotations:

POST events-2015.03.10/events/1
{
    "@timestamp": "2015-03-05T14:09:00.000+01:00",
    "tittle": "Annotation tittle",
    "tags" : "issue",
    "text" : "Information related"
}

Then you can configure in the view:

And annotations are shown in histogram charts:

[arnisoph]

+1 for the grafana approach

[arabold]

+1 that sounds super useful! Plus a small API that allows to automatically create new annotations, e.g. through a web hook when deploying or when pushing to a repository.

[johnpickett]

+1 This is crucial to be able and visually correlate system metrics with other event data such as monitoring events, changes, etc...

Grafana seems to be an interesting starting point for this kind of feature.

[maheshnayak1]

+1. A must feature to have in monitoring infrastructure.

[vornet]

+1 Would love to see this

[ycombinator]

+1 from two developers at DevOps Days Chicago. They specifically brought this up as a feature available in Grafana.

[ankopainting]

+1

[cphoover]

+1 Is this even being worked on?

[keyurfaldu]

+1 this is very very useful

[lin11230]

+1

[zdrummond]

+1

[kawikao]

+1. Glaring hole in Kibana. Annotations show events that are a critical part of seeing what happened. A little ironic because Grafana is the time-series tool and Kibana is the event viewer.

[IngaFeick]

+1

[RajeevNambiar]

+100

[mizalewski]

+1

[marcinkubica]

Useful thing it can be.

Conditional annotations? If { query } ( mark { datetime, comment }

Sent from Gmail Mobile

[spalger]

@marcinkubica can you expand on your idea

[ppadovani]

+1 However I would like to add that markers should be able to come from any defined index, and have the ability to define different icons to represent different types of events. For example, a deploy event may have one icon and an incident/Jira ticket might have another. Finally be able to assign multiple marker 'types' to a specified visualization.

[lefebsy]

+1
Data annotations on dashboard could be great to.
Some dataviz tools provide persistent chatiness on dashboards... (maybe a kibana plugin ?)

[pemontto]

👍

[vsellier]

👍
highly expected feature

[spyrospph]

+1 need this too

[fikriauliya]

From the screenshot @simianhacker , it seems it was available in Kibana 3? Is it dropped in Kibana 4?

[Funbit]

+100

[Secespitus]

+1

[billw2012]

+1, using grafana until both this and support for displaying times correctly is implemented. Without them kibana is basically useless for metrics.

[hbaeumer]

+1

[pw64]

+100 i would like to see markers like 'start change XXX' and 'end change XXX', preferably clickable so I can look up the change in my change tool, and ignore (or investigate..!) messages that are between the change interval.

[styfle]

Kibana already stores some data in an index called .kibana or a custom name defined in the config. This index contains at least two different _type documents: index-pattern and config. We could use this index and add a new _type for kibana-marker.

Has anyone started working on an implementation yet?

[andreamaruccia]

+1

[mshadeck]

+1

[Lomexian]

+1

[KieranPeat]

+1 this would be very helpful, as I'm currently considering writing a plugin to do this similar to the grafana approach...

[mcorten]

+1

[styfle]

I would like to work on this since it is a much needed feature, but I will probably need an Elastic Member @w33ble @simianhacker @Bargs to discuss implementation details with. Some questions I need answered:

• What should this feature be called? markers/events/annotations/etc?
• Should this feature wait until VisLib is rewritten or is that far away? Visualization Library Redesign #3456
• Should this extra data be added to the .kibana index or should it be a new index?
• How should it look when adding a marker/event/annotation?
• Should this feature only be available in v6 or should it be backported to other versions of kibana?

[Bargs]

@spalger and @ppisljar could probably answer the question about vislib rewrite, and the other questions best tbh.

[w33ble]

My understanding is that the vislib is on the forefront, but I don't know what the plan is (rewrite vs replacement vs something else).

I believe @ppisljar is the one to chime in on that. Whoever is spearheading that effort is the one you'll want to work with though.

[spalger]

What should this feature be called? markers/events/annotations/etc?

Kibana 3 and Grafana call them annotations, so if there aren't any objections let's just stick with that.

Should this feature wait until VisLib is rewritten or is that far away? #3456

The fate of the vislib isn't set in stone, but for the time being @ppisljar and I are working to make it more maintainable as is. Either way, a good chunk of the work here will be implementing the supplemental query necessary to pull down the annotations.

The logic to load the annotations probably need to be the responsibility of either the SearchSource or SavedVis class. SavedVis currently teaches the SearchSource how to send it's queries, but doesn't actually do the fetching itself. The fetching all happens in the courier and search results are pushed out to subscribers via the SearchSource. This means that the courier is currently the only thing that knows how to talk to elasticsearch and will probably need to be extended if we want to go that path... I'll keep thinking about this, but if you'd like to brainstorm I suggest pinging me on IRC (spalger)

Should this extra data be added to the .kibana index or should it be a new index?

I definitely think that annotation docs should be stored in user indices, not .kibana.

How should it look when adding a marker/event/annotation?

I would probably add an "annotations" section to the data tab of the visualize editor. There, users could create one or more annotation configurations, probably with similar configuration options to other implementations.

Should this feature only be available in v6 or should it be backported to other versions of kibana?

Right now that's dependent on when the feature is ready

[ThomasFlanaghan]

Hi,
Is this still being worked on? Any update?

[simianhacker]

This will be going into the PR #9725 that is currently under review:

[styfle]

@simianhacker Very cool! It looks like the PR just got merged and is labeled v5.4.0 and v6.0.0. So does that mean we have to upgrade our ES nodes to at least 5.4.0 in order to use this new Kibana feature?

[simianhacker]

@styfle Yes

[alexfrancoeur]

@w33ble @simianhacker Can we close this now that time series visual builder has been merged (5.4) with annotation capabilities?

[source-ram]

+1

[alexfrancoeur]

Closing this issue as this feature is now supported in Time Series Visual Builder as of 5.4

[cawoodm]

Broken Link/404
https://www.elastic.co/guide/en/kibana/current/TSVB.html

[drewdaemon]

For any future Googlers who land on this issue, this capability was added to Lens under the name "Annotations" in #169971. See the doc for usage instructions.