Bytes field format uses binary units while showing decimal unit labels

[leeway23]

Kibana 4.5 is given file sizes in bytes, and the functionality should conveniently translate this number of many digits into the common KB, MB, GB nomenclature using decimal units. The problem is, Kibana uses binary units while showing decimal unit labels KB, MB, GB, TB, PB. This results in misleading and confusing information (see this graphic)

There are two options how to fix this issue (both could be implemented):

1. use decimal units assigning decimal labels (KB, MB, GB)
2. use binary units as now while fixing the labels to KiB, MiB, GiB, TiB, PiB etc.

Currently 719373000 bytes gets translated into "685.7MB" instead of "685.7MiB" or "719.4MB".

[Bargs]

Agreed, however this isn't a quick fix since it's an issue in the upstream library we use for number formatting: adamwdraper/Numeral-js#312

[leeway23]

Hi All,

For your parameter e.g. number of bytes of memory used called ‘avgRSS’ one can create a scripted field in Kibana with the following formula:

name of the scripted field: avgRSS_main_reco
Type : Bytes
Script: doc[‘avgRSS'].value * pow(1.024, floor(ln(doc['avgRSS'].value)/ln(1000)))

This corrects for the wrong binary conversion factor finally using decimal one in line with the decimal unit labels.

It carries with it the inconvenience, that one has to be alert in case Kibana fixes the issue in the future. but as far as I have heard, this is not going to happen anytime soon (problem located in one of the upstream libraries).

Cheers,
J.

PS: for general n-core jobs e.g.: (doc[‘avgRSS'].value/doc['ncores'].value) * pow(1.024, ln(doc['avgRSS'].value/doc['ncores'].value)/ln(1000))

[Prazzy]

This has been fixed in upstream library.

"Breaking change / Feature: Bytes are now formatted as: b (base 1000) and ib (base 1024)"

So this needs to be upgraded in Kibana. I tried myself to upgrade and it's working as expected.

--- a/package.json
+++ b/package.json
@@ -68,7 +68,7 @@
     "@spalger/filesaver": "1.1.2",
     "@spalger/leaflet-draw": "0.2.3",
     "@spalger/leaflet-heat": "0.1.3",
-    "@spalger/numeral": "^2.0.0",
+    "numeral": "^2.0.4",
     "@spalger/test-subj-selector": "0.2.1",
     "@spalger/ui-ace": "0.2.3",
     "angular": "1.4.7",
diff --git a/webpackShims/numeral.js b/webpackShims/numeral.js
index 71326a8..5a6f996 100644
--- a/webpackShims/numeral.js
+++ b/webpackShims/numeral.js
@@ -1 +1,4 @@


-module.exports = require('@spalger/numeral');
+//module.exports = require('@spalger/numeral');
+
+module.exports = require('../node_modules/numeral/min/numeral.min.js');

ib (base 1024)

b (base 1000)

[leeway23]

Would anyone know the Kibana version number from which this fix has been included ?

[jimgoodwin]

@spalger Did we pull in this fix to our fork of numeral.js ?

[spalger]

@jimgoodwin nope, this is not fixed in our fork.

I spent a little time upgrading to the latest version of numeral so we could get support for differentiating between ib and b but unfortunately numeral 2.0 has too many breaking changes that would break existing field formats... This means it's probably time for us to face the reality that we should have never exposed raw numeral patterns in the UI...

[shaharmor]

Hey, I think that 6.X is a good chance for the upgrade of the numeral.js library. (We're looking for the b/ib change as well).

Any chance to get this implemented?

[leeway23]

Hi is this currently fixed in Kibana 6.2.X ???

[wylieconlon]

I think the second option (use binary units as now while fixing the labels to KiB, MiB, GiB, TiB, PiB etc.) is the smallest change, and doesn't sound like a breaking change to me.

[wylieconlon]

I'm working on a fix to this in our numeral fork: #7543

As described there, this changes the b format to use KiB and introduces the bd (bytes decimal) format which would use kB, MB etc for base 1000 numbers.

If we decide to merge that change and upgrade our version of @elastic/numeral-js in a minor version, we may also want to introduce a separate Field Formatter for "bytes decimal."

[wylieconlon]

After some more discussion and investigation, it seems like there are many entities that use the decimal units labels for binary bytes, so 1024 bytes = 1 KB (not kB) and 1024*1024 bytes = 1 MB in many of these. We use this notation for Elasticsearch. https://www.elastic.co/guide/en/elasticsearch/reference/current/common-options.html#byte-units

We should support all the inconsistencies in the real world, which means that I think we need to support three options and clarify this to our users:

• Binary bytes with decimal labels
• Binary bytes with kibi/mebi labels
• Decimal bytes with decimal labels

[elasticmachine]

Pinging @elastic/kibana-app-arch (Team:AppArch)

[wylieconlon]

Closing this issue because it's now supported to use different Kibana numeral formatting patterns for all options discussed.