[Lens] Document common formulas in product and add formula tutorial

docs/user/dashboard/lens-advanced.asciidoc

@@ -104,7 +104,7 @@ To quickly create many copies of a percentile metric that shows distribution of
 . From the *Chart Type* dropdown, select *Line*.
 +
 [role="screenshot"]
-image::images/lens_advanced_2_1.png[Chart type menu with Line selected]
+image::images/lens_advanced_2_1.png[Chart type menu with Line selected, width=50%]
 
 . From the *Available fields* list, drag and drop *products.price* to the visualization builder.
 
@@ -239,12 +239,11 @@ For each category type that you want to break down, create a filter.
 Change the legend position to the top of the chart.
 
 . From the *Legend* dropdown, select the top position.
-
 +
 [role="screenshot"]
 image::images/lens_advanced_4_1.png[Prices share by category]
 
- Click *Save and return*.
+. Click *Save and return*.
 
 [discrete]
 [[view-the-cumulative-number-of-products-sold-on-weekends]]
@@ -299,7 +298,8 @@ image::images/lens_advanced_5_2.png[Line chart with cumulative sum of orders mad
 [[compare-time-ranges]]
 === Compare time ranges
 
-*Lens* allows you to compare the currently selected time range with historical data using the *Time shift* option.
+*Lens* allows you to compare the currently selected time range with historical data using the *Time shift* option. To calculate the percent
+change, use *Formula*.
 
 Time shifts can be used on any metric. The special shift *previous* will show the time window preceding the currently selected one, spanning the same duration.
 For example, if *Last 7 days* is selected in the time filter, *previous* will show data from 14 days ago to 7 days ago.
@@ -326,9 +326,32 @@ To compare current sales numbers with sales from a week ago, follow these steps:
 .. Click *Time shift*
 
 .. Click the *1 week* option. You can also define custom shifts by typing amount followed by time unit (like *1w* for a one week shift), then hit enter.
-
++
 [role="screenshot"]
-image::images/lens_time_shift.png[Line chart with week-over-week sales comparison]
+image::images/lens_time_shift.png[Line chart with week-over-week sales comparison, width=50%]
+
+. Click *Save and return*.
+
+[float]
+[[compare-time-as-percent]]
+==== Compare time ranges as a percent change
+
+To view the percent change in sales between the current time and the previous week, use a *Formula*:
+
+. Open *Lens*.
+
+. From the *Available fields* list, drag and drop *Records* to the visualization builder.
+
+. Click *Count of Records*, then click *Formula*.
+
+. Type `count() / count(shift='1w') - 1`. To learn more about the formula
+syntax, click *Help*.
+
+. Click *Value format* and select *Percent* with 0 decimals.
+
+. In the *Display name* field, enter `Percent change`, then click *Close*.
+
+. Click *Save and return*.
 
 [discrete]
 [[view-customers-over-time-by-continents]]
@@ -366,18 +389,14 @@ To split the customers count by continent:
 . From the *Available fields* list, drag and drop *geoip.continent_name* to the *Columns* field of the editor.
 +
 [role="screenshot"]
-image::images/lens_advanced_6_1.png[Table with daily customers by continent configuration]
+image::images/lens_advanced_6_1.png[Table with daily customers by continent configuration, width=50%]
 
 . Click *Save and return*.
 
+
 [discrete]
 === Save the dashboard
 
-By default the dashboard attempts to match the palette across panels, but in this case there's no need for that, so it can be disabled.
-
-[role="screenshot"]
-image::images/lens_advanced_7_1.png[Disable palette sync in dashboard]
-
 Now that you have a complete overview of your ecommerce sales data, save the dashboard.
 
 . In the toolbar, click *Save*.

x-pack/plugins/lens/public/indexpattern_datasource/operations/definitions/formula/editor/formula_help.tsx

@@ -64,6 +64,13 @@ function FormulaHelp({
  items: [],
  });
 
+ helpGroups.push({
+ label: i18n.translate('xpack.lens.formulaFrequentlyUsedHeading', {
+ defaultMessage: 'Common formulas',
+ }),
+ items: [],
+ });
+
  helpGroups.push({
  label: i18n.translate('xpack.lens.formulaDocumentation.elasticsearchSection', {
  defaultMessage: 'Elasticsearch',
@@ -78,7 +85,7 @@ function FormulaHelp({
  const availableFunctions = getPossibleFunctions(indexPattern);
 
  // Es aggs
- helpGroups[1].items.push(
+ helpGroups[2].items.push(
  ...availableFunctions
  .filter(
  (key) =>
@@ -104,20 +111,20 @@ function FormulaHelp({
 
  helpGroups.push({
  label: i18n.translate('xpack.lens.formulaDocumentation.columnCalculationSection', {
- defaultMessage: 'Column-wise calculation',
+ defaultMessage: 'Column calculations',
  }),
  description: i18n.translate(
  'xpack.lens.formulaDocumentation.columnCalculationSectionDescription',
  {
  defaultMessage:
- 'These functions will be executed for reach row of the resulting table, using data from cells from other rows as well as the current value.',
+ 'These functions are executed for each row, but are provided with the whole column as context. This is also known as a window function.',
  }
  ),
  items: [],
  });
 
  // Calculations aggs
- helpGroups[2].items.push(
+ helpGroups[3].items.push(
  ...availableFunctions
  .filter(
  (key) =>
@@ -170,7 +177,7 @@ function FormulaHelp({
  });
  }, [indexPattern]);
 
- helpGroups[3].items.push(
+ helpGroups[4].items.push(
  ...tinymathFns.map(({ label, description, examples }) => {
  return {
  label,
@@ -312,9 +319,9 @@ round(100 * moving_average(
 \`\`\`
 
 Elasticsearch functions take a field name, which can be in quotes. \`sum(bytes)\` is the same
-as \`sum("bytes")\`.
+as \`sum('bytes')\`.
 
-Some functions take named arguments, like moving_average(count(), window=5)
+Some functions take named arguments, like \`moving_average(count(), window=5)\`.
 
 Elasticsearch metrics can be filtered using KQL or Lucene syntax. To add a filter, use the named
 parameter \`kql='field: value'\` or \`lucene=''\`. Always use single quotes when writing KQL or Lucene
@@ -325,12 +332,63 @@ Math functions can take positional arguments, like pow(count(), 3) is the same a
 Use the symbols +, -, /, and * to perform basic math.
  `,
  description:
- 'Text is in markdown. Do not translate function names or field names like sum(bytes)',
+ 'Text is in markdown. Do not translate function names, special characters, or field names like sum(bytes)',
+ })}
+ />
+ </section>
+
+ <section
+ className="lnsFormula__docsTextGroup"
+ ref={(el) => {
+ if (el) {
+ scrollTargets.current[helpGroups[1].label] = el;
+ }
+ }}
+ >
+ <Markdown
+ markdown={i18n.translate('xpack.lens.formulaCommonFormulaDocumentation', {
+ defaultMessage: `
+## Common formulas
+
+The most common formulas are dividing two values to produce a percent.
+To display accurately, set *Value format* to *Percent*.
+
+### Filter ratio:
+
+Use \`kql=''\` to filter one set of documents and compare it to other documents within the same grouping.
+For example, to see how the error rate changes over time:
+
+\`\`\`
+count(kql='response.status_code > 400') / count()
+\`\`\`
+
+### Week over week:
+
+Use \`shift='1w'\` to get the value of each grouping from
+the previous week. Time shift should not be used with the *Top values* function.
+
+\`\`\`
+percentile(system.network.in.bytes, percentile=99) /
+percentile(system.network.in.bytes, percentile=99, shift='1w')
+\`\`\`
+
+### Percent of total:
+
+Formulas can calculate \`overall_sum\` for all the groupings,
+which lets you convert each grouping into a percent of total:
+
+\`\`\`
+sum(products.base_price) / overall_sum(sum(products.base_price))
+\`\`\`
+
+ `,
+ description:
+ 'Text is in markdown. Do not translate function names, special characters, or field names like sum(bytes)',
  })}
  />
  </section>
 
- {helpGroups.slice(1).map((helpGroup, index) => {
+ {helpGroups.slice(2).map((helpGroup, index) => {
  return (
  <section
  className="lnsFormula__docsTextGroup"
@@ -345,7 +403,7 @@ Use the symbols +, -, /, and * to perform basic math.
 
  <p>{helpGroup.description}</p>
 
- {helpGroups[index + 1].items.map((helpItem) => {
+ {helpGroups[index + 2].items.map((helpItem) => {
  return (
  <article
  className="lnsFormula__docsTextItem"