Merge branch 'main' into checkstyle-needs-braces

williamrandolph · Dec 8, 2023 · b66fa69 · b66fa69
2 parents f7e320b + bbbc58f
commit b66fa69
Show file tree

Hide file tree

Showing 274 changed files with 3,142 additions and 1,727 deletions.
diff --git a/docs/changelog/102435.yaml b/docs/changelog/102435.yaml
@@ -0,0 +1,5 @@
+pr: 102435
+summary: S3 first byte latency metric
+area: Search
+type: enhancement
+issues: []
diff --git a/docs/changelog/103024.yaml b/docs/changelog/103024.yaml
@@ -0,0 +1,6 @@
+pr: 103024
+summary: Fix template simulate setting application ordering
+area: Indices APIs
+type: bug
+issues:
+ - 103008
diff --git a/docs/changelog/103025.yaml b/docs/changelog/103025.yaml
@@ -0,0 +1,5 @@
+pr: 103025
+summary: "Metrics: Allow `AsyncCounters` to switch providers"
+area: Infra/Core
+type: bug
+issues: []
diff --git a/docs/changelog/103091.yaml b/docs/changelog/103091.yaml
@@ -0,0 +1,5 @@
+pr: 103091
+summary: "Metrics: Handle null observations in observers"
+area: Infra/Core
+type: bug
+issues: []
diff --git a/docs/changelog/103130.yaml b/docs/changelog/103130.yaml
@@ -0,0 +1,5 @@
+pr: 103130
+summary: Create a DSL health indicator as part of the health API
+area: Health
+type: feature
+issues: []
diff --git a/docs/changelog/103203.yaml b/docs/changelog/103203.yaml
@@ -0,0 +1,5 @@
+pr: 103203
+summary: Fix NPE & empty result handling in `CountOnlyQueryPhaseResultConsumer`
+area: Search
+type: bug
+issues: []
diff --git a/docs/reference/data-streams/lifecycle/apis/put-lifecycle.asciidoc b/docs/reference/data-streams/lifecycle/apis/put-lifecycle.asciidoc
@@ -59,6 +59,16 @@ duration the document could be deleted. When empty, every document in this data
 If defined, it turns data streqm lifecycle on/off (`true`/`false`) for this data stream.
 A data stream lifecycle that's disabled (`enabled: false`) will have no effect on the
 data stream. Defaults to `true`.
+
+`downsampling`::
+(Optional, array)
+An optional array of downsampling configuration objects, each defining an `after`
+interval representing when the backing index is meant to be downsampled (the time 
+frame is calculated since the index was rolled over, i.e. generation time) and
+a `fixed_interval` representing the downsampling interval (the minimum `fixed_interval`
+value is `5m`). A maximum number of 10 downsampling rounds can be configured.
+See <<data-streams-put-lifecycle-downsampling-example, configuration example>> below.
+
 ====
 
 [[data-streams-put-lifecycle-example]]
@@ -84,3 +94,29 @@ When the lifecycle is successfully updated in all data streams, you receive the
  "acknowledged": true
 }
 --------------------------------------------------
+
+[[data-streams-put-lifecycle-downsampling-example]]
+==== {api-examples-title}
+
+The following example configures two downsampling rounds, the first one starting
+one day after the backing index is rolled over (or later, if the index is still 
+within its write-accepting <<time-bound-indices, time bounds>>) with an interval
+of `10m`, and a second round starting 7 days after rollover at an interval of `1d`:
+
+[source,console]
+--------------------------------------------------------------------
+PUT _data_stream/my-weather-sensor-data-stream/_lifecycle
+{
+ "downsampling": [
+ {
+ "after": "1d",
+ "fixed_interval": "10m"
+ },
+ {
+ "after": "7d",
+ "fixed_interval": "1d"
+ }
+ ]
+}
+--------------------------------------------------------------------
+//TEST[skip:downsampling requires waiting for indices to be out of time bounds]
diff --git a/docs/reference/data-streams/lifecycle/index.asciidoc b/docs/reference/data-streams/lifecycle/index.asciidoc
@@ -18,6 +18,10 @@ and backwards incompatible mapping changes.
 * Configurable retention, which allows you to configure the time period for which your data is guaranteed to be stored.
 {es} is allowed at a later time to delete data older than this time period.
 
+A data stream lifecycle also supports downsampling the data stream backing indices.
+See <<data-streams-put-lifecycle-downsampling-example, the downsampling example>> for 
+more details.
+
 [discrete]
 [[data-streams-lifecycle-how-it-works]]
 === How does it work?
@@ -35,7 +39,9 @@ into tiers of exponential sizes, merging the long tail of small segments is only
 fraction of the cost of force mergeing to a single segment. The small segments would usually
 hold the most recent data so tail mergeing will focus the merging resources on the higher-value
 data that is most likely to keep being queried.
-4. Applies retention to the remaining backing indices. This means deleting the backing indices whose
+4. If <<data-streams-put-lifecycle-downsampling-example, downsampling>> is configured it will execute 
+all the configured downsampling rounds.
+5. Applies retention to the remaining backing indices. This means deleting the backing indices whose
 `generation_time` is longer than the configured retention period. The `generation_time` is only applicable to rolled over backing
 indices and it is either the time since the backing index got rolled over, or the time optionally configured in the
 <<index-data-stream-lifecycle-origination-date,`index.lifecycle.origination_date`>> setting.

diff --git a/docs/reference/esql/functions/binary.asciidoc b/docs/reference/esql/functions/binary.asciidoc
@@ -9,4 +9,12 @@ These binary comparison operators are supported:
 * less than: `<`
 * less than or equal: `<=`
 * larger than: `>`
-* larger than or equal: `>=`
+* larger than or equal: `>=`
+
+And these mathematical operators are supported:
+
+[.text-center]
+image::esql/functions/signature/add.svg[Embedded,opts=inline]
+
+[.text-center]
+image::esql/functions/signature/sub.svg[Embedded,opts=inline]
diff --git a/docs/reference/esql/functions/cos.asciidoc b/docs/reference/esql/functions/cos.asciidoc
@@ -4,7 +4,7 @@
 [.text-center]
 image::esql/functions/signature/cos.svg[Embedded,opts=inline]
 
-https://en.wikipedia.org/wiki/Sine_and_cosine[Cosine] trigonometric function.
+https://en.wikipedia.org/wiki/Sine_and_cosine[Cosine] trigonometric function. Input expected in radians.
 
 [source.merge.styled,esql]
 ----

diff --git a/docs/reference/esql/functions/operators.asciidoc b/docs/reference/esql/functions/operators.asciidoc
@@ -9,6 +9,7 @@ Boolean operators for comparing against one or multiple expressions.
 
 // tag::op_list[]
 * <<esql-binary-operators>>
+* <<esql-unary-operators>>
 * <<esql-logical-operators>>
 * <<esql-predicates>>
 * <<esql-cidr_match>>
@@ -23,6 +24,7 @@ Boolean operators for comparing against one or multiple expressions.
 // end::op_list[]
 
 include::binary.asciidoc[]
+include::unary.asciidoc[]
 include::logical.asciidoc[]
 include::predicates.asciidoc[]
 include::cidr_match.asciidoc[]

diff --git a/docs/reference/esql/functions/signature/add.svg b/docs/reference/esql/functions/signature/add.svg
diff --git a/docs/reference/esql/functions/signature/div.svg b/docs/reference/esql/functions/signature/div.svg
diff --git a/docs/reference/esql/functions/signature/equals.svg b/docs/reference/esql/functions/signature/equals.svg
diff --git a/docs/reference/esql/functions/signature/greater_than.svg b/docs/reference/esql/functions/signature/greater_than.svg
diff --git a/docs/reference/esql/functions/signature/less_than.svg b/docs/reference/esql/functions/signature/less_than.svg
diff --git a/docs/reference/esql/functions/signature/mod.svg b/docs/reference/esql/functions/signature/mod.svg
diff --git a/docs/reference/esql/functions/signature/mul.svg b/docs/reference/esql/functions/signature/mul.svg
diff --git a/docs/reference/esql/functions/signature/neg.svg b/docs/reference/esql/functions/signature/neg.svg
diff --git a/docs/reference/esql/functions/signature/not_equals.svg b/docs/reference/esql/functions/signature/not_equals.svg
diff --git a/docs/reference/esql/functions/signature/sub.svg b/docs/reference/esql/functions/signature/sub.svg
diff --git a/docs/reference/esql/functions/sin.asciidoc b/docs/reference/esql/functions/sin.asciidoc
@@ -4,7 +4,7 @@
 [.text-center]
 image::esql/functions/signature/sin.svg[Embedded,opts=inline]
 
-https://en.wikipedia.org/wiki/Sine_and_cosine[Sine] trigonometric function.
+https://en.wikipedia.org/wiki/Sine_and_cosine[Sine] trigonometric function. Input expected in radians.
 
 [source.merge.styled,esql]
 ----

diff --git a/docs/reference/esql/functions/tan.asciidoc b/docs/reference/esql/functions/tan.asciidoc
@@ -4,7 +4,7 @@
 [.text-center]
 image::esql/functions/signature/tan.svg[Embedded,opts=inline]
 
-https://en.wikipedia.org/wiki/Sine_and_cosine[Tangent] trigonometric function.
+https://en.wikipedia.org/wiki/Sine_and_cosine[Tangent] trigonometric function. Input expected in radians.
 
 [source.merge.styled,esql]
 ----

diff --git a/docs/reference/esql/functions/unary.asciidoc b/docs/reference/esql/functions/unary.asciidoc
@@ -0,0 +1,8 @@
+[discrete]
+[[esql-unary-operators]]
+=== Unary operators
+
+These unary mathematical operators are supported:
+
+[.text-center]
+image::esql/functions/signature/neg.svg[Embedded,opts=inline]
diff --git a/docs/reference/release-notes.asciidoc b/docs/reference/release-notes.asciidoc
@@ -8,6 +8,7 @@ This section summarizes the changes in each release.
 
 * <<release-notes-8.13.0>>
 * <<release-notes-8.12.0>>
+* <<release-notes-8.11.2>>
 * <<release-notes-8.11.1>>
 * <<release-notes-8.11.0>>
 * <<release-notes-8.10.4>>
@@ -58,6 +59,7 @@ This section summarizes the changes in each release.
 
 include::release-notes/8.13.0.asciidoc[]
 include::release-notes/8.12.0.asciidoc[]
+include::release-notes/8.11.2.asciidoc[]
 include::release-notes/8.11.1.asciidoc[]
 include::release-notes/8.11.0.asciidoc[]
 include::release-notes/8.10.4.asciidoc[]

diff --git a/docs/reference/release-notes/8.11.2.asciidoc b/docs/reference/release-notes/8.11.2.asciidoc
@@ -0,0 +1,83 @@
+[[release-notes-8.11.2]]
+== {es} version 8.11.2
+
+Also see <<breaking-changes-8.11,Breaking changes in 8.11>>.
+
+[[known-issues-8.11.2]]
+[float]
+=== Known issues
+include::8.10.3.asciidoc[tag=no-preventive-gc-issue]
+
+[[bug-8.11.2]]
+[float]
+=== Bug fixes
+
+Allocation::
+* Improve failure handling in `ContinuousComputation` {es-pull}102281[#102281]
+
+Application::
+* Default `run_ml_inference` should be true {es-pull}102151[#102151]
+* [Query Rules] Fix bug where combining the same metadata with text/numeric values leads to error {es-pull}102891[#102891] (issue: {es-issue}102827[#102827])
+
+Cluster Coordination::
+* Synchronize Coordinator#onClusterStateApplied {es-pull}100986[#100986] (issue: {es-issue}99023[#99023])
+
+Data streams::
+* [Usage API] Count all the data streams that have lifecycle {es-pull}102259[#102259]
+
+ES|QL::
+* ES|QL: Fix drop of renamed grouping {es-pull}102282[#102282] (issue: {es-issue}102121[#102121])
+* ES|QL: Fix layout management for Project {es-pull}102399[#102399] (issue: {es-issue}102120[#102120])
+* Fix DISSECT with empty patterns {es-pull}102580[#102580] (issue: {es-issue}102577[#102577])
+* Fix leaking blocks in TopN {es-pull}102715[#102715] (issue: {es-issue}102646[#102646])
+* Fix leaking blocks in `BlockUtils` {es-pull}102716[#102716]
+* Fix memory tracking in TopN.Row {es-pull}102831[#102831] (issues: {es-issue}100640[#100640], {es-issue}102784[#102784], {es-issue}102790[#102790], {es-issue}102683[#102683])
+
+ILM+SLM::
+* [ILM] Fix downsample to skip already downsampled indices {es-pull}102250[#102250] (issue: {es-issue}102249[#102249])
+
+Infra/Circuit Breakers::
+* Add more logging to the real memory circuit breaker and lower minimum interval {es-pull}102396[#102396]
+
+Ingest Node::
+* Better processor stat merge {es-pull}102821[#102821]
+
+Machine Learning::
+* Ensure datafeed previews with no start or end time don't search the cold or frozen tiers {es-pull}102492[#102492]
+* Recreate the Elasticsearch private temporary directory if it doesn't exist when an ML job is opened {es-pull}102599[#102599]
+
+Mapping::
+* Fix dense_vector cluster stats indexed_vector_dim_min/max values {es-pull}102467[#102467] (issue: {es-issue}102416[#102416])
+
+Search::
+* Allow mismatched sort-by field types if there are no docs to sort {es-pull}102779[#102779]
+
+Security::
+* Fix double-completion in `SecurityUsageTransportAction` {es-pull}102114[#102114] (issue: {es-issue}102111[#102111])
+
+Snapshot/Restore::
+* Set region for the STS client via privileged calls in AWS SDK {es-pull}102230[#102230] (issue: {es-issue}102173[#102173])
+* Simplify `BlobStoreRepository` idle check {es-pull}102057[#102057] (issue: {es-issue}101948[#101948])
+
+Transform::
+* Ensure transform updates only modify the expected transform task {es-pull}102934[#102934] (issue: {es-issue}102933[#102933])
+* Exclude stack traces from transform audit messages and health {es-pull}102240[#102240]
+
+[[enhancement-8.11.2]]
+[float]
+=== Enhancements
+
+Machine Learning::
+* Add inference counts by model to the machine learning usage stats {es-pull}101915[#101915]
+
+Security::
+* Upgrade xmlsec to 2.3.4 {es-pull}102220[#102220]
+
+[[upgrade-8.11.2]]
+[float]
+=== Upgrades
+
+Snapshot/Restore::
+* Upgrade reactor netty http version {es-pull}102311[#102311]
+
+
diff --git a/docs/reference/search/search.asciidoc b/docs/reference/search/search.asciidoc
@@ -111,6 +111,13 @@ By default, you cannot page through more than 10,000 hits using the `from` and
 
 include::{es-repo-dir}/rest-api/common-parms.asciidoc[tag=ignore_throttled]
 
+`include_named_queries_score`::
+(Optional, Boolean) If `true`, includes the score contribution from any named queries.
+This functionality reruns each named query on every hit in a search
+response. Typically, this adds a small overhead to a request. However, using
+computationally expensive named queries on a large number of hits may add
+significant overhead. Defaults to `false`.
+
 include::{es-repo-dir}/rest-api/common-parms.asciidoc[tag=index-ignore-unavailable]
 
 include::{es-repo-dir}/rest-api/common-parms.asciidoc[tag=lenient]