Skip to content

Commit

Permalink
[DOCS] Document range enrich policy (elastic#79607)
Browse files Browse the repository at this point in the history
Adding docs for the range enrich policy

Co-authored-by: James Rodewig <[email protected]>
  • Loading branch information
2 people authored and Adam Locke committed Oct 28, 2021
1 parent bbe5749 commit 7ba79ee
Show file tree
Hide file tree
Showing 5 changed files with 177 additions and 3 deletions.
5 changes: 5 additions & 0 deletions docs/reference/ingest/apis/enrich/put-enrich-policy.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -98,6 +98,11 @@ Matches enrich data to incoming documents based on a
Matches enrich data to incoming documents based on a
<<query-dsl-term-query,`term` query>>. For an example, see
<<match-enrich-policy-type>>.

`range`:::
Matches a number, date, or IP address in incoming documents to a range in the
enrich index based on a <<query-dsl-term-query,`term` query>>. For an example,
see <<range-enrich-policy-type>>.
--
+
.Properties of `<policy-type>`
Expand Down
7 changes: 4 additions & 3 deletions docs/reference/ingest/enrich.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -218,9 +218,6 @@ Instead, you can:
to delete the previous enrich policy.
// end::update-enrich-policy[]

include::geo-match-enrich-policy-type-ex.asciidoc[]
include::match-enrich-policy-type-ex.asciidoc[]

[[ingest-enrich-components]]
==== Enrich components

Expand Down Expand Up @@ -271,3 +268,7 @@ How often {es} checks whether unused enrich indices can be deleted. Defaults to

`enrich.max_concurrent_policy_executions`::
Maximum number of enrich policies to execute concurrently. Defaults to `50`.

include::geo-match-enrich-policy-type-ex.asciidoc[]
include::match-enrich-policy-type-ex.asciidoc[]
include::range-enrich-policy-type-ex.asciidoc[]
Original file line number Diff line number Diff line change
Expand Up @@ -165,6 +165,8 @@ The API returns the following response:
--------------------------------------------------
DELETE /_ingest/pipeline/postal_lookup
DELETE /_enrich/policy/postal_policy
DELETE /postal_codes
DELETE /users
--------------------------------------------------
// TEST[continued]
////
2 changes: 2 additions & 0 deletions docs/reference/ingest/match-enrich-policy-type-ex.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -146,6 +146,8 @@ The API returns the following response:
--------------------------------------------------
DELETE /_ingest/pipeline/user_lookup
DELETE /_enrich/policy/users-policy
DELETE /my-index-000001
DELETE /users
--------------------------------------------------
// TEST[continued]
////
164 changes: 164 additions & 0 deletions docs/reference/ingest/range-enrich-policy-type-ex.asciidoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,164 @@
[role="xpack"]
[testenv="basic"]
[[range-enrich-policy-type]]
=== Example: Enrich your data by matching a value to a range

A `range` <<enrich-policy,enrich policy>> uses a <<query-dsl-term-query,`term`
query>> to match a number, date, or IP address in incoming documents to a range
of the same type in the enrich index. Matching a range to a range is not
supported.

The following example creates a `range` enrich policy that adds a descriptive network name and
responsible department to incoming documents based on an IP address. It then
adds the enrich policy to a processor in an ingest pipeline.

Use the <<indices-create-index, create index API>> with the appropriate mappings to create a source index.

[source,console]
----
PUT /networks
{
"mappings": {
"properties": {
"range": { "type": "ip_range" },
"name": { "type": "keyword" },
"department": { "type": "keyword" }
}
}
}
----

The following index API request indexes a new document to that index.

[source,console]
----
PUT /networks/_doc/1?refresh=wait_for
{
"range": "10.100.0.0/16",
"name": "production",
"department": "OPS"
}
----
// TEST[continued]

Use the create enrich policy API to create an enrich policy with the
`range` policy type. This policy must include:

* One or more source indices
* A `match_field`,
the field from the source indices used to match incoming documents
* Enrich fields from the source indices you'd like to append to incoming
documents

Since we plan to enrich documents based on an IP address, the policy's
`match_field` must be an `ip_range` field.

[source,console]
----
PUT /_enrich/policy/networks-policy
{
"range": {
"indices": "networks",
"match_field": "range",
"enrich_fields": ["name", "department"]
}
}
----
// TEST[continued]

Use the <<execute-enrich-policy-api,execute enrich policy API>> to create an
enrich index for the policy.

[source,console]
----
POST /_enrich/policy/networks-policy/_execute
----
// TEST[continued]


Use the <<put-pipeline-api,create or update pipeline API>> to create an ingest
pipeline. In the pipeline, add an <<enrich-processor,enrich processor>> that
includes:

* Your enrich policy.
* The `field` of incoming documents used to match documents
from the enrich index.
* The `target_field` used to store appended enrich data for incoming documents.
This field contains the `match_field` and `enrich_fields` specified in your
enrich policy.

[source,console]
----
PUT /_ingest/pipeline/networks_lookup
{
"processors" : [
{
"enrich" : {
"description": "Add 'network' data based on 'ip'",
"policy_name": "networks-policy",
"field" : "ip",
"target_field": "network",
"max_matches": "10"
}
}
]
}
----
// TEST[continued]

Use the ingest pipeline to index a document. The incoming document should
include the `field` specified in your enrich processor.

[source,console]
----
PUT /my-index-000001/_doc/my_id?pipeline=networks_lookup
{
"ip": "10.100.34.1"
}
----
// TEST[continued]

To verify the enrich processor matched and appended the appropriate field data,
use the <<docs-get,get API>> to view the indexed document.

[source,console]
----
GET /my-index-000001/_doc/my_id
----
// TEST[continued]

The API returns the following response:

[source,console-result]
----
{
"_index" : "my-index-000001",
"_id" : "my_id",
"_version" : 1,
"_seq_no" : 0,
"_primary_term" : 1,
"found" : true,
"_source" : {
"ip" : "10.100.34.1",
"network" : [
{
"name" : "production",
"range" : "10.100.0.0/16",
"department" : "OPS"
}
]
}
}
----
// TESTRESPONSE[s/"_seq_no": \d+/"_seq_no" : $body._seq_no/ s/"_primary_term":1/"_primary_term" : $body._primary_term/]

////
[source,console]
--------------------------------------------------
DELETE /_ingest/pipeline/networks_lookup
DELETE /_enrich/policy/networks-policy
DELETE /networks
DELETE /my-index-000001
--------------------------------------------------
// TEST[continued]
////

0 comments on commit 7ba79ee

Please sign in to comment.