Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Detection Engine][Exceptions] - Update docs for single and shared exceptions #4021

Open
wants to merge 45 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 37 commits
Commits
Show all changes
45 commits
Select commit Hold shift + click to select a range
206f23d
First draft
nastasha-solomon Oct 9, 2023
55fbab7
Fixed include
nastasha-solomon Oct 9, 2023
df3e171
Merge branch 'main' into issue-3491-exception-item-list
nastasha-solomon Oct 13, 2023
cf640b2
Merge branch 'main' into issue-3491-exception-item-list
nastasha-solomon Oct 27, 2023
cc62c0c
Update/create exception changes
nastasha-solomon Oct 29, 2023
692568f
Update docs/detections/api/exceptions/api-create-rule-default-excepti…
nastasha-solomon Oct 29, 2023
c51783a
Merge branch 'main' into issue-3491-exception-item-list
nastasha-solomon Oct 29, 2023
9c31311
Merge branch 'main' into issue-3491-exception-item-list
nastasha-solomon Nov 2, 2023
a41f72e
Merge branch 'main' into issue-3491-exception-item-list
nastasha-solomon Nov 13, 2023
7ec89a2
Merge branch 'main' into issue-3491-exception-item-list
nastasha-solomon Jan 4, 2024
a8cd5c5
Merge branch 'main' into issue-3491-exception-item-list
nastasha-solomon Mar 13, 2024
d5a7b9a
Revising intro
nastasha-solomon Mar 13, 2024
0503015
Merge branch 'main' into issue-3491-exception-item-list
nastasha-solomon Mar 19, 2024
db76cae
Adding new topic
nastasha-solomon Mar 19, 2024
d17654c
Merge branch 'main' into issue-3491-exception-item-list
nastasha-solomon Mar 20, 2024
ee215e5
Fixing refs
nastasha-solomon Mar 20, 2024
f1a983c
Removed duplicate entry
nastasha-solomon Mar 20, 2024
e6788ed
Merge branch 'main' into issue-3491-exception-item-list
nastasha-solomon Mar 20, 2024
eebdef6
Merge branch 'main' into issue-3491-exception-item-list
nastasha-solomon Apr 2, 2024
5353b60
Yara's input
nastasha-solomon Apr 2, 2024
76d7af1
Minor wording changes
nastasha-solomon Apr 2, 2024
1c8efcb
Title update
nastasha-solomon Apr 2, 2024
3e3bea2
Modifies title for shared exceptions
nastasha-solomon Apr 2, 2024
516e652
Fixing conflicts
nastasha-solomon Apr 3, 2024
dfba7d2
attempt number 2
nastasha-solomon Apr 3, 2024
7a4ac5f
Added file ext to index file
nastasha-solomon Apr 3, 2024
4a6e620
Update docs/detections/api/rules/rules-api-overview.asciidoc
nastasha-solomon Apr 3, 2024
8f586d5
Closing table
nastasha-solomon Apr 3, 2024
d6f4967
Update docs/detections/api/rules/rules-api-create-rule-default-except…
nastasha-solomon Apr 3, 2024
7086704
Merge branch 'issue-3491-exception-item-list' of github.com:elastic/s…
nastasha-solomon Apr 3, 2024
7cbe9c8
Minor changes and notes
nastasha-solomon Apr 3, 2024
fb59901
Update docs/detections/api/rules/rules-api-create-rule-default-except…
nastasha-solomon Apr 3, 2024
31e2127
Update docs/detections/api/rules/rules-api-create-single-rule-excepti…
nastasha-solomon Apr 3, 2024
ec9997b
Update docs/detections/api/rules/rules-api-create-single-rule-excepti…
nastasha-solomon Apr 3, 2024
b9589c5
Fixed route in example
nastasha-solomon Apr 3, 2024
845b50d
Merge branch 'issue-3491-exception-item-list' of github.com:elastic/s…
nastasha-solomon Apr 3, 2024
2b47c51
Remove comment chars
nastasha-solomon Apr 3, 2024
94af96d
Merge branch 'main' into issue-3491-exception-item-list
nastasha-solomon May 28, 2024
585ed2c
Input from Yara
nastasha-solomon Aug 19, 2024
4ca0d5b
Refreshed params
nastasha-solomon Aug 20, 2024
92543bd
minor style fixes
nastasha-solomon Aug 20, 2024
3cca05f
Merge branch 'main' into issue-3491-exception-item-list
nastasha-solomon Aug 20, 2024
cc59ab9
Merge branch 'main' into issue-3491-exception-item-list
nastasha-solomon Aug 22, 2024
547bcd0
Merge branch 'main' into issue-3491-exception-item-list
nastasha-solomon Sep 8, 2024
20bd1da
Update docs/detections/api/rules/rules-api-create-rule-default-except…
nastasha-solomon Oct 10, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions docs/detections/api/det-api-index.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,10 @@ include::rules/rules-api-delete.asciidoc[]

include::rules/rules-api-bulk-actions.asciidoc[]

include::rules/rules-api-create-rule-default-exception-list.asciidoc[]

include::rules/rules-api-create-single-rule-exception-item.asciidoc[]

include::rules/index-api-overview.asciidoc[]

include::rules/tags-api-overview.asciidoc[]
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -41,10 +41,11 @@ provided.
|No, defaults to `single`.
|`tags` |String[] |String array containing words and phrases to help categorize
exception containers. |No
|`type` |String a|The type of exception, which must be one of these:
|`type` |String a|The type of exception list, which must be one of these:

* `detection`: Detection rule exception
* `endpoint`: Endpoint alert exception
* `detection`: Shared rule exception
* `endpoint`: Endpoint rule exception
* `rule_default`: Single rule exception

|Yes

Expand Down
28 changes: 15 additions & 13 deletions docs/detections/api/exceptions/api-create-exception-item.asciidoc
Original file line number Diff line number Diff line change
@@ -1,24 +1,26 @@
[[exceptions-api-create-exception-item]]
=== Create exception item
=== Create exceptions used by multiple rules

Creates an exception item and associates it with the specified
<<exceptions-api-create-container, exception container>>.
Creates an exception item that you can add to a shared exception list or endpoint exception list.

Refer to <<lists-api-overview>> for information about creating exception items from
lists, such as a list of IP addresses or host names.
TIP: Refer to <<exceptions-api-create-rule-default-exception-item>> for information about creating exceptions for a single rule.

NOTE: Before creating exception items, you must create an exception container.

IMPORTANT: Endpoint rule exception items cannot use
[IMPORTANT]
=====
* Before creating exception items, you must create an <<exceptions-api-create-container, exception container>>. After creating the container, you can associate exception items with it.
* Endpoint rule exception items cannot use
<<lists-api-overview, lists>> (the `list` in the `entries` array), and the
following fields cannot be used in exception queries (as `field` values in the
`entries` object):

* `file.Ext.quarantine_path`
* `file.Ext.quarantine_result`
* `process.entity_id`
* `process.parent.entity_id`
* `process.ancestry`
** `file.Ext.quarantine_path`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@caitlinbetz is this still accurate? I know we opened endpoint exceptions up to have less field restrictions, not sure if these still hold.

** `file.Ext.quarantine_result`
** `process.entity_id`
** `process.parent.entity_id`
** `process.ancestry`
=====

NOTE: Refer to <<lists-api-overview>> for information about creating exception items from lists, such as a list of IP addresses or host names.

==== Request URL

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -29,8 +29,11 @@ the container's `id` field is not used.
exception containers. |No
|`type` |String a|The type of exception, which must be one of these:

* `detection`: Detection rule exception
* `endpoint`: Endpoint alert exception
|`type` |String a|The type of exception list, which must be one of these:

* `detection`: Shared rule exception
* `endpoint`: Endpoint rule exception
* `rule_default`: Single rule exception

|Yes

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,105 @@
[[exceptions-api-create-rule-default-exception-list]]
=== Create default exception list for a rule

Creates a default exception list for the rule you specify. Default exception lists can only be associated with a single rule.

To add exception items to a default exception list, pass in exceptions items that you want applied to the rule. Refer to <<exceptions-api-create-rule-default-exception-item>> for more information.

When an exception item’s query evaluates to `true`, the associated rule does not issue alerts even when its other criteria are met.

NOTE: Default exception lists do not display on the <<shared-exception-lists, Shared Exception Lists>> page in the {security-app} UI.
nastasha-solomon marked this conversation as resolved.
Show resolved Hide resolved

==== Request URL

`POST <kibana host>:<port>/api/detection_engine/<rule_id>/exceptions`

==== Request body

A JSON object with these fields:

[width="100%",options="header"]
|==============================================
|Name |Type |Description |Required

|`description` |String |Describes the exception container. |Yes
|`list_id` |String |Unique identifier. |No, automatically created when it is not
provided.
|`meta` |Object |Placeholder for metadata about the list container. |No
|`name` |String |The exception container's name. |Yes
|`namespace_type` |String a|Determines whether the exception container is available in all {kib} spaces or just the space in which it is created, where:

* `single`: Only available in the {kib} space in which it is created.
* `agnostic`: Available in all {kib} spaces.

|No, defaults to `single`.
|`tags` |String[] |String array containing words and phrases to help categorize
exception containers. |No
|`type` |String a|The type of exception, which must be:

* `rule_default`: Exception list that belongs to a single rule

|Yes

|==============================================

===== Example requests

Creates an exception container for holding trusted Linux process exception
items:

[source,console]
--------------------------------------------------
POST api/exception_lists
nastasha-solomon marked this conversation as resolved.
Show resolved Hide resolved
{
"description": "Excludes Linux trusted processes",
"name": "Linux process exceptions",
"list_id": "trusted-linux-processes",
"type": "detection",
"namespace_type": "single",
"tags": [
"linux",
"processes"
]
}
--------------------------------------------------
// KIBANA

==== Response code

`200`::
Indicates a successful call.


==== Response payload

The exception container object with a unique ID.

[source,json]
--------------------------------------------------
{
"_tags": [],
"created_at": "2020-07-13T09:33:46.187Z",
"created_by": "elastic",
"description": "Excludes Linux trusted processes",
"id": "f320c070-c4eb-11ea-80bb-11861bae2798", <1>
"list_id": "trusted-linux-processes", <2>
"name": "Linux process exceptions",
"namespace_type": "single", <3>
"tags": [
"linux",
"processes"
],
"tie_breaker_id": "2c08d5a5-2ecc-4d5a-acfb-0a367f25b3f3",
"type": "detection", <4>
"updated_at": "2020-07-13T09:33:46.359Z",
"updated_by": "elastic"
}
--------------------------------------------------

These values are required to associate the exception container with detection
rules:

<1> `id`
<2> `list_id`
<3> `namespace_type`
<4> `type`
Original file line number Diff line number Diff line change
@@ -0,0 +1,86 @@
[[exceptions-api-create-rule-default-exception-item]]
=== Create exceptions for a rule

Allows you to create exception items that are associated with a specified rule `id`.
nastasha-solomon marked this conversation as resolved.
Show resolved Hide resolved

==== Request URL

`POST <kibana host>:<port>/api/detection_engine/rules/<id>/exceptions`

//include a tip on how to find a rule's ID
nastasha-solomon marked this conversation as resolved.
Show resolved Hide resolved

==== Request body

A JSON object with an array of exception items, where each exception item has the <<exceptions-api-create-exception-item,required fields>>.

[width="100%",options="header"]
|==============================================
nastasha-solomon marked this conversation as resolved.
Show resolved Hide resolved
|Name |Type |Description |Required

|`items` |String | Specify an array of exception list items to create. |Yes

|==============================================

===== Example requests

[source,console]
--------------------------------------------------
POST api/detection_engine/rules/<id>/exceptions
nastasha-solomon marked this conversation as resolved.
Show resolved Hide resolved
{
"items": [
{
"field": "process.name",
"operator": "included",
"type": "match",
"value": "maintenance-job"
}
],
"list_id": "trusted-linux-processes",
"name": "Linux maintenance job",
"namespace_type": "single",
"tags": [
"in-house processes",
"linux"
],
"type": "simple"
}
--------------------------------------------------

==== Response code

`200`::
Indicates a successful call.

==== Response payload

The returned exception item.

Example response:

[source,json]
--------------------------------------------------
{
"body": [
{
"comments": [],
"created_by": "elastic",
"description": "Exception item for rule default exception list",
"entries": [
{
"field": "host.name",
"operator": "included",
"type": "match",
"value": "foo",
},
],
"name": "Sample exception item",
"list_id": "e6c44050-c661-11ea-bab5-9d6ae015701b",
"namespace_type": "single",
"os_types": [],
"tags": [],
"type": "simple",
"updated_by": "elastic"
}
]
}
--------------------------------------------------
2 changes: 2 additions & 0 deletions docs/detections/api/rules/rules-api-overview.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,8 @@ tags
returns alerts, and updates their statuses
* `<kibana host>:<port>/api/detection_engine/rules/prepackaged` - Loads and retrieves
the status of Elastic <<prebuilt-rules, prebuilt rules>>
* `<kibana host>:<port>/api/detection_engine/<rule_id>/exceptions` - Creates a default exception list for the rule you specify
* `<kibana host>:<port>/api/detection_engine/rules/<id>/exceptions` - Creates exception items for the rule you specify

TIP: You can view and download a Detections API Postman collection
https:/elastic/examples/tree/master/Security%20Analytics/SIEM-examples/Detections-API[here].
Expand Down
5 changes: 3 additions & 2 deletions docs/siem-apis.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,9 @@ You can use these APIs to interface with {elastic-sec} features:
NOTE: Console supports sending requests to {kib} APIs. Prepend any {kib} API endpoint with `kbn:` and send the request via Console. For example:
`GET kbn:/api/index_management/indices`

* <<rule-api-overview>>: Manage detection rules and alerts
* <<exceptions-api-overview>>: Create and manage rule exceptions
* <<rule-api-overview>>: Manage detection rules, rule exceptions for individual rules, and alerts
* <<exceptions-api-overview>>: Create and manage shared rule exceptions
//Will finalize descriptions once get confirmation from Yara about exception API endpoints
nastasha-solomon marked this conversation as resolved.
Show resolved Hide resolved
* <<lists-api-overview>>: Create source event value lists for use with rule exceptions
* <<timeline-api-overview>>: Import and export timelines
* <<cases-api-overview>>: Open and manage cases
Expand Down