Merge branch 'main' into issue-5897-alert-suppress-ga

docs/advanced-entity-analytics/ml-req.asciidoc

@@ -7,6 +7,10 @@ To run and create {ml} jobs and rules, you need all of these:
 * There must be at least one {ml} node in your cluster
 * The `machine_learning_admin` user role
 
+Additionally, to configure <<alert-suppression,alert suppression>> for {ml} rules, your role needs the following {kibana-ref}/kibana-role-management.html#adding_index_privileges[index privilege]:
+
+* `read` permission for the `.ml-anomalies-*` index
+
 For more information, go to {ml-docs}/setup.html[Set up {ml-features}].
 
 [IMPORTANT]

docs/detections/alert-suppression.asciidoc

@@ -4,7 +4,9 @@
 .Requirements and notices
 [sidebar]
 --
-Alert suppression requires a https://www.elastic.co/pricing[Platinum or higher subscription].
+* Alert suppression requires a https://www.elastic.co/pricing[Platinum or higher subscription].
+
+* {ml-cap} rules have <<ml-requirements,additional requirements>> for alert suppression.
 
 preview::["Alert suppression is in technical preview for event correlation rules. The functionality may be changed or removed in a future release. Elastic will work to fix any issues, but features in technical preview are not subject to the support SLA of official GA features."]
 --
@@ -17,7 +19,7 @@ Alert suppression allows you to reduce the number of repeated or duplicate detec
 * <<create-eql-rule,Event correlation>> (non-sequence queries only)
 * <<create-new-terms-rule,New terms>> 
 * <<create-esql-rule,{esql}>>
-* <<create-ml-rule,{ml-app}>>
+* <<create-ml-rule,{ml-cap}>>
 
 Normally, when a rule meets its criteria repeatedly, it creates multiple alerts, one for each time the rule's criteria are met. When alert suppression is configured, duplicate qualifying events are grouped, and only one alert is created for each group. Depending on the rule type, you can configure alert suppression to create alerts each time the rule runs, or once within a specified time window. You can also specify multiple fields to group events by unique combinations of values.
 

docs/detections/api/rules/rules-api-bulk-actions.asciidoc

@@ -289,14 +289,24 @@ to apply.
 * `duplicate`
 * `export`
 * `edit`
+* `run`
 
 | Yes
+
+| `run` | <<bulk-manual-rule-run, BulkManualRuleRun[]>>
+| Object that describes applying a manual rule run action.
+|No.
+
+Yes, if action is `run`.
+
 | `edit` | <<bulk-edit-object-schema, BulkEditAction[]>>
 | Edit object that describes applying an update action.
 
 |No.
 
 Yes, if action is `edit`.
+
+
 | `duplicate` | <<bulk-duplicate-object-schema, BulkDuplicateAction[]>>
 | Duplicate object that describes applying an update action.
 
@@ -314,6 +324,13 @@ To enable dry run mode on a request, add the query parameter `dry_run=true` to t
 
 IMPORTANT: Dry run mode is not supported for the `export` bulk action. A `400` error will be returned in the request response.
 
+[[bulk-manual-rule-run]]
+[discrete]
+==== BulkManualRuleRun object
+
+* `start_date` field: (String, Required) Defines the start date of the manual run.
+* `end_date` field: (String, Optional) Defines the end date of the manual run.
+
 [[bulk-duplicate-object-schema]]
 [discrete]
 ==== BulkDuplicateAction object
@@ -525,14 +542,14 @@ POST api/detection_engine/rules/_bulk_action
 [discrete]
 ===== Response payload
 
-For `enable`, `disable`, `delete`, `edit`, and `duplicate` actions, a JSON object containing the action's outcome:
+For `enable`, `disable`, `delete`, `edit`, `duplicate`, and `run` actions, a JSON object containing the action's outcome:
 
 - `attributes.summary.total`: Total number of rules matching the bulk action
 - `attributes.summary.succeeded`: Number of successful outcomes (number of rules that were enabled, deleted, or updated)
 - `attributes.summary.failed`: Number of failed outcomes
 - `attributes.summary.skipped`: Number of rules that were skipped due to various reasons (explained below)
 - `attributes.results.created`: Rule objects that were created during the action's execution
-- `attributes.results.updated`: Rule objects that were updated during the action's execution
+- `attributes.results.updated`: Rule objects that were updated during the action's execution. If the action execution is `run`, it returns rule objects that were scheduled for manual runs.
 - `attributes.results.deleted`: Rule objects that were deleted during the action's execution
 - `attributes.results.skipped`: Rules that were skipped during the action's execution
 

docs/detections/rules-ui-create.asciidoc

@@ -820,3 +820,21 @@ TIP: Avoid setting long time ranges with short rule intervals, or the rule previ
 
 * To close the preview, click the *Rule preview* button again.
 
+[discrete]
+[[view-rule-es-queries]]
+==== View your rule's {es} queries (optional)
+
+NOTE: This option is only offered for {esql} and event correlation rules.
+
+When previewing a rule, you can also learn about its {es} queries, which are submitted when the rule runs. This information can help you identify and troubleshoot potential rule issues. You can also use it to confirm that your rule is retrieving the expected data. 
+
+To learn more about your rule's {es} queries, preview its results and do the following:
+
+. Select the **Show {es} requests, ran during rule executions** option below the preview's date and time picker. The **Preview logged results** section displays under the histogram and alerts table. 
+. Click the **Preview logged results** section to expand it. Within the section, each rule execution is shown on an individual row. 
+. Expand each row to learn more about the {es} queries that the rule submits each time it executes. The following details are provided:
+** When the rule execution started, and how long it took to complete
+** A brief explanation of what the {es} queries do
+** The actual {es} queries that the rule submits to indices containing events that are used during the rule execution 
++
+TIP: Run the queries in {kibana-ref}/console-kibana.html[Console] to determine if your rule is retrieving the expected data. For example, to test your rule’s exceptions, run the rule’s {es} queries, which will also contain exceptions added to the rule. If your rule’s exceptions are working as intended, the query will not return events that should be ignored.

docs/detections/rules-ui-manage.asciidoc

@@ -17,6 +17,7 @@ On the Rules page, you can:
 * <<rule-status>>
 * <<edit-rules-settings>>
 * <<manage-rules-ui>>
+* <<manually-run-rules>>
 * <<snooze-rule-actions>>
 * <<import-export-rules-ui>>
 * <<rule-prerequisites>>
@@ -104,6 +105,34 @@ NOTE: When duplicating a rule with exceptions, you can choose to duplicate the r
 * To enable or disable a single rule, switch on the rule's *Enabled* toggle.
 * To <<snooze-rule-actions,snooze>> actions for rules, click the bell icon.
 
+[float]
+[[manually-run-rules]]
+=== Run rules manually
+
+beta::[]
+
+Manually run enabled rules for a specified period of time for testing purposes or additional rule coverage. 
+
+IMPORTANT: Before manually running rules, make sure you properly understand and plan for rule dependencies. Incorrect scheduling can lead to inconsistent rule results.
+
+1. Navigate to the detection rules page, and do one of the following:
+* Select the **All actions** menu (**...**) on a rule, then select **Manual run**.
+* Select all the rules you want to manually run, select the **Bulk actions** menu, then select **Manual run**.
+. Specify when the manual run starts and ends. The default selection is the current day starting three hours in the past. The rule will search for events during the selected time range.
+. Click **Run** to manually run the rule.
++
+NOTE: Manual runs can produce multiple rule executions. This is determined by the manual run's time range and the rule's execution schedule.
+
+The manual run's details are shown in the <<manual-runs-table,Manual runs>> table on the *Execution results* tab. Changes you make to the manual run or rule settings will display in the Manual runs table after the current run completes.
+
+[NOTE] 
+=====
+Be mindful of the following:
+* Rule actions are not activated during manual runs. 
+* Except for threshold rules, duplicate alerts aren't created if you manually run a rule during a time range that was already covered by a scheduled run.
+* Manual runs are executed with low priority and limited concurrency, meaning they might take longer to complete. This can be especially apparent for rules requiring multiple executions.
+=====
+
 [float]
 [[snooze-rule-actions]]
 === Snooze rule actions

docs/detections/rules-ui-monitor.asciidoc

@@ -37,28 +37,62 @@ For detailed information on a rule, the alerts it generated, and associated erro
 [[rule-execution-logs]]
 === Execution results
 
-Each detection rule execution is logged, including its success or failure, any warning or error messages, and how long it took to search for data, create alerts, and complete. This can help you troubleshoot a particular rule if it isn't behaving as expected (for example, if it isn't creating alerts or takes a long time to run).
+Each detection rule execution is logged, including the execution type, the execution's success or failure, any warning or error messages, how long it took to search for data, create alerts, and complete. This can help you troubleshoot a particular rule if it isn't behaving as expected (for example, if it isn't creating alerts or takes a long time to run).
 
-To access a rule's execution log, go to *Rules* -> *Detection rules (SIEM)*, click the rule's name to open its details, then scroll down and select the *Execution results* tab. You can expand a long warning or error message by clicking the arrow at the end of a row.
+To access a rule's execution log, go to **Rules** → **Detection rules (SIEM)**, click the rule's name to open its details, then scroll down and select the **Execution results** tab. Within the Execution log table, you can click the arrow at the end of a row to expand a long warning or error message.
 
 [role="screenshot"]
-image::images/rule-execution-logs.png[Rule execution results tab]
+image::images/rule-execution-logs.png[Execution log table on the rule execution results tab]
 
 You can hover over each column heading to display a tooltip about that column's data. Click a column heading to sort the table by that column.
 
 Use these controls to filter what's included in the logs table:
 
+* The **Run type** drop-down filters the table by rule execution type: 
+** **Scheduled**: Automatic, scheduled rule executions.
+** **Manual**: Rule executions that were <<manually-run-rules,started manually>>.
+
 * The *Status* drop-down filters the table by rule execution status: 
 ** *Succeeded*: The rule completed its defined search. This doesn't necessarily mean it generated an alert, just that it ran without error.
 ** *Failed*: The rule encountered an error that prevented it from running. For example, a {ml} rule whose corresponding {ml} job wasn't running.
 ** *Warning*: Nothing prevented the rule from running, but it might have returned unexpected results. For example, a custom query rule tried to search an index pattern that couldn't be found in {es}.
 
 * The date and time picker sets the time range of rule executions included in the table. This is separate from the global date and time picker at the top of the rule details page.
 
+* The **Source event time range** button toggles the display of data pertaining to the time range of manual runs.
+
 * The *Show metrics columns* toggle includes more or less data in the table, pertaining to the timing of each rule execution.
 
 * The *Actions* column allows you to show alerts generated from a given rule execution. Click the filter icon (image:images/filter-icon.png[Filter icon,18,17]) to create a global search filter based on the rule execution's ID value. This replaces any previously applied filters, changes the global date and time range to 24 hours before and after the rule execution, and displays a confirmation notification. You can revert this action by clicking *Restore previous filters* in the notification.
 
+[float]
+[[manual-runs-table]]
+==== Manual runs table
+
+beta::[]
+
+Each manual run can produce multiple rule executions, depending on the time range of the run and the rule's execution schedule. These details are shown in the Manual runs table.
+
+To access the table, navigate to the detection rules page, click the rule's name to open its details, then scroll down and select the **Execution results** tab. Scroll down again to find the Manual runs table. 
+
+To stop an active run, go to the appropriate row and click **Stop run** in the **Actions** column. Completed rule executions for each manual run are logged in the Execution log table.
+
+[role="screenshot"]
+image::images/manual-rule-run-table.png[Manual rule runs table on the rule execution results tab]
+
+The Manual runs table displays important details such as:
+
+* The status of each manual run:
+** **Pending**: The rule is not yet running. 
+** **Running**: The rule is executing during the time range you specified. Some rules, such as indicator match rules, can take longer to run.
+** **Error**: The rule's configuration is preventing it from running correctly. For example, the rule's conditions cannot be validated.
+
+* When a manual run started and the time in which it will run
+
+* The number of rule executions that are failing, pending, running, and completed for a manual run
+
+* The total number of rule executions that are occurring for a manual run
+
 [float]
 [[troubleshoot-signals]]
 === Troubleshoot missing alerts

docs/getting-started/configure-integration-policy.asciidoc

@@ -84,7 +84,7 @@ image::images/install-endpoint/malware-protection.png[Detail of malware protecti
 [[manage-quarantined-files]]
 === Manage quarantined files
 
-When *Prevent* is enabled for malware protection, {elastic-defend} will quarantine any malicious file it finds. Specifically {elastic-defend} will remove the file from its current location, encrypt it with the encryption key `ELASTIC`, move it to a different folder, and rename it as a GUID string, such as `318e70c2-af9b-4c3a-939d-11410b9a112c`.
+When *Prevent* is enabled for malware protection, {elastic-defend} will quarantine any malicious file it finds (this includes files defined in the <<blocklist>>). Specifically {elastic-defend} will remove the file from its current location, encrypt it with the encryption key `ELASTIC`, move it to a different folder, and rename it as a GUID string, such as `318e70c2-af9b-4c3a-939d-11410b9a112c`.
 
 The quarantine folder location varies by operating system:
 

docs/reference/alert-schema.asciidoc

@@ -191,4 +191,13 @@ UIDs are linked to user profiles that are automatically created when users first
 
 Type: string[] 
 
+|N/A | `kibana.alert.intended_timestamp` a| 
+
+Shows the alert’s estimated timestamp, had the alert been created when the source event initially occurred. The value in this field is determined by the way the rule was run:
+
+* **Scheduled run**: Alerts created by scheduled runs have the same timestamp as the `kibana.alert.rule.execution.timestamp` field, which shows when the rule was executed.
+* **Manual run**: Alerts created by manual runs have a timestamp that falls within the time range specified for the manual run. For example, if you set a rule to manually run on event data from `10/01/2024 05:00 PM` to `10/07/2024 05:00 PM`, the `kibana.alert.intended_timestamp` value will be a date and time within that range.
+
+Type: date
+
 |==============================================

docs/serverless/advanced-entity-analytics/ml-requirements.mdx

@@ -10,6 +10,10 @@ tags: [ 'serverless', 'security', 'reference', 'manage' ]
 
 To run and create ((ml)) jobs and rules, you need the appropriate <DocLink slug="/serverless/general/assign-user-roles">user role</DocLink>.
 
+Additionally, for <DocLink slug="/serverless/custom-roles">custom roles</DocLink>, to configure <DocLink slug="/serverless/security/alert-suppression">alert suppression</DocLink> for ((ml)) rules, your role needs the following index privilege:
+
+* `read` permission for the `.ml-anomalies-*` index
+
 For more information, go to [Set up ((ml-features))](((ml-docs))/setup.html).
 
 <DocCallOut title="Important" color="warning">

docs/serverless/alerts/alert-schema.mdx

@@ -904,4 +904,17 @@ The non-ECS fields listed below are beta and subject to change.
 
  </DocCell>
  </DocRow>
+ <DocRow>
+ <DocCell> `kibana.alert.intended_timestamp`</DocCell>
+ <DocCell>
+ Shows the alert’s estimated timestamp, had the alert been created when the source event initially occurred. The value in this field is determined by the way the rule was run:
+
+ * **Scheduled run**: Alerts created by scheduled runs have the same timestamp as the `kibana.alert.rule.execution.timestamp` field, which shows when the rule was executed.
+ * **Manual run**: Alerts created by manual runs have a timestamp that falls within the time range specified for the manual run. For example, if you set a rule to manually run on event data from `10/01/2024 05:00 PM` to `10/07/2024 05:00 PM`, the `kibana.alert.intended_timestamp` value will be a date and time within that range.
+
+
+ Type: date
+
+ </DocCell>
+ </DocRow>
 </DocTable>

docs/serverless/alerts/alert-suppression.mdx

@@ -9,8 +9,10 @@ status: in review
 <DocBadge template="technical preview" />
 <div id="alert-suppression"></div>
 
-<DocCallOut color="warning" title="Technical Preview">
-Alert suppression is in technical preview for event correlation rules. The functionality may be changed or removed in a future release. Elastic will work to fix any issues, but features in technical preview are not subject to the support SLA of official GA features.
+<DocCallOut color="warning" title="Requirements and notice">
+ - ((ml-cap)) rules have <DocLink slug="/serverless/security/ml-requirements">additional requirements</DocLink> for alert suppression.
+
+ - Alert suppression is in technical preview for threshold, indicator match, event correlation, and new terms rules. The functionality may be changed or removed in a future release. Elastic will work to fix any issues, but features in technical preview are not subject to the support SLA of official GA features.
 </DocCallOut>
 
 Alert suppression allows you to reduce the number of repeated or duplicate detection alerts created by these detection rule types:

docs/serverless/edr-install-config/configure-endpoint-integration-policy.mdx

@@ -103,7 +103,7 @@ Endpoint Protection Complete customers can customize these notifications using t
 
 ### Manage quarantined files
 
-When **Prevent** is enabled for malware protection, ((elastic-defend)) will quarantine any malicious file it finds. Specifically ((elastic-defend)) will remove the file from its current location, encrypt it with the encryption key `ELASTIC`, move it to a different folder, and rename it as a GUID string, such as `318e70c2-af9b-4c3a-939d-11410b9a112c`.
+When **Prevent** is enabled for malware protection, ((elastic-defend)) will quarantine any malicious file it finds (this includes files defined in the <DocLink slug="/serverless/security/blocklist">blocklist</DocLink>). Specifically ((elastic-defend)) will remove the file from its current location, encrypt it with the encryption key `ELASTIC`, move it to a different folder, and rename it as a GUID string, such as `318e70c2-af9b-4c3a-939d-11410b9a112c`.
 
 The quarantine folder location varies by operating system: