Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[gcp] Clarify service account privileges #3206

Merged
merged 1 commit into from
Apr 28, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 23 additions & 14 deletions packages/gcp/_dev/build/docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,26 +6,37 @@ The Google Cloud integration collects and parses Google Cloud [Audit Logs](https

## Authentication

To use this Google Cloud Platform (GCP) integration, you need to set up a *Service Account* with a few *Roles* and a *Service Account Key* to access data on your GCP project.
To use this Google Cloud Platform (GCP) integration, you need to set up a
*Service Account* with a *Role* and a *Service Account Key* to access data on
your GCP project.

### Service Account

First, you need to [create a Service Account](https://cloud.google.com/iam/docs/creating-managing-service-accounts). A Service Account (SA) is a particular type of Google account intended to represent a non-human user who needs to access the GCP resources.

The Elastic Agent uses the SA to access data on Google Cloud Platform using the Google APIs.

### Roles
If you haven't already, this might be a good moment to check out the [best
practices for securing service
accounts](https://cloud.google.com/iam/docs/best-practices-for-securing-service-accounts)
guide.

You need to grant your Service Account (SA) access to Google Cloud Platform resources adding one or more roles.
### Role

For this integration to work, you need to add the following roles to your SA:
You need to grant your Service Account (SA) access to Google Cloud Platform
resources by assigning a role to the account. In order to assign minimal
privileges, create a custom role that has only the privileges required by Agent.
Those privileges are:

- `Compute Viewer`
- `Monitoring Viewer`
- `Pub/Sub Viewer`
- `Pub/Sub Subscriber`
- `pubsub.subscriptions.consume`
- `pubsub.subscriptions.create` *
- `pubsub.subscriptions.get`
- `pubsub.topics.attachSubscription` *

Always follow the "principle of least privilege" when adding a new role to your SA. If you haven't already, this might be a good moment to check out the [best practices for securing service accounts](https://cloud.google.com/iam/docs/best-practices-for-securing-service-accounts) guide.
\* Only required if Agent is expected to create a new subscription. If you
create the subscriptions yourself you may omit these privileges.

After you have created the custom role, assign the role to your service account.

### Service Account Keys

Expand All @@ -37,8 +48,6 @@ From the list of SA:
2. From the Keys section, click "Add key" > "Create new key" and select JSON as the type.
3. Download and store the generated private key securely (remember, the private key can't be recovered from GCP if lost).

Optional: take some time to review the GCP's [best practices for managing service account keys](https://cloud.google.com/iam/docs/best-practices-for-managing-service-account-keys).

## Configure the Integration Settings

The next step is to configure the general integration settings used for all logs from the supported services (Audit, DNS, Firewall, and VPC Flow).
Expand Down Expand Up @@ -89,7 +98,7 @@ At a high level, the steps required are:

- Visit "Logging" > "Log Router" > "Create Sink" and provide a sink name and description.
- In "Sink destination", select "Cloud Pub/Sub topic" as the sink service. Select an existing topic or "Create a topic". Note the topic name, as it will be provided in the Topic field in the Elastic agent configuration.
- If you created a new topic, you must remember to go to that topic and create a subscription for it. A subscription directs messages on a topic to subscribers. Note the "Subscription ID", as it will need to be entered in the "Subscription name" field in the ingtegration settings.
- If you created a new topic, you must remember to go to that topic and create a subscription for it. A subscription directs messages on a topic to subscribers. Note the "Subscription ID", as it will need to be entered in the "Subscription name" field in the integration settings.
- Under "Choose logs to include in sink", for example add `logName:"cloudaudit.googleapis.com"` in the "Inclusion filter" to include all audit logs.

This is just an example; you will need to create your filter expression to select the log types you want to export to the Pub/Sub topic.
Expand All @@ -109,7 +118,7 @@ resource.labels.subnetwork_name"=[SUBNET_NAME]"
#
resource.type="gce_firewall_rule" AND
log_id("cloudaudit.googleapis.com/activity") AND
protoPayload.methodName:"firewalls.delete"
protoPayload.methodName:"firewalls.delete"

#
# DNS: all DNS queries
Expand All @@ -124,7 +133,7 @@ log_id("compute.googleapis.com/firewall") AND
jsonPayload.remote_location.country=[COUNTRY_ISO_ALPHA_3]
```

Start working on your query using the Google Cloud [Logs Explorer](https://console.cloud.google.com/logs/query), so you can preview and pintpoint the exact log types you want to forward to your Elastic Stack.
Start working on your query using the Google Cloud [Logs Explorer](https://console.cloud.google.com/logs/query), so you can preview and pinpoint the exact log types you want to forward to your Elastic Stack.

To learn more, please read how to [Build queries in the Logs Explorer](https://cloud.google.com/logging/docs/view/building-queries), and take a look at the [Sample queries using the Logs Explorer](https://cloud.google.com/logging/docs/view/query-library-preview) page in the Google Cloud docs.

Expand Down
5 changes: 5 additions & 0 deletions packages/gcp/changelog.yml
Original file line number Diff line number Diff line change
@@ -1,4 +1,9 @@
# newer versions go on top
- version: "1.6.1"
changes:
- description: Clarify the GCP privileges required by the Pub/Sub input.
type: enhancement
link: https:/elastic/integrations/pull/3206
- version: "1.6.0"
changes:
- description: Update to ECS 8.2
Expand Down
37 changes: 23 additions & 14 deletions packages/gcp/docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,26 +6,37 @@ The Google Cloud integration collects and parses Google Cloud [Audit Logs](https

## Authentication

To use this Google Cloud Platform (GCP) integration, you need to set up a *Service Account* with a few *Roles* and a *Service Account Key* to access data on your GCP project.
To use this Google Cloud Platform (GCP) integration, you need to set up a
*Service Account* with a *Role* and a *Service Account Key* to access data on
your GCP project.

### Service Account

First, you need to [create a Service Account](https://cloud.google.com/iam/docs/creating-managing-service-accounts). A Service Account (SA) is a particular type of Google account intended to represent a non-human user who needs to access the GCP resources.

The Elastic Agent uses the SA to access data on Google Cloud Platform using the Google APIs.

### Roles
If you haven't already, this might be a good moment to check out the [best
practices for securing service
accounts](https://cloud.google.com/iam/docs/best-practices-for-securing-service-accounts)
guide.

You need to grant your Service Account (SA) access to Google Cloud Platform resources adding one or more roles.
### Role

For this integration to work, you need to add the following roles to your SA:
You need to grant your Service Account (SA) access to Google Cloud Platform
resources by assigning a role to the account. In order to assign minimal
privileges, create a custom role that has only the privileges required by Agent.
Those privileges are:

- `Compute Viewer`
- `Monitoring Viewer`
- `Pub/Sub Viewer`
- `Pub/Sub Subscriber`
- `pubsub.subscriptions.consume`
- `pubsub.subscriptions.create` *
- `pubsub.subscriptions.get`
- `pubsub.topics.attachSubscription` *

Always follow the "principle of least privilege" when adding a new role to your SA. If you haven't already, this might be a good moment to check out the [best practices for securing service accounts](https://cloud.google.com/iam/docs/best-practices-for-securing-service-accounts) guide.
\* Only required if Agent is expected to create a new subscription. If you
create the subscriptions yourself you may omit these privileges.

After you have created the custom role, assign the role to your service account.

### Service Account Keys

Expand All @@ -37,8 +48,6 @@ From the list of SA:
2. From the Keys section, click "Add key" > "Create new key" and select JSON as the type.
3. Download and store the generated private key securely (remember, the private key can't be recovered from GCP if lost).

Optional: take some time to review the GCP's [best practices for managing service account keys](https://cloud.google.com/iam/docs/best-practices-for-managing-service-account-keys).

## Configure the Integration Settings

The next step is to configure the general integration settings used for all logs from the supported services (Audit, DNS, Firewall, and VPC Flow).
Expand Down Expand Up @@ -89,7 +98,7 @@ At a high level, the steps required are:

- Visit "Logging" > "Log Router" > "Create Sink" and provide a sink name and description.
- In "Sink destination", select "Cloud Pub/Sub topic" as the sink service. Select an existing topic or "Create a topic". Note the topic name, as it will be provided in the Topic field in the Elastic agent configuration.
- If you created a new topic, you must remember to go to that topic and create a subscription for it. A subscription directs messages on a topic to subscribers. Note the "Subscription ID", as it will need to be entered in the "Subscription name" field in the ingtegration settings.
- If you created a new topic, you must remember to go to that topic and create a subscription for it. A subscription directs messages on a topic to subscribers. Note the "Subscription ID", as it will need to be entered in the "Subscription name" field in the integration settings.
- Under "Choose logs to include in sink", for example add `logName:"cloudaudit.googleapis.com"` in the "Inclusion filter" to include all audit logs.

This is just an example; you will need to create your filter expression to select the log types you want to export to the Pub/Sub topic.
Expand All @@ -109,7 +118,7 @@ resource.labels.subnetwork_name"=[SUBNET_NAME]"
#
resource.type="gce_firewall_rule" AND
log_id("cloudaudit.googleapis.com/activity") AND
protoPayload.methodName:"firewalls.delete"
protoPayload.methodName:"firewalls.delete"

#
# DNS: all DNS queries
Expand All @@ -124,7 +133,7 @@ log_id("compute.googleapis.com/firewall") AND
jsonPayload.remote_location.country=[COUNTRY_ISO_ALPHA_3]
```

Start working on your query using the Google Cloud [Logs Explorer](https://console.cloud.google.com/logs/query), so you can preview and pintpoint the exact log types you want to forward to your Elastic Stack.
Start working on your query using the Google Cloud [Logs Explorer](https://console.cloud.google.com/logs/query), so you can preview and pinpoint the exact log types you want to forward to your Elastic Stack.

To learn more, please read how to [Build queries in the Logs Explorer](https://cloud.google.com/logging/docs/view/building-queries), and take a look at the [Sample queries using the Logs Explorer](https://cloud.google.com/logging/docs/view/query-library-preview) page in the Google Cloud docs.

Expand Down
2 changes: 1 addition & 1 deletion packages/gcp/manifest.yml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
name: gcp
title: Google Cloud Platform
version: 1.6.0
version: 1.6.1
release: ga
description: Collect logs from Google Cloud Platform with Elastic Agent.
type: integration
Expand Down