diff --git a/pkg/query-service/app/integrations/builtin_integrations/postgres/config/collect-logs.md b/pkg/query-service/app/integrations/builtin_integrations/postgres/config/collect-logs.md index cbdbe6f1ac..f49e722856 100644 --- a/pkg/query-service/app/integrations/builtin_integrations/postgres/config/collect-logs.md +++ b/pkg/query-service/app/integrations/builtin_integrations/postgres/config/collect-logs.md @@ -68,6 +68,12 @@ exporters: headers: "signoz-access-token": "${env:SIGNOZ_INGESTION_KEY}" + # export to local collector + # otlp/local: + # endpoint: "localhost:4317" + # tls: + # insecure: true + service: pipelines: postgresql: @@ -82,19 +88,22 @@ Set the following environment variables in your otel-collector environment: ```bash -# path of postgresql log file +# path of Postgres server log file. must be accessible by the otel collector export POSTGRESQL_LOG_FILE=/usr/local/var/log/postgres.log -# region specific signoz cloud ingestion endpoint +# region specific SigNoz cloud ingestion endpoint export OTLP_DESTINATION_ENDPOINT="ingest.us.signoz.cloud:443" -# your signoz ingestion key -export SIGNOZ_INGESTION_KEY="key" +# your SigNoz ingestion key +export SIGNOZ_INGESTION_KEY="signoz-ingestion-key" ``` #### Use collector config file -Make the `postgres-logs-collection-config.yaml` file available to your otel collector and add the flag `--config postgres-logs-collection-config.yaml` to the command for running your otel collector. +Make the collector config file available to your otel collector and use it by adding the following flag to the command for running your collector +```bash +--config postgres-logs-collection-config.yaml +``` Note: the collector can use multiple config files, specified by multiple occurrences of the --config flag. diff --git a/pkg/query-service/app/integrations/builtin_integrations/postgres/config/collect-metrics.md b/pkg/query-service/app/integrations/builtin_integrations/postgres/config/collect-metrics.md index e9814fac4c..ad1971fe35 100644 --- a/pkg/query-service/app/integrations/builtin_integrations/postgres/config/collect-metrics.md +++ b/pkg/query-service/app/integrations/builtin_integrations/postgres/config/collect-metrics.md @@ -1,5 +1,7 @@ ### Collect Postgres Metrics +You can configure Postgres metrics collection by providing the required collector config to your collector. + #### Create collector config file Save the following config for collecting postgres metrics in a file named `postgres-metrics-collection-config.yaml` @@ -12,7 +14,7 @@ receivers: # The frequency at which to collect metrics from the Postgres instance. collection_interval: 60s # The username used to access the postgres instance - username: monitoring + username: ${env:POSTGRESQL_USERNAME} # The password used to access the postgres instance password: ${env:POSTGRESQL_PASSWORD} # The list of databases for which the receiver will attempt to collect statistics. If an empty list is provided, the receiver will attempt to collect statistics for all non-template databases @@ -72,21 +74,28 @@ Set the following environment variables in your otel-collector environment: ```bash -# password for postgres monitoring user" -export POSTGRESQL_PASSWORD="password" +# password for Postgres monitoring user" +export POSTGRESQL_USERNAME="monitoring" + +# password for Postgres monitoring user" +export POSTGRESQL_PASSWORD="" -# postgres endpoint reachable from the otel collector" +# Postgres endpoint reachable from the otel collector" export POSTGRESQL_ENDPOINT="host:port" -# region specific signoz cloud ingestion endpoint + +# region specific SigNoz cloud ingestion endpoint export OTLP_DESTINATION_ENDPOINT="ingest.us.signoz.cloud:443" -# your signoz ingestion key -export SIGNOZ_INGESTION_KEY="key" +# your SigNoz ingestion key +export SIGNOZ_INGESTION_KEY="signoz-ingestion-key" ``` #### Use collector config file -Make the `postgres-metrics-collection-config.yaml` file available to your otel collector and add the flag `--config postgres-metrics-collection-config.yaml` to the command for running your otel collector. +Make the collector config file available to your otel collector and use it by adding the following flag to the command for running your collector +```bash +--config postgres-metrics-collection-config.yaml +``` Note: the collector can use multiple config files, specified by multiple occurrences of the --config flag. diff --git a/pkg/query-service/app/integrations/builtin_integrations/postgres/config/prerequisites.md b/pkg/query-service/app/integrations/builtin_integrations/postgres/config/prerequisites.md index 249279e82b..82c6e72661 100644 --- a/pkg/query-service/app/integrations/builtin_integrations/postgres/config/prerequisites.md +++ b/pkg/query-service/app/integrations/builtin_integrations/postgres/config/prerequisites.md @@ -1,29 +1,37 @@ -## Prerequisites - -### Prepare postgres for monitoring - -- Have a running postgresql instance -- Have the monitoring user created -- Have the monitoring user granted the necessary permissions - -This receiver supports PostgreSQL versions 9.6+ - -For PostgreSQL versions 10+, run: - -```sql -create user monitoring with password ''; -grant pg_monitor to monitoring; -grant SELECT ON pg_stat_database to monitoring; -``` - -For PostgreSQL versions >= 9.6 and <10, run: - -```sql -create user monitoring with password ''; -grant SELECT ON pg_stat_database to monitoring; -``` +## Before You Begin + +To configure metrics and logs collection for a Postgres server, you need the following. + +### Ensure Postgres server is prepared for monitoring + +- **Ensure that the Postgres server is running a supported version** + Postgres versions 9.6+ are supported. + You can use the following SQL statement to determine server version + ```SQL + SELECT version(); + ``` + +- **If collecting metrics, ensure that there is a Postgres user with required permissions** + To create a monitoring user for Postgres versions 10+, run: + ```SQL + create user monitoring with password ''; + grant pg_monitor to monitoring; + grant SELECT ON pg_stat_database to monitoring; + ``` - -### Install Otel Collector - -To start gathering postgres logs and metrics, please [install otel collector](https://signoz.io/docs/tutorial/opentelemetry-binary-usage-in-virtual-machine/) in your deployment. + To create a monitoring user for Postgres versions >= 9.6 and <10, run: + ```SQL + create user monitoring with password ''; + grant SELECT ON pg_stat_database to monitoring; + ``` + + +### Ensure that an OTEL Collector is running with access to the Postgres server + +- **Ensure that an OTEL collector is running in your deployment environment** + If needed, please [install an OTEL Collector](https://signoz.io/docs/tutorial/opentelemetry-binary-usage-in-virtual-machine/) + If already installed, ensure that you can provide config to the collector with environment variables and command line flags. + +- **Ensure that the OTEL collector can access the Postgres server** + In order to collect metrics, the collector must be able to access the Postgres server as a client using the monitoring user. + In order to collect logs, The collector must be able to read the Postgres server log file. diff --git a/pkg/query-service/app/integrations/builtin_integrations/postgres/integration.json b/pkg/query-service/app/integrations/builtin_integrations/postgres/integration.json index c1f0dcfa3d..c796a886ee 100644 --- a/pkg/query-service/app/integrations/builtin_integrations/postgres/integration.json +++ b/pkg/query-service/app/integrations/builtin_integrations/postgres/integration.json @@ -1,7 +1,7 @@ { "id": "postgres", "title": "PostgreSQL", - "description": "Monitor postgres using logs and metrics.", + "description": "Monitor Postgres with metrics and logs", "author": { "name": "SigNoz", "email": "integrations@signoz.io", @@ -54,17 +54,21 @@ "data_collected": { "logs": [ { - "name": "pid", + "name": "Process ID", "path": "attributes.pid", "type": "string" }, { - "name": "timestamp", + "name": "Timestamp", "path": "timestamp", "type": "timestamp" }, { - "name": "severity", - "path": "severity", - "type": "severity" + "name": "Severity Text", + "path": "severity_text", + "type": "string" + }, { + "name": "Severity Number", + "path": "severity_number", + "type": "number" } ], "metrics": [ @@ -77,19 +81,19 @@ { "name": "postgresql.bgwriter.buffers.allocated", "type": "sum", - "unit": "{buffers}", + "unit": "number", "description": "Number of buffers allocated." }, { "name": "postgresql.bgwriter.buffers.writes", "type": "sum", - "unit": "{buffers}", + "unit": "number", "description": "Number of buffers written." }, { "name": "postgresql.bgwriter.checkpoint.count", "type": "sum", - "unit": "{checkpoints}", + "unit": "number", "description": "The number of checkpoints performed." }, { @@ -119,43 +123,43 @@ { "name": "postgresql.connection.max", "type": "gauge", - "unit": "{connections}", + "unit": "number", "description": "Configured maximum number of client connections allowed" }, { "name": "postgresql.database.count", "type": "sum", - "unit": "{databases}", + "unit": "number", "description": "Number of user databases." }, { "name": "postgresql.database.locks", "type": "gauge", - "unit": "{lock}", + "unit": "number", "description": "The number of database locks." }, { "name": "postgresql.db_size", "type": "sum", - "unit": "By", + "unit": "Bytes", "description": "The database disk usage." }, { "name": "postgresql.deadlocks", "type": "sum", - "unit": "{deadlock}", + "unit": "number", "description": "The number of deadlocks." }, { "name": "postgresql.index.scans", "type": "sum", - "unit": "{scans}", + "unit": "number", "description": "The number of index scans on a table." }, { "name": "postgresql.index.size", "type": "gauge", - "unit": "By", + "unit": "Bytes", "description": "The size of the index on disk." }, { @@ -167,7 +171,7 @@ { "name": "postgresql.replication.data_delay", "type": "gauge", - "unit": "By", + "unit": "Bytes", "description": "The amount of data delayed in replication." }, { @@ -185,49 +189,49 @@ { "name": "postgresql.sequential_scans", "type": "sum", - "unit": "{sequential_scan}", + "unit": "number", "description": "The number of sequential scans." }, { "name": "postgresql.table.count", "type": "sum", - "unit": "{table}", + "unit": "number", "description": "Number of user tables in a database." }, { "name": "postgresql.table.size", "type": "sum", - "unit": "By", + "unit": "Bytes", "description": "Disk space used by a table." }, { "name": "postgresql.table.vacuum.count", "type": "sum", - "unit": "{vacuums}", + "unit": "number", "description": "Number of times a table has manually been vacuumed." }, { "name": "postgresql.temp_files", "type": "sum", - "unit": "{temp_file}", + "unit": "number", "description": "The number of temp files." }, { "name": "postgresql.wal.age", "type": "gauge", - "unit": "s", + "unit": "seconds", "description": "Age of the oldest WAL file." }, { "name": "postgresql.wal.delay", "type": "gauge", - "unit": "s", + "unit": "seconds", "description": "Time between flushing recent WAL locally and receiving notification that the standby server has completed an operation with it." }, { "name": "postgresql.wal.lag", "type": "gauge", - "unit": "s", + "unit": "seconds", "description": "Time between flushing recent WAL locally and receiving notification that the standby server has completed an operation with it." } ] diff --git a/pkg/query-service/app/integrations/builtin_integrations/postgres/overview.md b/pkg/query-service/app/integrations/builtin_integrations/postgres/overview.md index 4af57e6b20..ac6e061eca 100644 --- a/pkg/query-service/app/integrations/builtin_integrations/postgres/overview.md +++ b/pkg/query-service/app/integrations/builtin_integrations/postgres/overview.md @@ -1,3 +1,5 @@ ### Monitor Postgres with SigNoz -Parse your Postgres logs and collect key metrics. +Collect key Postgres metrics and view them with an out of the box dashboard. + +Collect and parse Postgres logs to populate timestamp, severity, and other log attributes for better querying and aggregation.