chore: some more cleanup

pkg/query-service/app/integrations/builtin_integrations/postgres/config/collect-logs.md

@@ -68,6 +68,12 @@ exporters:
  headers:
  "signoz-access-token": "${env:SIGNOZ_INGESTION_KEY}"
 
+ # export to local collector
+ # otlp/local:
+ # endpoint: "localhost:4317"
+ # tls:
+ # insecure: true
+
 service:
  pipelines:
  postgresql:
@@ -82,19 +88,22 @@ Set the following environment variables in your otel-collector environment:
 
 ```bash
 
-# path of postgresql log file
+# path of Postgres server log file. must be accessible by the otel collector
 export POSTGRESQL_LOG_FILE=/usr/local/var/log/postgres.log
 
-# region specific signoz cloud ingestion endpoint
+# region specific SigNoz cloud ingestion endpoint
 export OTLP_DESTINATION_ENDPOINT="ingest.us.signoz.cloud:443"
 
-# your signoz ingestion key
-export SIGNOZ_INGESTION_KEY="key"
+# your SigNoz ingestion key
+export SIGNOZ_INGESTION_KEY="signoz-ingestion-key"
 
 ```
 
 #### Use collector config file
 
-Make the `postgres-logs-collection-config.yaml` file available to your otel collector and add the flag `--config postgres-logs-collection-config.yaml` to the command for running your otel collector. 
+Make the collector config file available to your otel collector and use it by adding the following flag to the command for running your collector 
+```bash
+--config postgres-logs-collection-config.yaml
+``` 
 Note: the collector can use multiple config files, specified by multiple occurrences of the --config flag.
 

pkg/query-service/app/integrations/builtin_integrations/postgres/config/collect-metrics.md

@@ -1,5 +1,7 @@
 ### Collect Postgres Metrics
 
+You can configure Postgres metrics collection by providing the required collector config to your collector.
+
 #### Create collector config file
 
 Save the following config for collecting postgres metrics in a file named `postgres-metrics-collection-config.yaml`
@@ -12,7 +14,7 @@ receivers:
  # The frequency at which to collect metrics from the Postgres instance.
  collection_interval: 60s
  # The username used to access the postgres instance
- username: monitoring
+ username: ${env:POSTGRESQL_USERNAME}
  # The password used to access the postgres instance
  password: ${env:POSTGRESQL_PASSWORD}
  # The list of databases for which the receiver will attempt to collect statistics. If an empty list is provided, the receiver will attempt to collect statistics for all non-template databases
@@ -72,21 +74,28 @@ Set the following environment variables in your otel-collector environment:
 
 ```bash
 
-# password for postgres monitoring user"
-export POSTGRESQL_PASSWORD="password"
+# password for Postgres monitoring user"
+export POSTGRESQL_USERNAME="monitoring"
+
+# password for Postgres monitoring user"
+export POSTGRESQL_PASSWORD="<PASSWORD>"
 
-# postgres endpoint reachable from the otel collector"
+# Postgres endpoint reachable from the otel collector"
 export POSTGRESQL_ENDPOINT="host:port"
 
-# region specific signoz cloud ingestion endpoint
+
+# region specific SigNoz cloud ingestion endpoint
 export OTLP_DESTINATION_ENDPOINT="ingest.us.signoz.cloud:443"
 
-# your signoz ingestion key
-export SIGNOZ_INGESTION_KEY="key"
+# your SigNoz ingestion key
+export SIGNOZ_INGESTION_KEY="signoz-ingestion-key"
 
 ```
 
 #### Use collector config file
 
-Make the `postgres-metrics-collection-config.yaml` file available to your otel collector and add the flag `--config postgres-metrics-collection-config.yaml` to the command for running your otel collector. 
+Make the collector config file available to your otel collector and use it by adding the following flag to the command for running your collector 
+```bash
+--config postgres-metrics-collection-config.yaml
+``` 
 Note: the collector can use multiple config files, specified by multiple occurrences of the --config flag.

pkg/query-service/app/integrations/builtin_integrations/postgres/config/prerequisites.md

@@ -1,29 +1,37 @@
-## Prerequisites
-
-### Prepare postgres for monitoring
-
-- Have a running postgresql instance
-- Have the monitoring user created
-- Have the monitoring user granted the necessary permissions
-
-This receiver supports PostgreSQL versions 9.6+
-
-For PostgreSQL versions 10+, run:
-
-```sql
-create user monitoring with password '<PASSWORD>';
-grant pg_monitor to monitoring;
-grant SELECT ON pg_stat_database to monitoring;
-```
-
-For PostgreSQL versions >= 9.6 and <10, run:
-
-```sql
-create user monitoring with password '<PASSWORD>';
-grant SELECT ON pg_stat_database to monitoring;
-```
+## Before You Begin 
+
+To configure metrics and logs collection for a Postgres server, you need the following.
+
+### Ensure Postgres server is prepared for monitoring
+
+- **Ensure that the Postgres server is running a supported version** 
+ Postgres versions 9.6+ are supported. 
+ You can use the following SQL statement to determine server version 
+ ```SQL
+ SELECT version();
+ ```
+
+- **If collecting metrics, ensure that there is a Postgres user with required permissions** 
+ To create a monitoring user for Postgres versions 10+, run: 
+ ```SQL
+ create user monitoring with password '<PASSWORD>';
+ grant pg_monitor to monitoring;
+ grant SELECT ON pg_stat_database to monitoring;
+ ``` 
 
-
-### Install Otel Collector
-
-To start gathering postgres logs and metrics, please [install otel collector](https://signoz.io/docs/tutorial/opentelemetry-binary-usage-in-virtual-machine/) in your deployment.
+ To create a monitoring user for Postgres versions >= 9.6 and <10, run: 
+ ```SQL
+ create user monitoring with password '<PASSWORD>';
+ grant SELECT ON pg_stat_database to monitoring;
+ ```
+
+
+### Ensure that an OTEL Collector is running with access to the Postgres server
+
+- **Ensure that an OTEL collector is running in your deployment environment** 
+ If needed, please [install an OTEL Collector](https://signoz.io/docs/tutorial/opentelemetry-binary-usage-in-virtual-machine/) 
+ If already installed, ensure that you can provide config to the collector with environment variables and command line flags. 
+
+- **Ensure that the OTEL collector can access the Postgres server** 
+ In order to collect metrics, the collector must be able to access the Postgres server as a client using the monitoring user. 
+ In order to collect logs, The collector must be able to read the Postgres server log file.

pkg/query-service/app/integrations/builtin_integrations/postgres/integration.json

@@ -1,7 +1,7 @@
 {
  "id": "postgres",
  "title": "PostgreSQL",
- "description": "Monitor postgres using logs and metrics.",
+ "description": "Monitor Postgres with metrics and logs",
  "author": {
  "name": "SigNoz",
  "email": "[email protected]",
@@ -54,17 +54,21 @@
  "data_collected": {
  "logs": [
  {
- "name": "pid",
+ "name": "Process ID",
  "path": "attributes.pid",
  "type": "string"
  }, {
- "name": "timestamp",
+ "name": "Timestamp",
  "path": "timestamp",
  "type": "timestamp"
  }, {
- "name": "severity",
- "path": "severity",
- "type": "severity"
+ "name": "Severity Text",
+ "path": "severity_text",
+ "type": "string"
+ }, {
+ "name": "Severity Number",
+ "path": "severity_number",
+ "type": "number"
  }
  ],
  "metrics": [
@@ -77,19 +81,19 @@
  {
  "name": "postgresql.bgwriter.buffers.allocated",
  "type": "sum",
- "unit": "{buffers}",
+ "unit": "number",
  "description": "Number of buffers allocated."
  },
  {
  "name": "postgresql.bgwriter.buffers.writes",
  "type": "sum",
- "unit": "{buffers}",
+ "unit": "number",
  "description": "Number of buffers written."
  },
  {
  "name": "postgresql.bgwriter.checkpoint.count",
  "type": "sum",
- "unit": "{checkpoints}",
+ "unit": "number",
  "description": "The number of checkpoints performed."
  },
  {
@@ -119,43 +123,43 @@
  {
  "name": "postgresql.connection.max",
  "type": "gauge",
- "unit": "{connections}",
+ "unit": "number",
  "description": "Configured maximum number of client connections allowed"
  },
  {
  "name": "postgresql.database.count",
  "type": "sum",
- "unit": "{databases}",
+ "unit": "number",
  "description": "Number of user databases."
  },
  {
  "name": "postgresql.database.locks",
  "type": "gauge",
- "unit": "{lock}",
+ "unit": "number",
  "description": "The number of database locks."
  },
  {
  "name": "postgresql.db_size",
  "type": "sum",
- "unit": "By",
+ "unit": "Bytes",
  "description": "The database disk usage."
  },
  {
  "name": "postgresql.deadlocks",
  "type": "sum",
- "unit": "{deadlock}",
+ "unit": "number",
  "description": "The number of deadlocks."
  },
  {
  "name": "postgresql.index.scans",
  "type": "sum",
- "unit": "{scans}",
+ "unit": "number",
  "description": "The number of index scans on a table."
  },
  {
  "name": "postgresql.index.size",
  "type": "gauge",
- "unit": "By",
+ "unit": "Bytes",
  "description": "The size of the index on disk."
  },
  {
@@ -167,7 +171,7 @@
  {
  "name": "postgresql.replication.data_delay",
  "type": "gauge",
- "unit": "By",
+ "unit": "Bytes",
  "description": "The amount of data delayed in replication."
  },
  {
@@ -185,49 +189,49 @@
  {
  "name": "postgresql.sequential_scans",
  "type": "sum",
- "unit": "{sequential_scan}",
+ "unit": "number",
  "description": "The number of sequential scans."
  },
  {
  "name": "postgresql.table.count",
  "type": "sum",
- "unit": "{table}",
+ "unit": "number",
  "description": "Number of user tables in a database."
  },
  {
  "name": "postgresql.table.size",
  "type": "sum",
- "unit": "By",
+ "unit": "Bytes",
  "description": "Disk space used by a table."
  },
  {
  "name": "postgresql.table.vacuum.count",
  "type": "sum",
- "unit": "{vacuums}",
+ "unit": "number",
  "description": "Number of times a table has manually been vacuumed."
  },
  {
  "name": "postgresql.temp_files",
  "type": "sum",
- "unit": "{temp_file}",
+ "unit": "number",
  "description": "The number of temp files."
  },
  {
  "name": "postgresql.wal.age",
  "type": "gauge",
- "unit": "s",
+ "unit": "seconds",
  "description": "Age of the oldest WAL file."
  },
  {
  "name": "postgresql.wal.delay",
  "type": "gauge",
- "unit": "s",
+ "unit": "seconds",
  "description": "Time between flushing recent WAL locally and receiving notification that the standby server has completed an operation with it."
  },
  {
  "name": "postgresql.wal.lag",
  "type": "gauge",
- "unit": "s",
+ "unit": "seconds",
  "description": "Time between flushing recent WAL locally and receiving notification that the standby server has completed an operation with it."
  }
  ]

pkg/query-service/app/integrations/builtin_integrations/postgres/overview.md

@@ -1,3 +1,5 @@
 ### Monitor Postgres with SigNoz
 
-Parse your Postgres logs and collect key metrics.
+Collect key Postgres metrics and view them with an out of the box dashboard. 
+
+Collect and parse Postgres logs to populate timestamp, severity, and other log attributes for better querying and aggregation.