Skip to content

Latest commit

 

History

History
161 lines (114 loc) · 8.8 KB

FAQ.md

File metadata and controls

161 lines (114 loc) · 8.8 KB
Raw

FAQ

What metrics does this exporter report?

The BOSH Prometheus Exporter gets the metrics from the BOSH Director, who gathers them from each VM BOSH Agent. The metrics that are being reported are pretty basic, but include:

  • Deployment metrics:
    • Releases in use
    • Stemcells in use
    • Number of Instances
  • Job metrics:
    • Health status
    • CPU
    • Load
    • Memory
    • Swap
    • System, Ephemeral and Persistent disk
  • Process metrics:
    • Health status
    • Uptime
    • CPU
    • Memory

How can I get more detailed metrics from each VM?

If you want to get more detailed VM system metrics, like disk I/O, network traffic, ..., it is recommended to deploy the Prometheus Node Exporter on each VM.

What are the caveats when using this exporter?

In order to get the metrics, the exporter calls the BOSH Director Instance details endpoint. This request results in potentially long running operations against each BOSH Agent, so such requests start a Director Task. Therefore, each exporter scrape will generate a new Director Task per deployment. This will NOT hurt your BOSH performance, but has the nasty effect that generates thousand of tasks per scrape (i.e. scrapping each minute will generate 1440 tasks per deployment per day).

It is, therefore, recommended to increase the scrape interval and the scrape timeout for this exporter:

scrape_configs:
  - job_name: bosh_exporter
    scrape_interval: 2m
    scrape_timeout: 1m

A longer scrape interval means less real time metrics, but for most use cases, this will be enough, specially when combined with the Node Exporter.

How can I get BOSH metrics without the above caveats?

Graphite Exporter

An alternative approach to gather BOSH metrics without using this exporter is to use the Graphite Exporter and configure a metric mapping:

*.*.*.*.system_healthy
name="bosh_job_healthy"
bosh_deployment="$1"
bosh_job_name="$2"
bosh_job_id="$3"
...

Then you will need to enable the Graphite Health Monitor plugin at your BOSH Health Monitor configuration pointing to the Graphite Exporter IP address.

BOSH HM metrics forwarder

Another alternative approach is to use the BOSH HM metrics forwarder that will forward BOSH health metrics into Loggregator thus making them available in the Cloud Foundry Firehose. Later, you can use the Cloud Foundry Firehose Exporter to get BOSH metrics.

Downsides

The downside of the above approaches is that you will NOT get the same level of metrics that this exporter reports and you cannot use the service discovery approach.

How can I enable only a particular collector?

The filter.collectors command flag allows you to filter what collectors will be enabled (if not set, all collectors will be enabled by default). Possible values are Deployments, Jobs, ServiceDiscovery (or a combination of them).

How can I filter by a particular BOSH deployment?

The filter.deployments command flag allows you to filter what BOSH deployments will be reported.

How can I filter by a particular BOSH AZ?

The filter.azs command flag allows you to filter what BOSH AZs will be reported.

Can I target multiple BOSH Directors with a single exporter instance?

No, this exporter only supports targetting a single BOSH Director. If you want to get metrics from several directors, you will need to use one exporter per director.

How can I get the BOSH CA certificate?

Communication between the exporter and the BOSH Director uses HTTPS. Actually, there is no way to disable the SSL certificate validation, so therefore, the certificates must be created setting a Subject Alternative Name (SAN) with the IP address of the BOSH Director; otherwise, you will get the following error message:

x509: cannot validate certificate for X.X.X.X because it doesn't contain any IP SANs

In order to generate the proper certificates, please refer to the Director SSL Certificate Configuration documentation.

Later, when starting the bosh_exporter you must specify the bosh.ca-cert-file command line flag pointing to the location of the ca.crt file.

For testing purposes, this repository includes the CA Cert to be used only when testing the exporter against a BOSH Lite.

How can I use the Service Discovery?

If you don't want to configure manually all exporters IP addresses at your prometheus configuration file, you can use the Prometheus file-based service discovery mechanism. Just point the file_sd_configs configuration to the output file (sd.filename command flag) of this exporter and use the Prometheus relabel configuration to get the IP address:

scrape_configs:
  - job_name: node_exporter
    file_sd_configs:
      - files:
        - /var/vcap/store/bosh_exporter/bosh_target_groups.json
    relabel_configs:
      - source_labels: [__meta_bosh_job_process_name]
        regex: node_exporter
        action: keep
      - source_labels: [__address__]
        regex: "(.*)"
        target_label: __address__
        replacement: "${1}:9100"

How can I filter the Service Discovery output file by a particular exporter?

The sd.processes_regexp command flag allows you to filter what BOSH Job processes will be reported.

Why is the BOSH Service Discovery a collector?

There are mainly two reasons:

  • Prometheus Service Discovery is not pluggable, which means that you either incorporate the BOSH Service Discovery as part of the official Prometheus core code or you create a separate executable that produces an output file that can be used by the Prometheus file-based service discovery mechanism. We decided to use the file-based service discovery mechanism because it was easier for us to test this approach.
  • We want to minimize the number of calls to the BOSH Director (see the above caveats). Having a different executable means that in order to get the BOSH Job IPs and processes we will need to generate a new Director Task. Using a new collector within this exporter allows us to reuse the same deployment calls.

What is the recommended deployment strategy?

Prometheus advises to collocate exporters near the metrics source, in this case, that means colocating this exporter within your BOSH Director VM. We encourage you to follow this approach whenever is possible.

But the downside of the above advice is when using the Service Discovery mechanism. In this case, the exporter must be located at the Prometheus VM in order to access the service discovery output file.

I have a question but I don't see it answered at this FAQ

We will be glad to address any questions not answered here. Please, just open a new issue.