Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Cherry-pick #16671 to 7.x: Add cloudfoundry module to metricbeat #16741

Merged
merged 6 commits into from
Mar 3, 2020
Merged

Cherry-pick #16671 to 7.x: Add cloudfoundry module to metricbeat #16741

merged 6 commits into from
Mar 3, 2020

Conversation

blakerouse
Copy link
Contributor

Cherry-pick of PR #16671 to 7.x branch. Original message:

What does this PR do?

Adds a new cloudfoundry module to x-pack/metricbeat. Adds 3 metric sets container, counter, and value.

This module/metricsets are different from standard metricsets as the Cloud Foundry loggregator pushes to metrics to the client. Each metricset shares the same connection from the module so it does not create a connection per metricset.

Why is it important?

Adds support for metrics from cloudfoundry.

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works

Author's Checklist

  • Receives metrics from Cloud Foundry loggregator.
  • Selecting individual metricsets works.

How to test this PR locally

With a running Cloud Foundry run metricbeat with the following configuration:

metricbeat.modules:
  - &cloudfoundry
    module: cloudfoundry
    api_address: https://api.dev.cfdev.sh
    client_id: firehose-to-filebeat
    client_secret: verysecret
    ssl:
      verification_mode: none

processors:
  - add_cloudfoundry_metadata:
      <<: *cloudfoundry

output.console:
  pretty: true

Related issues

Use cases

Screenshots

Logs

@blakerouse blakerouse requested a review from a team as a code owner March 2, 2020 22:49
@mtojek
Copy link
Contributor

mtojek commented Mar 3, 2020

I think you need to rebase this PR against the 7.x branch (Go modules).

@blakerouse
Copy link
Contributor Author

@mtojek I have rebased on latest 7.x branch for go modules. If you can give it another look.

mtojek
mtojek approved these changes Mar 3, 2020
View reviewed changes
Copy link
Contributor

@mtojek mtojek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If it passes the build phase, feel free to merge it!

@blakerouse
Add cloudfoundry module to metricbeat (elastic#16671)
44e4375 
* Split up parts into different metricsets.

* Add value and container metricsets.

* Add documentation.

* Revert metricbeat.yml.

* Run make update.

* Fix documentation add changelog.

* Fix bad change in azure module.

* Run make update.

* Improve reference config.

* Document Kibana Space ID setting (elastic#16622)

* Document Kibana Space ID setting

* Update reference YML files

* Update libbeat/docs/shared-kibana-config.asciidoc

Co-Authored-By: DeDe Morton <[email protected]>

* Updating reference YML files

* Adding link to doc

Co-authored-by: DeDe Morton <[email protected]>

* Remove the special pins type for `ca_sha256` (elastic#16615)

We have changed the type from a special custom type to a slice of
string. This reduce the number of exposed types and make it easier in
general to work with in the tests.

* Remove feature.Describer and enhance Details (elastic#16658)

* Make memqueue.Broker internal (now memqueue.broker) (elastic#16667)

* [docs] Add skeleton files for 7.7 release highlights and breaking changes (elastic#16584)

* [Metricbeat] Add vpc metricset for aws module (elastic#16111)

* Add vpn into vpc metricset
* Add NATGateway and TrasitGateway

* Update cisco.asciidoc (elastic#16434)

* Update cisco.asciidoc

Making Elasticsearch explicit in the instructions

* Update docs source in the module dir

Co-authored-by: Loek van Gool <[email protected]>

* Fix wording in dev guide (elastic#16497)

* Added NetBeat (elastic#12612)

* Close files earlier when generating fields files (elastic#16681)

Fields generator keeps all open files while generating the global
fields.yml file. This is not needed and may lead to "too many open
files" errors in some environments.

* [Metricbeat] Add new module for Redis Enterprise (elastic#16482)

Add metricsets: node, proxy

* Add doppler_address, uaa_address, rlp_address to config.reference.yml.

Co-authored-by: Shaunak Kashyap <[email protected]>
Co-authored-by: DeDe Morton <[email protected]>
Co-authored-by: Pier-Hugues Pellerin <[email protected]>
Co-authored-by: Steffen Siering <[email protected]>
Co-authored-by: Fae Charlton <[email protected]>
Co-authored-by: kaiyan-sheng <[email protected]>
Co-authored-by: Loek van Gool <[email protected]>
Co-authored-by: Harold Schreckengost <[email protected]>
Co-authored-by: Jaime Soriano Pastor <[email protected]>
Co-authored-by: Marcin Tojek <[email protected]>
(cherry picked from commit 07b03a7)
@blakerouse
Fix changelog.
0c8d6fb
@blakerouse
Run mage fmt.
eb22a66
@blakerouse
Fix imports to have v7.
67541ce
@blakerouse
Run mage fmt update.
622547d
@blakerouse
Fix changelog.
2c042d7
@blakerouse blakerouse merged commit 5f7c5a6 into elastic:7.x Mar 3, 2020
@blakerouse blakerouse deleted the backport_16671_7.x branch March 3, 2020 20:05
@blakerouse blakerouse added the test-plan Add this PR to be manual test plan label Mar 23, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@mtojek mtojek mtojek approved these changes

Assignees
No one assigned
Labels
backport review test-plan Add this PR to be manual test plan
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@blakerouse @mtojek