Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Metricbeat][Aerospike Module] Add support for TLS #38126

Merged
merged 23 commits into from
Apr 13, 2024
Merged
Show file tree
Hide file tree
Changes from 22 commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
b7221a6
Add TLS Support for Metricbeat module aerospike
herrBez Feb 22, 2024
7ba6b99
Module aerospike: add support for basic auth
herrBez Feb 23, 2024
d89729b
Move the initialization of the clientpolicy from the metricset to the…
herrBez Feb 23, 2024
b3f3677
Aerospike: add test to check the initialization of ssl and user/passw…
herrBez Feb 23, 2024
972a183
Aerospike: Add support for cluster_name setting
herrBez Feb 23, 2024
12d0878
Aerospike: Add test for ClusterName
herrBez Feb 23, 2024
59df82f
Aerospike module: add commented username, password, cluster_name and …
herrBez Feb 23, 2024
8965a52
Aerospike Module: Fix unit test
herrBez Feb 23, 2024
baec16b
Fix test to avoid using hard-coded value
herrBez Feb 28, 2024
0e7291c
Adjust configuration of aerospike to show username/password and TLS s…
herrBez Mar 7, 2024
4517d20
Executed goimports -w -local github.com/elastic metricbeat/module/aer…
herrBez Mar 7, 2024
d5c1279
Improve Test error messages to provide more context
herrBez Mar 15, 2024
26dc676
Drop support for Basic Auth that would
herrBez Mar 15, 2024
62ebf2a
Fix Documentation by adding SSL reference
herrBez Mar 15, 2024
5635e75
Update metricbeat/module/aerospike/aerospike_test.go
herrBez Apr 8, 2024
76772c9
Address reviewer's request:
herrBez Apr 8, 2024
7d057a4
Run mage config
herrBez Apr 8, 2024
95544f3
mage check
shmsr Apr 8, 2024
689c651
Merge branch 'main' into aerospike-tls-spike
shmsr Apr 8, 2024
4a15b5b
Replace "Optional SSL/TLS. By default is off." with disabled by default
herrBez Apr 8, 2024
5db38b6
Convert ClusterName to a simple string instead of pointer to a string
herrBez Apr 8, 2024
43f4a74
Merge branch 'main' into aerospike-tls-spike
herrBez Apr 13, 2024
c737807
Update x-pack/metricbeat.reference.yml
herrBez Apr 13, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 17 additions & 0 deletions metricbeat/docs/modules/aerospike.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -41,8 +41,25 @@ metricbeat.modules:
enabled: true
period: 10s
hosts: ["localhost:3000"]

# Aerospike Cluster Name
#cluster_name: myclustername
shmsr marked this conversation as resolved.
Show resolved Hide resolved

# Optional SSL/TLS (disabled by default)
#ssl.enabled: true

# List of root certificates for SSL/TLS server verification
#ssl.certificate_authorities: ["/etc/pki/root/ca.crt"]

# Certificate for SSL/TLS client authentication
#ssl.certificate: "/etc/pki/client/cert.crt"

# Client certificate key file
#ssl.key: "/etc/pki/client/cert.key"
----

This module supports TLS connections when using `ssl` config field, as described in <<configuration-ssl>>.

[float]
=== Metricsets

Expand Down
15 changes: 15 additions & 0 deletions metricbeat/metricbeat.reference.yml
Original file line number Diff line number Diff line change
Expand Up @@ -149,6 +149,21 @@ metricbeat.modules:
period: 10s
hosts: ["localhost:3000"]

# Aerospike Cluster Name
#cluster_name: myclustername

# Optional SSL/TLS (disabled by default)
#ssl.enabled: true

# List of root certificates for SSL/TLS server verification
#ssl.certificate_authorities: ["/etc/pki/root/ca.crt"]

# Certificate for SSL/TLS client authentication
#ssl.certificate: "/etc/pki/client/cert.crt"

# Client certificate key file
#ssl.key: "/etc/pki/client/cert.key"

#-------------------------------- Apache Module --------------------------------
- module: apache
metricsets: ["status"]
Expand Down
15 changes: 15 additions & 0 deletions metricbeat/module/aerospike/_meta/config.reference.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,3 +3,18 @@
enabled: true
period: 10s
hosts: ["localhost:3000"]
herrBez marked this conversation as resolved.
Show resolved Hide resolved

# Aerospike Cluster Name
#cluster_name: myclustername

# Optional SSL/TLS (disabled by default)
#ssl.enabled: true

# List of root certificates for SSL/TLS server verification
#ssl.certificate_authorities: ["/etc/pki/root/ca.crt"]

# Certificate for SSL/TLS client authentication
#ssl.certificate: "/etc/pki/client/cert.crt"

# Client certificate key file
#ssl.key: "/etc/pki/client/cert.key"
15 changes: 15 additions & 0 deletions metricbeat/module/aerospike/_meta/config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,3 +3,18 @@
# - namespace
period: 10s
hosts: ["localhost:3000"]

# Aerospike Cluster Name
#cluster_name: myclustername

# Optional SSL/TLS (disabled by default)
#ssl.enabled: true

# List of root certificates for SSL/TLS server verification
#ssl.certificate_authorities: ["/etc/pki/root/ca.crt"]

# Certificate for SSL/TLS client authentication
#ssl.certificate: "/etc/pki/client/cert.crt"

# Client certificate key file
#ssl.key: "/etc/pki/client/cert.key"
1 change: 1 addition & 0 deletions metricbeat/module/aerospike/_meta/fields.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
description: >
Aerospike module
release: ga
settings: ["ssl"]
fields:
- name: aerospike
type: group
Expand Down
28 changes: 28 additions & 0 deletions metricbeat/module/aerospike/aerospike.go
Original file line number Diff line number Diff line change
Expand Up @@ -22,9 +22,37 @@ import (
"strconv"
"strings"

"github.com/elastic/elastic-agent-libs/transport/tlscommon"

as "github.com/aerospike/aerospike-client-go"
)

type Config struct {
ClusterName string `config:"cluster_name"`
TLS *tlscommon.Config `config:"ssl"`
}

// DefaultConfig return default config for the aerospike module.
func DefaultConfig() Config {
return Config{}
}

func ParseClientPolicy(config Config) (*as.ClientPolicy, error) {
clientPolicy := as.NewClientPolicy()
if config.TLS.IsEnabled() {
tlsconfig, err := tlscommon.LoadTLSConfig(config.TLS)
if err != nil {
return nil, fmt.Errorf("could not initialize TLS configurations %w", err)
}
clientPolicy.TlsConfig = tlsconfig.ToConfig()
}

if config.ClusterName != "" {
clientPolicy.ClusterName = config.ClusterName
}
return clientPolicy, nil
}

func ParseHost(host string) (*as.Host, error) {
pieces := strings.Split(host, ":")
if len(pieces) != 2 {
Expand Down
67 changes: 67 additions & 0 deletions metricbeat/module/aerospike/aerospike_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,8 @@ import (

"github.com/stretchr/testify/assert"

"github.com/elastic/elastic-agent-libs/transport/tlscommon"

as "github.com/aerospike/aerospike-client-go"
)

Expand Down Expand Up @@ -96,3 +98,68 @@ func TestParseInfo(t *testing.T) {
assert.Equal(t, test.expected, result, test.Name)
}
}

func pointer[T any](d T) *T {
return &d
}

func TestParseClientPolicy(t *testing.T) {
sampleClusterName := "TestCluster"

TLSPolicy := as.NewClientPolicy()
tlsconfig, _ := tlscommon.LoadTLSConfig(&tlscommon.Config{Enabled: pointer(true)})
TLSPolicy.TlsConfig = tlsconfig.ToConfig()

ClusterNamePolicy := as.NewClientPolicy()
ClusterNamePolicy.ClusterName = sampleClusterName

tests := []struct {
Name string
Config Config
expectedClientPolicy *as.ClientPolicy
expectedErr error
}{
{
Name: "Empty configuration leads to default policy",
Config: Config{},
expectedClientPolicy: as.NewClientPolicy(),
expectedErr: nil,
},
{
Name: "TLS Declaration",
Config: Config{
TLS: &tlscommon.Config{
Enabled: pointer(true),
},
},
expectedClientPolicy: TLSPolicy,
expectedErr: nil,
},
{
Name: "Cluster Name Setting",
Config: Config{
ClusterName: sampleClusterName,
},
expectedClientPolicy: ClusterNamePolicy,
expectedErr: nil,
},
}

for _, test := range tests {
result, err := ParseClientPolicy(test.Config)
if err != nil {
if test.expectedErr != nil {
assert.Equalf(t, test.expectedErr.Error(), err.Error(),
"Aerospike policy the error produced is not the one expected: got '%s', expected '%s'", err.Error(), test.expectedErr.Error())
continue
}
t.Error(err)
continue
}
assert.Equalf(t, test.expectedClientPolicy.ClusterName, result.ClusterName,
"Aerospike policy cluster name is wrong. Got '%s' expected '%s'", result.ClusterName, test.expectedClientPolicy.ClusterName)
if test.Config.TLS.IsEnabled() {
assert.NotNil(t, result.TlsConfig, "Aerospike policy: TLS is not set even though TLS is specified in the configuration")
}
}
}
2 changes: 1 addition & 1 deletion metricbeat/module/aerospike/fields.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

15 changes: 11 additions & 4 deletions metricbeat/module/aerospike/namespace/namespace.go
Original file line number Diff line number Diff line change
Expand Up @@ -42,15 +42,16 @@ func init() {
// multiple fetch calls.
type MetricSet struct {
mb.BaseMetricSet
host *as.Host
client *as.Client
host *as.Host
clientPolicy *as.ClientPolicy
client *as.Client
}

// New create a new instance of the MetricSet
// Part of new is also setting up the configuration by processing additional
// configuration entries if needed.
func New(base mb.BaseMetricSet) (mb.MetricSet, error) {
config := struct{}{}
config := aerospike.DefaultConfig()
if err := base.Module().UnpackConfig(&config); err != nil {
return nil, err
}
Expand All @@ -60,9 +61,15 @@ func New(base mb.BaseMetricSet) (mb.MetricSet, error) {
return nil, fmt.Errorf("Invalid host format, expected hostname:port: %w", err)
}

clientPolicy, err := aerospike.ParseClientPolicy(config)
if err != nil {
return nil, fmt.Errorf("could not initialize aerospike client policy: %w", err)
}

return &MetricSet{
BaseMetricSet: base,
host: host,
clientPolicy: clientPolicy,
}, nil
}

Expand Down Expand Up @@ -105,7 +112,7 @@ func (m *MetricSet) Fetch(reporter mb.ReporterV2) error {
// create an aerospike client if it doesn't exist yet
func (m *MetricSet) connect() error {
gpop63 marked this conversation as resolved.
Show resolved Hide resolved
if m.client == nil {
client, err := as.NewClientWithPolicyAndHost(as.NewClientPolicy(), m.host)
client, err := as.NewClientWithPolicyAndHost(m.clientPolicy, m.host)
if err != nil {
return err
}
Expand Down
15 changes: 15 additions & 0 deletions metricbeat/modules.d/aerospike.yml.disabled
herrBez marked this conversation as resolved.
Show resolved Hide resolved
Original file line number Diff line number Diff line change
Expand Up @@ -6,3 +6,18 @@
# - namespace
period: 10s
hosts: ["localhost:3000"]

# Aerospike Cluster Name
#cluster_name: myclustername

# Optional SSL/TLS (disabled by default)
#ssl.enabled: true

# List of root certificates for SSL/TLS server verification
#ssl.certificate_authorities: ["/etc/pki/root/ca.crt"]

# Certificate for SSL/TLS client authentication
#ssl.certificate: "/etc/pki/client/cert.crt"

# Client certificate key file
#ssl.key: "/etc/pki/client/cert.key"
15 changes: 15 additions & 0 deletions x-pack/metricbeat/metricbeat.reference.yml
Original file line number Diff line number Diff line change
Expand Up @@ -158,6 +158,21 @@ metricbeat.modules:
period: 10s
hosts: ["localhost:3000"]

# Aerospike Cluster Name
#cluster_name: myclustername

# Optional SSL/TLS. By default is off.
herrBez marked this conversation as resolved.
Show resolved Hide resolved
#ssl.enabled: true

# List of root certificates for SSL/TLS server verification
#ssl.certificate_authorities: ["/etc/pki/root/ca.crt"]

# Certificate for SSL/TLS client authentication
#ssl.certificate: "/etc/pki/client/cert.crt"

# Client certificate key file
#ssl.key: "/etc/pki/client/cert.key"

#------------------------------- Airflow Module -------------------------------
- module: airflow
host: "localhost"
Expand Down
Loading