Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make k8sattributesprocessor available in Agent running in OTel mode #4893

Merged

Conversation

ChrsMark
Copy link
Member

@ChrsMark ChrsMark commented Jun 10, 2024

What does this PR do?

This PR makes the k8sattributesprocessor available to users running Elastic Agent in otel mode.

Why is it important?

To allow users of Elastic Agent running as as OpenTelemetry Collector to configure pipelines containing the k8sattributesprocessor.

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works
  • I have added an entry in ./changelog/fragments using the changelog tool
  • I have added an integration test or an E2E test

How to test this PR locally

  1. Prepare the binaryDEV=true EXTERNAL=true SNAPSHOT=true PLATFORMS=linux/amd64 PACKAGES=docker mage -v package
  2. build the image cd build/package/elastic-agent/elastic-agent-linux-amd64.docker/docker-build && docker build -t agent-otel-dev .
  3. Load to local cluster kind load docker-image agent-otel-dev:latest
  4. prepare the ES secrets
kubectl create secret generic elastic-secret \
     --from-literal=es_endpoint='https://test-otel-blablabla.es.us-central1.gcp.cloud.es.io/' \                 
     --from-literal=es_password='asdfgp' \     
     --from-literal=es_user='elastic'
  1. specify the image:
image: agent-otel-dev:latest
imagePullPolicy: Never
  1. Deploy the manifest from https:/elastic/opentelemetry-dev/pull/258 : k apply -f elastic-otel-collector.yaml
  2. Verify the output:
2024-06-10T13:17:25.677Z	info	ResourceLog #0
Resource SchemaURL: 
Resource attributes:
     -> k8s.namespace.name: Str(kube-system)
     -> k8s.pod.name: Str(kindnet-6lgrl)
     -> k8s.container.restart_count: Str(0)
     -> k8s.pod.uid: Str(3844b30e-67ee-4bcc-98ac-8f2196852939)
     -> k8s.container.name: Str(kindnet-cni)
     -> k8s.daemonset.name: Str(kindnet)
     -> k8s.node.name: Str(kind-control-plane)
     -> k8s.pod.start_time: Str(2024-06-10T13:09:03Z)
ScopeLogs #0
ScopeLogs SchemaURL: 
InstrumentationScope  
LogRecord #0
ObservedTimestamp: 2024-06-10 13:17:25.420729149 +0000 UTC
Timestamp: 2024-06-10 13:17:25.300601954 +0000 UTC
SeverityText: 
SeverityNumber: Unspecified(0)
Body: Str(I0610 13:17:25.300466       1 main.go:223] Handling node with IPs: map[172.21.0.2:{}])
Attributes:
     -> logtag: Str(F)
     -> log.file.path: Str(/var/log/pods/kube-system_kindnet-6lgrl_3844b30e-67ee-4bcc-98ac-8f2196852939/kindnet-cni/0.log)
     -> log.iostream: Str(stderr)
     -> time: Str(2024-06-10T13:17:25.300601954Z)
Trace ID: 
Span ID: 
Flags: 0
LogRecord #1
ObservedTimestamp: 2024-06-10 13:17:25.420987445 +0000 UTC
Timestamp: 2024-06-10 13:17:25.300633177 +0000 UTC
SeverityText: 
SeverityNumber: Unspecified(0)
Body: Str(I0610 13:17:25.300494       1 main.go:227] handling current node)
Attributes:
     -> log.iostream: Str(stderr)
     -> logtag: Str(F)
     -> log.file.path: Str(/var/log/pods/kube-system_kindnet-6lgrl_3844b30e-67ee-4bcc-98ac-8f2196852939/kindnet-cni/0.log)
     -> time: Str(2024-06-10T13:17:25.300633177Z)
Trace ID: 
Span ID: 
Flags: 0
	{"kind": "exporter", "data_type": "logs", "name": "debug"}

Copy link
Contributor

mergify bot commented Jun 10, 2024

This pull request does not have a backport label. Could you fix it @ChrsMark? 🙏
To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

  • backport-v./d./d./d is the label to automatically backport to the 8./d branch. /d is the digit

NOTE: backport-skip has been added to this pull request.

@ChrsMark ChrsMark force-pushed the add_filestorage_extention_otel_component branch from fa14718 to f6dc899 Compare June 10, 2024 12:44
@ChrsMark ChrsMark marked this pull request as ready for review June 10, 2024 13:42
@ChrsMark ChrsMark requested a review from a team as a code owner June 10, 2024 13:42
@pierrehilbert pierrehilbert added the Team:Elastic-Agent-Control-Plane Label for the Agent Control Plane team label Jun 10, 2024
@elasticmachine
Copy link
Contributor

Pinging @elastic/elastic-agent-control-plane (Team:Elastic-Agent-Control-Plane)

@ChrsMark ChrsMark force-pushed the add_filestorage_extention_otel_component branch from f6dc899 to 920a942 Compare June 10, 2024 14:49
Copy link
Contributor

@michalpristas michalpristas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@michalpristas michalpristas merged commit 2e09c33 into elastic:main Jun 11, 2024
14 checks passed
@michalpristas michalpristas added backport-v8.14.0 Automated backport with mergify and removed backport-skip labels Jun 21, 2024
mergify bot pushed a commit that referenced this pull request Jun 21, 2024
Signed-off-by: ChrsMark <[email protected]>
Co-authored-by: Michal Pristas <[email protected]>
(cherry picked from commit 2e09c33)

# Conflicts:
#	go.mod
#	go.sum
#	internal/pkg/otel/README.md
#	internal/pkg/otel/components.go
michalpristas pushed a commit that referenced this pull request Jun 21, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport-v8.14.0 Automated backport with mergify Team:Elastic-Agent-Control-Plane Label for the Agent Control Plane team
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants