Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduce Buffer Strategy agent config options #14805

Closed
powersj opened this issue Feb 13, 2024 · 5 comments
Closed

Introduce Buffer Strategy agent config options #14805

powersj opened this issue Feb 13, 2024 · 5 comments
Assignees
Labels
feature request Requests for new plugin and for new features to existing plugins

Comments

@powersj
Copy link
Contributor

powersj commented Feb 13, 2024

Use Case

Introduce a new agent config option which specifies a specific buffer strategy. It would have three options:

  • memory: current behavior, in memory, limited, overflows drop metrics
  • write-through: writes metrics to disk, all to disk in line protocol
  • mem+disk: only writes to disk on overflow, an entire buffer at a time.

The key items to consider are:

  • tracking metrics: when to acknowledge them with the write-through.
  • ensuring we write the oldest to disk first
  • write batch size at a time to disk
  • ensure the rate-limiting PR lands as well

The config would take two parameters:

  • the method as defined above
  • the directory to write metrics to when writing to disk

These additional options could cause the disk to fill up and that is on the user to figure out and avoid.

Expected behavior

Allow the user to determine which strategy to use for the buffer.

Actual behavior

Only use in-memory buffer and when it overflows, metrics are discarded.

Additional info

No response

@powersj powersj added the feature request Requests for new plugin and for new features to existing plugins label Feb 13, 2024
@powersj powersj assigned srebhan and DStrand1 and unassigned srebhan Feb 21, 2024
@powersj
Copy link
Contributor Author

powersj commented Feb 22, 2024

next steps: get draft PR up to start testing. Can do this one option at a time.

@srebhan
Copy link
Member

srebhan commented Feb 23, 2024

@powersj now that we do have the spec thing, should we first get a PR for the spec and then put up the actual PR referencing this spec? Doesn't exclude a draft-PR before the spec though...

@powersj
Copy link
Contributor Author

powersj commented Feb 23, 2024

Yes, I agree, it would be good to at least have a PR for the draft.

@knollet
Copy link
Contributor

knollet commented Mar 6, 2024

We have this overflow problem, too.
+1 for implementation.

@powersj
Copy link
Contributor Author

powersj commented Jul 22, 2024

#15564 has landed and is available in nighties. We will keep the older issue open while we collect feedback as we do consider this experimental.

@powersj powersj closed this as completed Jul 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature request Requests for new plugin and for new features to existing plugins
Projects
None yet
Development

No branches or pull requests

4 participants