You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is your feature request related to a problem? Please describe.
The old kinesis plugin still has some features like gzip compression and KPL aggregation that the kinesis_streams plugin written in C does not have.
This makes migrating from the 3rd party kinesis plugin to the kinesis_streams plugin somewhat difficult.
Describe the solution you'd like
The kinesis_streams plugin should support aggregation of records as well as compression (at least gzip).
Describe alternatives you've considered
Sticking with the kinesis go plugin until it ultimately becomes unmaintained (it likely already is, no new commits or releases since 18 months and they point to the upstream kinesis_streams plugin as the alternative).
Additional context
We are using fluent-bit to collect application logs. We use both KPL aggregation and gzip in our log consumption pipeline to improve log throughput. In our current solution we had to monkey-patch the official fluent-bit container image to include the kinesis go plugin. Ideally the KPL and aggregation features should be supported natively by fluent-bit without any customization of the container image however.
The text was updated successfully, but these errors were encountered:
Is your feature request related to a problem? Please describe.
The old kinesis plugin still has some features like gzip compression and KPL aggregation that the
kinesis_streams
plugin written in C does not have.This makes migrating from the 3rd party
kinesis
plugin to thekinesis_streams
plugin somewhat difficult.Describe the solution you'd like
The
kinesis_streams
plugin should support aggregation of records as well as compression (at least gzip).Describe alternatives you've considered
Sticking with the
kinesis
go plugin until it ultimately becomes unmaintained (it likely already is, no new commits or releases since 18 months and they point to the upstreamkinesis_streams
plugin as the alternative).Additional context
We are using fluent-bit to collect application logs. We use both KPL aggregation and gzip in our log consumption pipeline to improve log throughput. In our current solution we had to monkey-patch the official fluent-bit container image to include the
kinesis
go plugin. Ideally the KPL and aggregation features should be supported natively by fluent-bit without any customization of the container image however.The text was updated successfully, but these errors were encountered: