Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[v24.2.x] Fix dissemination large shard count #22982

Merged
merged 2 commits into from
Aug 22, 2024
Merged

[v24.2.x] Fix dissemination large shard count #22982

merged 2 commits into from
Aug 22, 2024

Commits on Aug 21, 2024

  1. c/m_dissemination: use chunked vector in update_leaders_request 

    The `update_leader_request` may only contain few of ntp leader updates.
In this case using a `fragmented_vector` with large chunk size is a
memory waste

Signed-off-by: Michał Maślanka <[email protected]>
(cherry picked from commit e1be74a)
    @mmaslankaprv @vbotbuildovich
    mmaslankaprv authored and vbotbuildovich committed Aug 21, 2024
    Configuration menu
    Copy the full SHA
    2a70f60 View commit details
    Browse the repository at this point in the history

  2. c/m_dissemination: avoid copying updates vector 

    Previously each time the leadership update request was received by the
`metadata_dissemination_handler` it created a copy for each of the shard
on the shard handling the request. This is inefficient and may lead to
OOM on the handler shard (especially for very large machines).

Instead of creating a copy for each shard we can simply use the const
reference as the updates vector doesn't change and it is safe to access
it from other cores.

Signed-off-by: Michał Maślanka <[email protected]>
(cherry picked from commit 8e4f51f)
    @mmaslankaprv @vbotbuildovich
    mmaslankaprv authored and vbotbuildovich committed Aug 21, 2024
    Configuration menu
    Copy the full SHA
    3863c87 View commit details
    Browse the repository at this point in the history