Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reclaim reserved buffer for unused ports #831

Merged

Conversation

stephenxs
Copy link
Collaborator

@stephenxs stephenxs commented Aug 4, 2021

201911 PRs

PR title state context
Upgrade Mellanox-SAI to 1.19.3 to support reclaiming reserved buffer on admin down ports GitHub issue/pull request detail GitHub pull request check contexts
[Buffer Manager] Reclaim unused buffer for admin-down ports GitHub issue/pull request detail GitHub pull request check contexts
[Buffer Orch] Handle DEL_COMMAND for BUFFER_PG and BUFFER_QUEUE table GitHub issue/pull request detail GitHub pull request check contexts
Regression test case for traditional buffer manager in 201911 GitHub issue/pull request detail GitHub pull request check contexts
[201911] [Submodule] Update sonic-swss GitHub issue/pull request detail GitHub pull request check contexts
Verify buffer priority groups for all platforms GitHub issue/pull request detail GitHub pull request check contexts
[201911] Verify buffer priority groups for all platforms GitHub issue/pull request detail GitHub pull request check contexts

Master PRs (currently draft)

PR title state context
[Reclaim buffer] Reclaim unused buffers by applying zero buffer profiles GitHub issue/pull request detail GitHub pull request check contexts
[Reclaim buffer] Common infrastructure update for reclaiming buffer GitHub issue/pull request detail GitHub pull request check contexts
[Reclaim buffer] Reclaim unused buffer for dynamic buffer model GitHub issue/pull request detail GitHub pull request check contexts
[Reclaim buffer] Reclaim unused buffer for traditional buffer model GitHub issue/pull request detail GitHub pull request check contexts
[Reclaim buffer] Common code update for swss GitHub issue/pull request detail GitHub pull request check contexts
[Reclaim buffer] [Mellanox] Db migrator support reclaiming reserved buffer for unused ports GitHub issue/pull request detail GitHub pull request check contexts
[Reclaiming buffer] Test cases for reclaiming buffer on both traditional and dynamic model GitHub issue/pull request detail GitHub pull request check contexts

202106 PRs

PR title state context
[Reclaim buffer][202106] Reclaim unused buffers by applying zero buffer profiles GitHub issue/pull request detail GitHub pull request check contexts
[Reclaim buffer][202106] Reclaim unused buffer for dynamic buffer model GitHub issue/pull request detail GitHub pull request check contexts
[Reclaim buffer][202106] Reclaim unused buffer in traditional buffer model GitHub issue/pull request detail GitHub pull request check contexts
[Reclaim buffer][202106] Common code update GitHub issue/pull request detail GitHub pull request check contexts
[Reclaim buffer][202106] Db migrator support reclaiming reserved buffer for unused ports GitHub issue/pull request detail GitHub pull request check contexts

202012 PRs

PR title state context
[Reclaim buffer][202012] Reclaim unused buffers by applying zero buffer profiles GitHub issue/pull request detail GitHub pull request check contexts
[Reclaim buffer][202012] Reclaim unused buffer for both traditional and dynamic buffer model GitHub issue/pull request detail GitHub pull request check contexts
[Reclaim buffer][202012] Reclaim unused buffer in traditional buffer model GitHub issue/pull request detail GitHub pull request check contexts
[Reclaim buffer][202012] Common code update GitHub issue/pull request detail GitHub pull request check contexts
[Reclaim buffer][202012] Db migrator support reclaiming reserved buffer for unused ports GitHub issue/pull request detail GitHub pull request check contexts

Reclaim reserved buffer for unused ports.
Originally, the reserved buffer is reclaimed by removing buffer objects of the unused ports. However, this introduces inconsistency. Consider the following scenarios:

  1. System starts without any buffer configuration. The buffer configuration in the ASIC is the SDK default value whose reserved size is not 0. This is to guarantee the traffic can be forwarded without buffer configuration in SONiC.
  2. After the system started and the buffer was configured, some buffer objects are removed from ports. There is still no buffer configuration in SONiC. However, in the ASIC the reserved size of some of the buffer objects is 0.

Now we have two scenarios with the same SONiC buffer configuration, resulting in different buffer configurations in ASIC.

To resolve this issue:

  • zero buffer profiles are introduced to indicate 0 reserved size of a buffer object.
  • Removing a buffer object indicates setting the buffer object to SDK default value.

The solution:

  • For lossy PG,
    • SONiC to create
      • a buffer pool with static threshold mode and zero size
      • a zero buffer profile with static threshold == 0, size == 0, referencing the pool created in above step
    • SONiC to configure the zero buffer profile on the unused port
    • SAI to set the reserved buffer size to 0 if a zero buffer profile is configured
  • For lossless PG, to remove the PG from SAI
  • For port buffer pool and queues,
    • SONiC to create a zero buffer profile with size == 0 and dynamic mode
    • SAI to set the reserved buffer size to 0 (no change on SAI)

The zero profiles and pools will be handled in different ways between the dynamic and traditional buffer models.

  • Traditional buffer model:
  • The zero profiles and pools will be configured into CONFIG_DB if there is at least one inactive port.
  • The zero profiles will be configured to corresponding buffer items of inactive ports, like priority groups, queues, ingress/egress profile lists.
  • Dynamic buffer model:
    • The normal profiles will always be configured to corresponding buffer items of all ports, including active ones and inactive ones.
    • The buffer manager will insert the zero profiles and pools into APPL_DB when the first port becomes admin down
    • The buffer manager will apply the zero profiles to corresponding buffer items of inactive ports into APPL_DB

@stephenxs stephenxs force-pushed the reclaim-reserved-buffer-github branch from 78da8fe to 6c89920 Compare August 4, 2021 08:22
@zhangyanzhao
Copy link
Collaborator

@neethajohn would you please help to approve this HLD PR? Thanks.

@stephenxs
Copy link
Collaborator Author

stephenxs commented Nov 10, 2021

Suggest merging order:

  1. Swss common PR ([Reclaiming buffer] Common code update sonic-swss#1996) and then submodule update PR which will be opened once the swss PR has been merged
  2. Build image common PR (#9133)
  3. Swss dynamic mode PR ([Reclaim buffer] Reclaim unused buffer for dynamic buffer model sonic-swss#1910)
  4. Db migrator PR ([Reclaim buffer] [Mellanox] Db migrator support reclaiming reserved buffer for unused ports sonic-utilities#1822)
  5. Submodule update PR which will be opened after swss/utilities PRs have been merged.
  6. Build image Mellanox specific PR ([Reclaim buffer] Reclaim unused buffers by applying zero buffer profiles sonic-buildimage#8768)

The swss static mode PR (sonic-net/sonic-swss#2011) doesn't depend on any PR and can be merged at any time.

Db migrator PR (sonic-net/sonic-utilities#1822) and build image Mellanox specific PR (sonic-net/sonic-buildimage#8768) can also be merged before swss dynamic mode PR (sonic-net/sonic-swss#1910).

@liat-grozovik
Copy link
Collaborator

@neethajohn could you please help to review?
Code changes are already in place and are in review. Better to have the HLD approved before all code is in

@liat-grozovik
Copy link
Collaborator

@neethajohn kindly reminder, this PR needs to get it as most of the code is already merged.

@liat-grozovik liat-grozovik merged commit bc9b0a4 into sonic-net:master Dec 6, 2021
@stephenxs stephenxs deleted the reclaim-reserved-buffer-github branch December 6, 2021 22:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants