Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[LLM INFER] Optimize fuse some kernels in postprocess #9201

Open
wants to merge 7 commits into
base: develop
Choose a base branch
from

Conversation

gzy19990617
Copy link
Contributor

PR types

Performance optimization

PR changes

Others

Description

1.get_padding_offset与remove_padding kernel fuse
2.stop_generation_multi_ends_v2与update_inputs kernel与前面的一些操作进行fuse
3.set_value_by_flags_and_idx_v2与set_stop_value_multi_ends_v2 kernel fuse

均增加测试代码,算子级别已对齐精度

Copy link

codecov bot commented Sep 26, 2024

Codecov Report

Attention: Patch coverage is 0% with 5 lines in your changes missing coverage. Please review.

Project coverage is 53.19%. Comparing base (8b2cc08) to head (7ed0de5).
Report is 10 commits behind head on develop.

Files with missing lines Patch % Lines
...enlp/experimental/transformers/generation_utils.py 0.00% 5 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff             @@
##           develop    #9201      +/-   ##
===========================================
+ Coverage    53.16%   53.19%   +0.02%     
===========================================
  Files          657      657              
  Lines       106311   106699     +388     
===========================================
+ Hits         56524    56762     +238     
- Misses       49787    49937     +150     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@gzy19990617 gzy19990617 changed the title 【Inference】Optimize fuse some kernels [LLM INFER] Optimize fuse some kernels in postprocess Oct 9, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant