Faster sparse_dense on GPUs #6580
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This new sparse_dense requires a padded matrix, so a new op `sparse_dense_padded` has been added. AlterOpLayout should transform `sparse_dense` to `sparse_dense_padded` when possible on the gpu.
tkonolige
force-pushed
the
faster_sparse_dense
branch
from
74cf118
to
b563b1c
Compare
@tkonolige : Thanks for the PR! The data looks quite impressive 👍 |
|
@ANSHUMAN87 Right now TVM does not do any testing for performance regressions. The hard part in setting up performance testing is that is varies from run to run and machine to machine. |
|
@tkonolige : I understand your concern clearly. However it was just a thought. Even if run to run or machine to machine difference, the relative reference would be same. But may be we don't have to do as part of this PR :) |
|
Have you considered your syntax errors? |
|
@vinx13 @antinucleon @Laurawly @jwfromm @ajtulloch I think this is ready for review. |
vinx13
reviewed
Oct 1, 2020
Laurawly
reviewed
Oct 2, 2020
ANSHUMAN87
reviewed
Oct 4, 2020
ANSHUMAN87
reviewed
Oct 4, 2020
ANSHUMAN87
reviewed
Oct 8, 2020
ANSHUMAN87
reviewed
Oct 8, 2020
ANSHUMAN87
reviewed
Oct 8, 2020
jroesch
approved these changes
Oct 9, 2020
|
@tkonolige @tqchen This commits fails in the master branch. see the CI: https:/apache/incubator-tvm/commits/master It introduces a flaky test that blocks two of my PRs. |
#6658 has resolved the issue i think. |
|
@merrymercy I think that was the diagnostics |
TusharKanekiDey
pushed a commit
to TusharKanekiDey/tvm
that referenced
this pull request
Oct 13, 2020
* Faster sparse_dense on GPUs. This new sparse_dense requires a padded matrix, so a new op `sparse_dense_padded` has been added. AlterOpLayout should transform `sparse_dense` to `sparse_dense_padded` when possible on the gpu. * formatting * more formatting * Check that alteroplayout is definedbefore using it * check if FTVMAlterOpLayout exists before using it * formatting * restore message passing * Fix sparse_dense and sparse_dense_padded docs * Fix old sparse_dense, autotvm and sparse_dense dont play well together * Remove unused imports * clarify warp count in cuda_transpose * Document multidimensional access * Warn users not to use sparse_dense_padded * rename nn.sparse_dense_padded to nn.internal.sparse_dense_padded
TusharKanekiDey
pushed a commit
to TusharKanekiDey/tvm
that referenced
this pull request
Oct 14, 2020
* Faster sparse_dense on GPUs. This new sparse_dense requires a padded matrix, so a new op `sparse_dense_padded` has been added. AlterOpLayout should transform `sparse_dense` to `sparse_dense_padded` when possible on the gpu. * formatting * more formatting * Check that alteroplayout is definedbefore using it * check if FTVMAlterOpLayout exists before using it * formatting * restore message passing * Fix sparse_dense and sparse_dense_padded docs * Fix old sparse_dense, autotvm and sparse_dense dont play well together * Remove unused imports * clarify warp count in cuda_transpose * Document multidimensional access * Warn users not to use sparse_dense_padded * rename nn.sparse_dense_padded to nn.internal.sparse_dense_padded
TusharKanekiDey
pushed a commit
to TusharKanekiDey/tvm
that referenced
this pull request
Oct 15, 2020
* Faster sparse_dense on GPUs. This new sparse_dense requires a padded matrix, so a new op `sparse_dense_padded` has been added. AlterOpLayout should transform `sparse_dense` to `sparse_dense_padded` when possible on the gpu. * formatting * more formatting * Check that alteroplayout is definedbefore using it * check if FTVMAlterOpLayout exists before using it * formatting * restore message passing * Fix sparse_dense and sparse_dense_padded docs * Fix old sparse_dense, autotvm and sparse_dense dont play well together * Remove unused imports * clarify warp count in cuda_transpose * Document multidimensional access * Warn users not to use sparse_dense_padded * rename nn.sparse_dense_padded to nn.internal.sparse_dense_padded
TusharKanekiDey
pushed a commit
to TusharKanekiDey/tvm
that referenced
this pull request
Oct 15, 2020
* Faster sparse_dense on GPUs. This new sparse_dense requires a padded matrix, so a new op `sparse_dense_padded` has been added. AlterOpLayout should transform `sparse_dense` to `sparse_dense_padded` when possible on the gpu. * formatting * more formatting * Check that alteroplayout is definedbefore using it * check if FTVMAlterOpLayout exists before using it * formatting * restore message passing * Fix sparse_dense and sparse_dense_padded docs * Fix old sparse_dense, autotvm and sparse_dense dont play well together * Remove unused imports * clarify warp count in cuda_transpose * Document multidimensional access * Warn users not to use sparse_dense_padded * rename nn.sparse_dense_padded to nn.internal.sparse_dense_padded
TusharKanekiDey
pushed a commit
to TusharKanekiDey/tvm
that referenced
this pull request
Oct 16, 2020
* Faster sparse_dense on GPUs. This new sparse_dense requires a padded matrix, so a new op `sparse_dense_padded` has been added. AlterOpLayout should transform `sparse_dense` to `sparse_dense_padded` when possible on the gpu. * formatting * more formatting * Check that alteroplayout is definedbefore using it * check if FTVMAlterOpLayout exists before using it * formatting * restore message passing * Fix sparse_dense and sparse_dense_padded docs * Fix old sparse_dense, autotvm and sparse_dense dont play well together * Remove unused imports * clarify warp count in cuda_transpose * Document multidimensional access * Warn users not to use sparse_dense_padded * rename nn.sparse_dense_padded to nn.internal.sparse_dense_padded
trevor-m
pushed a commit
to neo-ai/tvm
that referenced
this pull request
Oct 19, 2020
* Faster sparse_dense on GPUs. This new sparse_dense requires a padded matrix, so a new op `sparse_dense_padded` has been added. AlterOpLayout should transform `sparse_dense` to `sparse_dense_padded` when possible on the gpu. * formatting * more formatting * Check that alteroplayout is definedbefore using it * check if FTVMAlterOpLayout exists before using it * formatting * restore message passing * Fix sparse_dense and sparse_dense_padded docs * Fix old sparse_dense, autotvm and sparse_dense dont play well together * Remove unused imports * clarify warp count in cuda_transpose * Document multidimensional access * Warn users not to use sparse_dense_padded * rename nn.sparse_dense_padded to nn.internal.sparse_dense_padded
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
I've written a faster sparse_dense for GPUs using tir. This sparse_dense requires a padded matrix, so I've added a new op sparse_dense_padded. AlterOpLayout should transform sparse_dense to sparse_dense_padded when using a gpu.
This new sparse_dense improves prunebert performance from 155.41ms mean to 7.75ms mean. In general, this implementation is faster than cublas dense on matrices with density < 0.05 and is often faster than cusparse for machine learning workloads.