Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Relay][AutoTVM] Relay op strategy #4644

Merged
merged 48 commits into from
Feb 24, 2020
Merged

Conversation

icemelon
Copy link
Member

@icemelon icemelon commented Jan 7, 2020

RFC & discussion: https://discuss.tvm.ai/t/relay-op-strategy/5247/18
Update AutoTVM log format: https://discuss.tvm.ai/t/autotvm-log-format/5630

Todo list

  • Add op strategy for conv2d & conv2d related ops (e.g., conv2d_nchwc, conv2d_winograd)
  • Add & update op strategy for conv2d_transpose, dense, bitpack, bitserial_conv2d, bitserial_dense
  • Clean up autotvm (after updating all old autotvm templates)
  • Rewrite TOPI test
  • Update AutoTVM related tutorials

Thanks @kevinthesun @comaniac for the help to this PR!

@MarisaKirisame
Copy link
Contributor

When is conv2d gonna be here? Need it for training.

@icemelon
Copy link
Member Author

x86 conv2d is already in the pr. working on cuda and more targets now

@icemelon icemelon force-pushed the op-dispatch branch 2 times, most recently from a90ee23 to 02cd271 Compare January 27, 2020 19:49
@icemelon icemelon marked this pull request as ready for review January 29, 2020 20:20
@icemelon
Copy link
Member Author

icemelon commented Jan 29, 2020

I've added the strategy for all ops. We can start to review this PR since it's huge. Could you help review it?
@tqchen @kevinthesun @comaniac @masahi @MarisaKirisame @hlu1 @yzhliu @zhiics @ZihengJiang @merrymercy @vinx13 @FrozenGene @jroesch @jwfromm

@icemelon icemelon changed the title [Draft] Relay op strategy [WIP] Relay op strategy Jan 29, 2020
Copy link
Contributor

@comaniac comaniac left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Leave some comments related to AutoTVM changes.

python/tvm/relay/backend/compile_engine.py Show resolved Hide resolved
python/tvm/build_module.py Outdated Show resolved Hide resolved
python/tvm/relay/backend/compile_engine.py Outdated Show resolved Hide resolved
python/tvm/relay/backend/compile_engine.py Outdated Show resolved Hide resolved
self._counter += 1
self.update(target, workload, cfg)
self.update(target, wkl, cfg)
cfg.workload = wkl
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where is cfg.workload initialized? I didn't find a definition in ConfigSpace. Also what's the purpose to have a workload field in a config space?

Copy link
Member Author

@icemelon icemelon Jan 30, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is only specific to ApplyGraphBest. The reason is complicated. Because ApplyGraphBest relies on the order of query, we cannot use relay.backend.compile_engine.select_implement to collect the autotvm workload as it may query more than once. Therefore, this is a temporary work around that we sneak in the workload in the return cfg. We can remove this part of logic after we make ApplyGraphBest no longer relies on the query order.
@kevinthesun

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see. Could we define self.workload in ConfigSpace and add your comment on it? So that we will remember to remove it in the future.

python/tvm/autotvm/task/task.py Outdated Show resolved Hide resolved
python/tvm/relay/quantize/_annotate.py Outdated Show resolved Hide resolved
kernel_L = s.cache_read(kernel, "local", [conv_L])

if temp.name == "pad_temp":
data = temp.op.input_tensors[0]
# TODO(@Laurawly): Do we need to schedule pad op here?
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah we need to schedule temp.

@tqchen
Copy link
Member

tqchen commented Jan 31, 2020

cc @merrymercy @vinx13 @ZihengJiang @jwfromm please help review if you have time

@icemelon icemelon force-pushed the op-dispatch branch 2 times, most recently from 897eb64 to b233005 Compare February 5, 2020 23:06
python/tvm/relay/op/strategy/hls.py Outdated Show resolved Hide resolved
python/tvm/relay/op/strategy/x86.py Outdated Show resolved Hide resolved
include/tvm/te/schedule.h Outdated Show resolved Hide resolved
include/tvm/relay/op_attr_types.h Outdated Show resolved Hide resolved
include/tvm/relay/op_attr_types.h Outdated Show resolved Hide resolved
include/tvm/te/schedule.h Outdated Show resolved Hide resolved
python/tvm/relay/backend/compile_engine.py Outdated Show resolved Hide resolved
python/tvm/schedule.py Outdated Show resolved Hide resolved
@icemelon
Copy link
Member Author

@merrymercy Now the tophub version number has been updated and VTA TSIM test time is also back to normal after the fix. Could you review the PR again?

@merrymercy merrymercy merged commit 623dd20 into apache:master Feb 24, 2020
@icemelon icemelon deleted the op-dispatch branch February 24, 2020 21:14
alexwong pushed a commit to alexwong/tvm that referenced this pull request Feb 26, 2020
* relay op strategy

fix lint

bitpack strategy

bitserial_dense (neo-ai#6)

* update strategy

* address comments

fix a few topi test

Dense strategy (neo-ai#5)

* dense

* add biforst; remove comments

* address comment

Refactor x86 conv2d_NCHWc (neo-ai#4)

* Refactor x86 conv2d

* Add x86 depthwise_conv2d_NCHWc

* Add back topi x86 conv2d_nchw

* Merge x86 conv2d_nchw and conv2d_NCHWc

* Minor fix for x86 conv2d

fix more strategy

Add x86 conv2d_NCHWc_int8 strategy (neo-ai#8)

* Add x86 conv2d_NCHWc_int8 strategy

* Remove contrib_conv2d_nchwc_int8

* Fix generic conv2d_NCHWc for int8

* Fix topi arm_cpu conv2d_NCHWc_int8

update x86 conv2d

enable specify relay ops to be tuned for autotvm

add cuda conv2d strategy

add conv2d strategy for rocm

add conv2d strategy for hls

add conv2d strategy for arm cpu

add conv2d strategy for mali

add conv2d strategy for bifrost

add conv2d strategy for intel graphics

clean up and fix lint

remove template keys from autotvm

remove 2 in the func name

address comments

fix

* fix bugs

* lint

* address comments

* add name to op implement

* Modify topi tests (neo-ai#9)

* Add pooling, reorg, softmax and vision

* Add lrn

* fix topi test

* fix more topi test

* lint

* address comments

* x

* fix more tests & bugs

* Modify more tests (neo-ai#10)

* Modify tests for bitserial_conv2d, bitserial_dense, bitserial_conv2d_rasp and bnn

* Minor fix

* More minor fix

* fix more test

* try to update vta using strategy

* fix cpptest

* x

* fix rebase err

* Fix two tests (neo-ai#11)

* change autotvm log format

* lint

* minor fix

* try fix vta test

* fix rebase err

* tweak

* tmp hack for vta pass

* fix tutorial

* fix

* fix more tutorials

* fix vta tutorial

* minor

* address comments

* fix

* address comments

* fix cpptest

* fix docs

* change data structure name and api

* address comments

* lint

* fix rebase err

* updates

* fix winograd test

* fix doc

* rebase

* upgrade tophub version number

* fix bug

* re-enable vta tsim test after tophub is upgraded

* fix vta test to use the correct args so the config can be found in tophub

Co-authored-by: Yao Wang <[email protected]>
alexwong pushed a commit to alexwong/tvm that referenced this pull request Feb 28, 2020
* relay op strategy

fix lint

bitpack strategy

bitserial_dense (neo-ai#6)

* update strategy

* address comments

fix a few topi test

Dense strategy (neo-ai#5)

* dense

* add biforst; remove comments

* address comment

Refactor x86 conv2d_NCHWc (neo-ai#4)

* Refactor x86 conv2d

* Add x86 depthwise_conv2d_NCHWc

* Add back topi x86 conv2d_nchw

* Merge x86 conv2d_nchw and conv2d_NCHWc

* Minor fix for x86 conv2d

fix more strategy

Add x86 conv2d_NCHWc_int8 strategy (neo-ai#8)

* Add x86 conv2d_NCHWc_int8 strategy

* Remove contrib_conv2d_nchwc_int8

* Fix generic conv2d_NCHWc for int8

* Fix topi arm_cpu conv2d_NCHWc_int8

update x86 conv2d

enable specify relay ops to be tuned for autotvm

add cuda conv2d strategy

add conv2d strategy for rocm

add conv2d strategy for hls

add conv2d strategy for arm cpu

add conv2d strategy for mali

add conv2d strategy for bifrost

add conv2d strategy for intel graphics

clean up and fix lint

remove template keys from autotvm

remove 2 in the func name

address comments

fix

* fix bugs

* lint

* address comments

* add name to op implement

* Modify topi tests (neo-ai#9)

* Add pooling, reorg, softmax and vision

* Add lrn

* fix topi test

* fix more topi test

* lint

* address comments

* x

* fix more tests & bugs

* Modify more tests (neo-ai#10)

* Modify tests for bitserial_conv2d, bitserial_dense, bitserial_conv2d_rasp and bnn

* Minor fix

* More minor fix

* fix more test

* try to update vta using strategy

* fix cpptest

* x

* fix rebase err

* Fix two tests (neo-ai#11)

* change autotvm log format

* lint

* minor fix

* try fix vta test

* fix rebase err

* tweak

* tmp hack for vta pass

* fix tutorial

* fix

* fix more tutorials

* fix vta tutorial

* minor

* address comments

* fix

* address comments

* fix cpptest

* fix docs

* change data structure name and api

* address comments

* lint

* fix rebase err

* updates

* fix winograd test

* fix doc

* rebase

* upgrade tophub version number

* fix bug

* re-enable vta tsim test after tophub is upgraded

* fix vta test to use the correct args so the config can be found in tophub

Co-authored-by: Yao Wang <[email protected]>
zhiics pushed a commit to neo-ai/tvm that referenced this pull request Mar 2, 2020
* relay op strategy

fix lint

bitpack strategy

bitserial_dense (#6)

* update strategy

* address comments

fix a few topi test

Dense strategy (#5)

* dense

* add biforst; remove comments

* address comment

Refactor x86 conv2d_NCHWc (#4)

* Refactor x86 conv2d

* Add x86 depthwise_conv2d_NCHWc

* Add back topi x86 conv2d_nchw

* Merge x86 conv2d_nchw and conv2d_NCHWc

* Minor fix for x86 conv2d

fix more strategy

Add x86 conv2d_NCHWc_int8 strategy (#8)

* Add x86 conv2d_NCHWc_int8 strategy

* Remove contrib_conv2d_nchwc_int8

* Fix generic conv2d_NCHWc for int8

* Fix topi arm_cpu conv2d_NCHWc_int8

update x86 conv2d

enable specify relay ops to be tuned for autotvm

add cuda conv2d strategy

add conv2d strategy for rocm

add conv2d strategy for hls

add conv2d strategy for arm cpu

add conv2d strategy for mali

add conv2d strategy for bifrost

add conv2d strategy for intel graphics

clean up and fix lint

remove template keys from autotvm

remove 2 in the func name

address comments

fix

* fix bugs

* lint

* address comments

* add name to op implement

* Modify topi tests (#9)

* Add pooling, reorg, softmax and vision

* Add lrn

* fix topi test

* fix more topi test

* lint

* address comments

* x

* fix more tests & bugs

* Modify more tests (#10)

* Modify tests for bitserial_conv2d, bitserial_dense, bitserial_conv2d_rasp and bnn

* Minor fix

* More minor fix

* fix more test

* try to update vta using strategy

* fix cpptest

* x

* fix rebase err

* Fix two tests (#11)

* change autotvm log format

* lint

* minor fix

* try fix vta test

* fix rebase err

* tweak

* tmp hack for vta pass

* fix tutorial

* fix

* fix more tutorials

* fix vta tutorial

* minor

* address comments

* fix

* address comments

* fix cpptest

* fix docs

* change data structure name and api

* address comments

* lint

* fix rebase err

* updates

* fix winograd test

* fix doc

* rebase

* upgrade tophub version number

* fix bug

* re-enable vta tsim test after tophub is upgraded

* fix vta test to use the correct args so the config can be found in tophub

Co-authored-by: Yao Wang <[email protected]>
tqchen pushed a commit to tqchen/tvm that referenced this pull request Mar 29, 2020
* relay op strategy

fix lint

bitpack strategy

bitserial_dense (apache#6)

* update strategy

* address comments

fix a few topi test

Dense strategy (apache#5)

* dense

* add biforst; remove comments

* address comment

Refactor x86 conv2d_NCHWc (#4)

* Refactor x86 conv2d

* Add x86 depthwise_conv2d_NCHWc

* Add back topi x86 conv2d_nchw

* Merge x86 conv2d_nchw and conv2d_NCHWc

* Minor fix for x86 conv2d

fix more strategy

Add x86 conv2d_NCHWc_int8 strategy (apache#8)

* Add x86 conv2d_NCHWc_int8 strategy

* Remove contrib_conv2d_nchwc_int8

* Fix generic conv2d_NCHWc for int8

* Fix topi arm_cpu conv2d_NCHWc_int8

update x86 conv2d

enable specify relay ops to be tuned for autotvm

add cuda conv2d strategy

add conv2d strategy for rocm

add conv2d strategy for hls

add conv2d strategy for arm cpu

add conv2d strategy for mali

add conv2d strategy for bifrost

add conv2d strategy for intel graphics

clean up and fix lint

remove template keys from autotvm

remove 2 in the func name

address comments

fix

* fix bugs

* lint

* address comments

* add name to op implement

* Modify topi tests (apache#9)

* Add pooling, reorg, softmax and vision

* Add lrn

* fix topi test

* fix more topi test

* lint

* address comments

* x

* fix more tests & bugs

* Modify more tests (apache#10)

* Modify tests for bitserial_conv2d, bitserial_dense, bitserial_conv2d_rasp and bnn

* Minor fix

* More minor fix

* fix more test

* try to update vta using strategy

* fix cpptest

* x

* fix rebase err

* Fix two tests (apache#11)

* change autotvm log format

* lint

* minor fix

* try fix vta test

* fix rebase err

* tweak

* tmp hack for vta pass

* fix tutorial

* fix

* fix more tutorials

* fix vta tutorial

* minor

* address comments

* fix

* address comments

* fix cpptest

* fix docs

* change data structure name and api

* address comments

* lint

* fix rebase err

* updates

* fix winograd test

* fix doc

* rebase

* upgrade tophub version number

* fix bug

* re-enable vta tsim test after tophub is upgraded

* fix vta test to use the correct args so the config can be found in tophub

Co-authored-by: Yao Wang <[email protected]>
@apivovarov
Copy link
Contributor

There are couple TODOs related to PR4644 - TODO enforce 4-way padding in topi/nn/conv2d after #4644 merged

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.