Skip to content
This repository has been archived by the owner on Nov 15, 2023. It is now read-only.

Metadata V15: Expose API to fetch metadata for version #13287

Merged
merged 51 commits into from
Mar 15, 2023

Conversation

lexnv
Copy link
Contributor

@lexnv lexnv commented Jan 31, 2023

This PR extends the Metadata API to fetch the metadata at a given provided version.

It lays the foundation for keeping the metadata V14 around and exposing the V15 for early adopters before making a major breaking change.

The runtime API comes with 2 new functions:

  • metadata_at_version - Fetch the metadata at a given version
  • metadata_versions - List the supported versions by metadata_at_version

At the current moment, substrate only supports V14.
Soon, the V15 will be exposed as unstable under the u32::max value.

Users could then fetch the v15 metadata, allowing for an appropriate time to migrate their use cases.

Part of: #12939.

polkadot companion: paritytech/polkadot#6655
cumulus companion: paritytech/cumulus#2155

CC: @paritytech/tools-team

@lexnv lexnv self-assigned this Jan 31, 2023
@github-actions github-actions bot added the A0-please_review Pull request needs code review. label Jan 31, 2023
@lexnv lexnv added B7-runtimenoteworthy C1-low PR touches the given topic and has a low impact on builders. D3-trivial 🧸 PR contains trivial changes in a runtime directory that do not require an audit labels Jan 31, 2023
@lexnv
Copy link
Contributor Author

lexnv commented Feb 2, 2023

bot rebase

@paritytech-processbot
Copy link

Rebased

@lexnv
Copy link
Contributor Author

lexnv commented Feb 7, 2023

bot rebase

@paritytech-processbot
Copy link

Rebased

Comment on lines 741 to 743
/// Returns the supported versions of metadata.
///
/// This can be used to call `metadata_at_version`.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
/// Returns the supported versions of metadata.
///
/// This can be used to call `metadata_at_version`.
/// Returns the available metadata versions.

Nit; Just because "This can be used to call.." felt a bit unclear :)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would keep "supported". Feels more clear on what this is about.

Copy link
Contributor

@jsdw jsdw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not an expert in this area but it looks good to me!

Copy link
Member

@bkchr bkchr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, but we should forward the selection of the supported metadata to the metadata crate. See my comments for more information.

Comment on lines 741 to 743
/// Returns the supported versions of metadata.
///
/// This can be used to call `metadata_at_version`.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would keep "supported". Feels more clear on what this is about.

primitives/api/src/lib.rs Outdated Show resolved Hide resolved
@the-right-joyce the-right-joyce added B1-note_worthy Changes should be noted in the release notes T1-runtime This PR/Issue is related to the topic “runtime”. and removed B7-runtimenoteworthy labels Feb 13, 2023
@jsdw jsdw self-requested a review February 13, 2023 12:30
@lexnv
Copy link
Contributor Author

lexnv commented Mar 15, 2023

bot merge

@paritytech-processbot
Copy link

Error: "Check reviews" status is not passing for paritytech/polkadot#6655

@lexnv
Copy link
Contributor Author

lexnv commented Mar 15, 2023

bot merge

@paritytech-processbot
Copy link

Error: "Check reviews" status is not passing for paritytech/polkadot#6655

@bkchr
Copy link
Member

bkchr commented Mar 15, 2023

Cc @jacogr

@lexnv
Copy link
Contributor Author

lexnv commented Mar 15, 2023

bot merge

@paritytech-processbot paritytech-processbot bot merged commit bd52212 into master Mar 15, 2023
@paritytech-processbot paritytech-processbot bot deleted the lexnv/md15_expose_md_at_versions branch March 15, 2023 17:49
breathx pushed a commit to gear-tech/substrate that referenced this pull request Apr 22, 2023
)

* impl_runtime_apis: Generate getters for `metadata_at` functions

Signed-off-by: Alexandru Vasile <[email protected]>

* runtime: Implement new `Metadata` runtime trait

Signed-off-by: Alexandru Vasile <[email protected]>

* runtime: Move `metadata_at` functions to construct_runtime macro

Signed-off-by: Alexandru Vasile <[email protected]>

* contruct_runtime: Use `OpaqueMetadata` from hidden imports

Signed-off-by: Alexandru Vasile <[email protected]>

* Adjust testing

Signed-off-by: Alexandru Vasile <[email protected]>

* frame/tests: Add tests for the new API

Signed-off-by: Alexandru Vasile <[email protected]>

* frame/tests: Adjust metdata naming

Signed-off-by: Alexandru Vasile <[email protected]>

* frame/support: Expose `metadata-v14` feature flag

Signed-off-by: Alexandru Vasile <[email protected]>

* frame/support: Expose metadata only under feature flags

Signed-off-by: Alexandru Vasile <[email protected]>

* frame/support: Expose v14 metadata by default

Signed-off-by: Alexandru Vasile <[email protected]>

* frame/support: Expose metadata feature for testing

Signed-off-by: Alexandru Vasile <[email protected]>

* frame/support: Test metadata under different feature flags

Signed-off-by: Alexandru Vasile <[email protected]>

* Update primitives/api/src/lib.rs

Co-authored-by: Bastian Köcher <[email protected]>

* Update primitives/api/src/lib.rs

Co-authored-by: Bastian Köcher <[email protected]>

* client/tests: Adjust testing to reflect trait Metadata change

Signed-off-by: Alexandru Vasile <[email protected]>

* frame/metadata-ir: Add intermediate representation types for metadata

Signed-off-by: Alexandru Vasile <[email protected]>

* frame/metadata-ir: Convert metadata to V14

Signed-off-by: Alexandru Vasile <[email protected]>

* frame/metadata-ir: Add API to convert metadata to multiple versions

Signed-off-by: Alexandru Vasile <[email protected]>

* frame/metadata-ir: Expose V14 under feature flag

Signed-off-by: Alexandru Vasile <[email protected]>

* frame/support: Adjust to metadata IR

Signed-off-by: Alexandru Vasile <[email protected]>

* frame/support: More adjustments

Signed-off-by: Alexandru Vasile <[email protected]>

* frame/support: Guard v14 details under feature flag

Signed-off-by: Alexandru Vasile <[email protected]>

* frame/support: Adjust testing

Signed-off-by: Alexandru Vasile <[email protected]>

* CI: Ensure `quick-benchmarks` uses `metadata-v14`

Signed-off-by: Alexandru Vasile <[email protected]>

* frame/support: Use `metadata-v14` for benchmarks

Signed-off-by: Alexandru Vasile <[email protected]>

* Adjust cargo fmt

Signed-off-by: Alexandru Vasile <[email protected]>

* kitchensink-runtime: Add feature flag for `metadata-v14`

Signed-off-by: Alexandru Vasile <[email protected]>

* frame/support/test: Adjust testing

Signed-off-by: Alexandru Vasile <[email protected]>

* frame/support/test: Check crates locally

Signed-off-by: Alexandru Vasile <[email protected]>

* Activate metadata-v14 for pallets

Signed-off-by: Alexandru Vasile <[email protected]>

* Remove metadata-v14 feature flag

Signed-off-by: Alexandru Vasile <[email protected]>

* frame/metadata_ir: Move `api.rs` to `mod.rs`

Signed-off-by: Alexandru Vasile <[email protected]>

* frame/support: Handle latest metadata conversion via IR

Signed-off-by: Alexandru Vasile <[email protected]>

* frame/tests: Add constant for metadata version 14

Signed-off-by: Alexandru Vasile <[email protected]>

* frame/support/test: Fix merge conflict

Signed-off-by: Alexandru Vasile <[email protected]>

* Update frame/support/Cargo.toml

Co-authored-by: Bastian Köcher <[email protected]>

* Update frame/support/src/metadata_ir/mod.rs

Co-authored-by: Bastian Köcher <[email protected]>

* Update frame/support/test/Cargo.toml

Co-authored-by: Bastian Köcher <[email protected]>

* Update primitives/api/src/lib.rs

Co-authored-by: Bastian Köcher <[email protected]>

* frame/metadata: Collect pallet documentation for MetadataIR

Signed-off-by: Alexandru Vasile <[email protected]>

* frame/tests: Check pallet documentation is propagated to MetadataIR

Signed-off-by: Alexandru Vasile <[email protected]>

* frame/support: Improve documentation

Signed-off-by: Alexandru Vasile <[email protected]>

---------

Signed-off-by: Alexandru Vasile <[email protected]>
Co-authored-by: parity-processbot <>
Co-authored-by: Bastian Köcher <[email protected]>
@Polkadot-Forum
Copy link

This pull request has been mentioned on Polkadot Forum. There might be relevant details there:

https://forum.polkadot.network/t/stablising-v15-metadata/2819/1

@lexnv lexnv mentioned this pull request Jun 29, 2023
AurevoirXavier added a commit to darwinia-network/darwinia that referenced this pull request Jul 6, 2023
nathanwhit pushed a commit to nathanwhit/substrate that referenced this pull request Jul 19, 2023
)

* impl_runtime_apis: Generate getters for `metadata_at` functions

Signed-off-by: Alexandru Vasile <[email protected]>

* runtime: Implement new `Metadata` runtime trait

Signed-off-by: Alexandru Vasile <[email protected]>

* runtime: Move `metadata_at` functions to construct_runtime macro

Signed-off-by: Alexandru Vasile <[email protected]>

* contruct_runtime: Use `OpaqueMetadata` from hidden imports

Signed-off-by: Alexandru Vasile <[email protected]>

* Adjust testing

Signed-off-by: Alexandru Vasile <[email protected]>

* frame/tests: Add tests for the new API

Signed-off-by: Alexandru Vasile <[email protected]>

* frame/tests: Adjust metdata naming

Signed-off-by: Alexandru Vasile <[email protected]>

* frame/support: Expose `metadata-v14` feature flag

Signed-off-by: Alexandru Vasile <[email protected]>

* frame/support: Expose metadata only under feature flags

Signed-off-by: Alexandru Vasile <[email protected]>

* frame/support: Expose v14 metadata by default

Signed-off-by: Alexandru Vasile <[email protected]>

* frame/support: Expose metadata feature for testing

Signed-off-by: Alexandru Vasile <[email protected]>

* frame/support: Test metadata under different feature flags

Signed-off-by: Alexandru Vasile <[email protected]>

* Update primitives/api/src/lib.rs

Co-authored-by: Bastian Köcher <[email protected]>

* Update primitives/api/src/lib.rs

Co-authored-by: Bastian Köcher <[email protected]>

* client/tests: Adjust testing to reflect trait Metadata change

Signed-off-by: Alexandru Vasile <[email protected]>

* frame/metadata-ir: Add intermediate representation types for metadata

Signed-off-by: Alexandru Vasile <[email protected]>

* frame/metadata-ir: Convert metadata to V14

Signed-off-by: Alexandru Vasile <[email protected]>

* frame/metadata-ir: Add API to convert metadata to multiple versions

Signed-off-by: Alexandru Vasile <[email protected]>

* frame/metadata-ir: Expose V14 under feature flag

Signed-off-by: Alexandru Vasile <[email protected]>

* frame/support: Adjust to metadata IR

Signed-off-by: Alexandru Vasile <[email protected]>

* frame/support: More adjustments

Signed-off-by: Alexandru Vasile <[email protected]>

* frame/support: Guard v14 details under feature flag

Signed-off-by: Alexandru Vasile <[email protected]>

* frame/support: Adjust testing

Signed-off-by: Alexandru Vasile <[email protected]>

* CI: Ensure `quick-benchmarks` uses `metadata-v14`

Signed-off-by: Alexandru Vasile <[email protected]>

* frame/support: Use `metadata-v14` for benchmarks

Signed-off-by: Alexandru Vasile <[email protected]>

* Adjust cargo fmt

Signed-off-by: Alexandru Vasile <[email protected]>

* kitchensink-runtime: Add feature flag for `metadata-v14`

Signed-off-by: Alexandru Vasile <[email protected]>

* frame/support/test: Adjust testing

Signed-off-by: Alexandru Vasile <[email protected]>

* frame/support/test: Check crates locally

Signed-off-by: Alexandru Vasile <[email protected]>

* Activate metadata-v14 for pallets

Signed-off-by: Alexandru Vasile <[email protected]>

* Remove metadata-v14 feature flag

Signed-off-by: Alexandru Vasile <[email protected]>

* frame/metadata_ir: Move `api.rs` to `mod.rs`

Signed-off-by: Alexandru Vasile <[email protected]>

* frame/support: Handle latest metadata conversion via IR

Signed-off-by: Alexandru Vasile <[email protected]>

* frame/tests: Add constant for metadata version 14

Signed-off-by: Alexandru Vasile <[email protected]>

* frame/support/test: Fix merge conflict

Signed-off-by: Alexandru Vasile <[email protected]>

* Update frame/support/Cargo.toml

Co-authored-by: Bastian Köcher <[email protected]>

* Update frame/support/src/metadata_ir/mod.rs

Co-authored-by: Bastian Köcher <[email protected]>

* Update frame/support/test/Cargo.toml

Co-authored-by: Bastian Köcher <[email protected]>

* Update primitives/api/src/lib.rs

Co-authored-by: Bastian Köcher <[email protected]>

* frame/metadata: Collect pallet documentation for MetadataIR

Signed-off-by: Alexandru Vasile <[email protected]>

* frame/tests: Check pallet documentation is propagated to MetadataIR

Signed-off-by: Alexandru Vasile <[email protected]>

* frame/support: Improve documentation

Signed-off-by: Alexandru Vasile <[email protected]>

---------

Signed-off-by: Alexandru Vasile <[email protected]>
Co-authored-by: parity-processbot <>
Co-authored-by: Bastian Köcher <[email protected]>
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
A0-please_review Pull request needs code review. B1-note_worthy Changes should be noted in the release notes C1-low PR touches the given topic and has a low impact on builders. D3-trivial 🧸 PR contains trivial changes in a runtime directory that do not require an audit T1-runtime This PR/Issue is related to the topic “runtime”.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants