Skip to content
This repository has been archived by the owner on Jun 6, 2024. It is now read-only.

Support pre-runtime digest and instant seal #15

Merged
merged 56 commits into from
Dec 8, 2021
Merged
Show file tree
Hide file tree
Changes from 38 commits
Commits
Show all changes
56 commits
Select commit Hold shift + click to select a range
0c3b5cc
checkpoint. Gonna try switching to preruntime digest first
JoshOrndorff Nov 2, 2021
699f27d
pallet allows preruntime digests and validates at end of block
JoshOrndorff Nov 2, 2021
aee4b36
Merge branch 'joshy-instant-seal' into joshy-runtime-supports-prerunt…
JoshOrndorff Nov 3, 2021
ce3e0cc
remove incorrect comment
JoshOrndorff Nov 3, 2021
4c142b9
factor slot prediction into helper function, start sketching manual s…
JoshOrndorff Nov 3, 2021
74eb50c
better message for future self
JoshOrndorff Nov 3, 2021
48c1877
preruntime digest interface
JoshOrndorff Nov 3, 2021
f34d824
`create_digest` compiles
JoshOrndorff Nov 3, 2021
51040b0
service compiles
JoshOrndorff Nov 3, 2021
791e919
whoe service checks
JoshOrndorff Nov 3, 2021
3eea250
allow running from cli
JoshOrndorff Nov 3, 2021
67c18cd
actually install our new manual seal consensus data provider
JoshOrndorff Nov 3, 2021
65ab064
Handle no-key scenario
JoshOrndorff Nov 3, 2021
c97b9f4
Preruntime digest an no memoizing author account
JoshOrndorff Nov 3, 2021
1cf6ede
remove wrong comment. Better to follow aura
JoshOrndorff Nov 3, 2021
2265da2
prune commented inherent-related code
JoshOrndorff Nov 3, 2021
5769f43
Hack in fake signature in manual seal.
JoshOrndorff Nov 3, 2021
aae5122
unused import
JoshOrndorff Nov 3, 2021
1acb47f
verifier supports both digest types
JoshOrndorff Nov 3, 2021
e8caac9
duplicate code from verifier to executor wrapper
JoshOrndorff Nov 3, 2021
2f3a9f4
author inherent no longer required
JoshOrndorff Nov 3, 2021
770820c
sign blocks in manual seal !!
JoshOrndorff Nov 3, 2021
521e5cb
Hack import queue so it works in instant seal although it is now brok…
JoshOrndorff Nov 3, 2021
912eb75
prune some debugging logs
JoshOrndorff Nov 3, 2021
caf54d3
NimbusBlockImport with optional parachain rules
JoshOrndorff Nov 4, 2021
3aecd84
we can use std on the client-side
JoshOrndorff Nov 4, 2021
c0867b0
remove unnecessary generic
JoshOrndorff Nov 4, 2021
263a286
indentation and comment
JoshOrndorff Nov 4, 2021
8b7adb4
use constant instead of hard-coded nmbs
JoshOrndorff Nov 4, 2021
c2ecc7d
code style: map instead of if let
JoshOrndorff Nov 4, 2021
df49153
assume relay chain slot beacon in manual seal for now
JoshOrndorff Nov 4, 2021
20fe8b4
Actually checkvalidity and send notification when using preruntime di…
JoshOrndorff Nov 12, 2021
8f232ac
Typos from code review
JoshOrndorff Nov 12, 2021
36c0aec
Proper index
JoshOrndorff Nov 12, 2021
20c07ee
Switch consensus worker from inherent to preruntime digest.
JoshOrndorff Nov 12, 2021
fdebe92
map from author id to account id in FindAuthor
JoshOrndorff Nov 12, 2021
807e72b
more toward parachain consensus worker
JoshOrndorff Nov 12, 2021
e663411
Merge branch 'main' into joshy-runtime-supports-preruntime-digests
JoshOrndorff Nov 18, 2021
f16dd8f
Update nimbus-consensus/src/import_queue.rs
JoshOrndorff Dec 1, 2021
03b3938
inherent is a no-op
JoshOrndorff Dec 1, 2021
8c8ce30
Revert "more toward parachain consensus worker"
JoshOrndorff Dec 1, 2021
9167b63
validators don't need backward compatability
JoshOrndorff Dec 1, 2021
227800a
Merge branch 'main' into joshy-runtime-supports-preruntime-digests
JoshOrndorff Dec 2, 2021
6272156
Merge branch 'main' into joshy-runtime-supports-preruntime-digests
JoshOrndorff Dec 6, 2021
46eed0d
Fix incorrect merge conflict resolution. Still detect NimbusApi
JoshOrndorff Dec 6, 2021
e2d9492
Add copyright to manual seal file.
JoshOrndorff Dec 6, 2021
9f65a45
Typo found by @girazoki
JoshOrndorff Dec 6, 2021
4eca377
stray space
JoshOrndorff Dec 6, 2021
7e18518
Single author in dev spec so that `run-instant-seal --dev` works with…
JoshOrndorff Dec 6, 2021
1617137
warnings
JoshOrndorff Dec 6, 2021
fe2315c
Apply suggestions from code review
JoshOrndorff Dec 7, 2021
05097c5
better weight in `on_initialize`
JoshOrndorff Dec 7, 2021
b0dd22c
thoughts about the legacy author inherent
JoshOrndorff Dec 7, 2021
75e256d
sketch kick-off inherent in pallet
JoshOrndorff Dec 7, 2021
c3cf4b1
fees
JoshOrndorff Dec 7, 2021
ffb03c0
Update the inherent extrinsics
JoshOrndorff Dec 7, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
362 changes: 212 additions & 150 deletions Cargo.lock

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion nimbus-consensus/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -15,10 +15,10 @@ sp-block-builder = { git = "https:/paritytech/substrate", branch = "
sp-api = { git = "https:/paritytech/substrate", branch = "master" }
sc-client-api = { git = "https:/paritytech/substrate", branch = "master" }
sc-consensus = { git = "https:/paritytech/substrate", branch = "master" }
sc-consensus-manual-seal = { git = "https:/paritytech/substrate", branch = "master" }
substrate-prometheus-endpoint = { git = "https:/paritytech/substrate", branch = "master" }
sp-keystore = { git = "https:/paritytech/substrate", branch = "master" }
sp-application-crypto = { git = "https:/paritytech/substrate", branch = "master" }
sp-std = { git = "https:/paritytech/substrate", branch = "master" }

# Polkadot dependencies
polkadot-client = { git = "https:/paritytech/polkadot", branch = "master", default-features = false }
Expand Down
86 changes: 72 additions & 14 deletions nimbus-consensus/src/import_queue.rs
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ use sp_runtime::{
traits::{Block as BlockT, Header as HeaderT},
DigestItem,
};
use nimbus_primitives::{NimbusId, NimbusPair, digests::CompatibleDigestItem};
use nimbus_primitives::{NimbusId, NimbusPair, digests::CompatibleDigestItem, NIMBUS_ENGINE_ID};
use sp_application_crypto::{Pair as _, Public as _};
use log::debug;

Expand Down Expand Up @@ -76,24 +76,20 @@ where

debug!(target: crate::LOG_TARGET, "🪲 Signature according to verifier is {:?}", signature);

// Grab the digest from the runtime
//TODO use the trait. Maybe this code should move to the trait.
let consensus_digest = block_params.header
// Grab the author information from either the preruntime digest or the consensus digest
//TODO use the trait
let claimed_author = block_params.header
.digest()
.logs
.iter()
.find(|digest| {
.find_map(|digest| {
match *digest {
DigestItem::Consensus(id, _) if id == b"nmbs" => true,
_ => false,
DigestItem::Consensus(id, ref author_id) if id == NIMBUS_ENGINE_ID => Some(author_id.clone()),
DigestItem::PreRuntime(id, ref author_id) if id == NIMBUS_ENGINE_ID => Some(author_id.clone()),
_ => None,
Comment on lines -87 to +89
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@joelamouche This is the change necessary to extract author information. Previously we just used the Consensus Digest. Now wee will use either the consensus digest or the preruntime digest. Eventually, support for the Consensus digest will be dropped.

}
})
.expect("A single consensus digest should be added by the runtime when executing the author inherent.");

let claimed_author = match *consensus_digest {
DigestItem::Consensus(id, ref author_id) if id == *b"nmbs" => author_id.clone(),
_ => panic!("Expected consensus digest to contains author id bytes"),
};
.expect("Expected one consensus or pre-runtime digest that contains author id bytes");

debug!(target: crate::LOG_TARGET, "🪲 Claimed Author according to verifier is {:?}", claimed_author);

Expand Down Expand Up @@ -152,6 +148,8 @@ where

block_params.post_digests.push(seal);

block_params.fork_choice = Some(sc_consensus::ForkChoiceStrategy::LongestChain);
JoshOrndorff marked this conversation as resolved.
Show resolved Hide resolved

debug!(target: crate::LOG_TARGET, "🪲 Just finished verifier. posthash from params is {:?}", &block_params.post_hash());

Ok((block_params, None))
Expand All @@ -165,6 +163,7 @@ pub fn import_queue<Client, Block: BlockT, I, CIDP>(
create_inherent_data_providers: CIDP,
spawner: &impl sp_core::traits::SpawnEssentialNamed,
registry: Option<&substrate_prometheus_endpoint::Registry>,
parachain: bool,
) -> ClientResult<BasicQueue<Block, I::Transaction>>
where
I: BlockImport<Block, Error = ConsensusError> + Send + Sync + 'static,
Expand All @@ -181,11 +180,70 @@ where

Ok(BasicQueue::new(
verifier,
Box::new(cumulus_client_consensus_common::ParachainBlockImport::new(
Box::new(NimbusBlockImport::new(
block_import,
parachain,
)),
None,
spawner,
registry,
))
}

/// Nimbus specific block import.
///
/// Nimbus supports both parachain and non-parachain contexts. In the parachain
/// context, new blocks should not be imported as best. Cumulus's ParachainBlockImport
/// handles this correctly, but does not work in non-parachain contexts.
/// This block import has a field indicating whether we should apply parachain rules or not.
///
/// There may be additional nimbus-specific logic here in the future, but for now it is
/// only the conditional parachain logic
pub struct NimbusBlockImport<I>{
inner: I,
parachain_context: bool,
}

impl<I> NimbusBlockImport<I> {
/// Create a new instance.
pub fn new(inner: I, parachain_context: bool) -> Self {
Self{
inner,
parachain_context,
}
}
}

#[async_trait::async_trait]
impl<Block, I> BlockImport<Block> for NimbusBlockImport<I>
where
Block: BlockT,
I: BlockImport<Block> + Send,
{
type Error = I::Error;
type Transaction = I::Transaction;

async fn check_block(
&mut self,
block: sc_consensus::BlockCheckParams<Block>,
) -> Result<sc_consensus::ImportResult, Self::Error> {
self.inner.check_block(block).await
}

async fn import_block(
&mut self,
mut block_import_params: sc_consensus::BlockImportParams<Block, Self::Transaction>,
cache: std::collections::HashMap<sp_consensus::CacheKeyId, Vec<u8>>,
) -> Result<sc_consensus::ImportResult, Self::Error> {
// If we are in the parachain context, best block is determined by the relay chain
// except during initial sync
if self.parachain_context {
block_import_params.fork_choice = Some(sc_consensus::ForkChoiceStrategy::Custom(
block_import_params.origin == sp_consensus::BlockOrigin::NetworkInitialSync,
));
}

// Now continue on to the rest of the import pipeline.
self.inner.import_block(block_import_params, cache).await
}
}
Loading