Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Electra: upgrade #13933

Merged
merged 42 commits into from
May 8, 2024
Merged
Show file tree
Hide file tree
Changes from 41 commits
Commits
Show all changes
42 commits
Select commit Hold shift + click to select a range
3b58756
wip fork logic upgrade
james-prysm Apr 29, 2024
e02475a
fixing replay and fork.go
james-prysm Apr 29, 2024
7d0c2bc
improving process function and adding tests for transition
james-prysm Apr 29, 2024
2f7f899
updating unit tests and temporarily removing some fields on state_tri…
james-prysm Apr 29, 2024
c594df8
updating state
james-prysm Apr 29, 2024
45ee712
wip adding upgrade to electra code
james-prysm Apr 29, 2024
0a2dae0
adding some comments
james-prysm Apr 30, 2024
5287f56
Merge branch 'develop' into electra-fork-logic
james-prysm May 3, 2024
b5560be
Merge branch 'develop' into electra-fork-logic
james-prysm May 6, 2024
1f7bb36
adding spec tests
james-prysm May 6, 2024
49cacc4
Merge branch 'develop' into electra-fork-logic
james-prysm May 6, 2024
38bca88
fixing values used in state transition logic
james-prysm May 6, 2024
de12137
updating upgrade test
james-prysm May 6, 2024
4dd8861
gofmt
james-prysm May 6, 2024
eae67f7
avoid dup word linting
james-prysm May 6, 2024
dd36ec6
fixing spec tests for fork
james-prysm May 6, 2024
6080275
gaz
james-prysm May 7, 2024
8ea9034
Merge branch 'develop' into electra-fork-logic
james-prysm May 7, 2024
abbdd1a
fixing tests
james-prysm May 7, 2024
0b34cf9
improving unit test with new getters
james-prysm May 7, 2024
5cfe8d2
fixing bazel for minimal fork test
james-prysm May 7, 2024
a4607c7
Merge branch 'develop' into electra-fork-logic
james-prysm May 7, 2024
3cf41ee
adding bazel file
james-prysm May 7, 2024
53552ad
Merge branch 'develop' into electra-fork-logic
james-prysm May 7, 2024
4cee2cb
Update beacon-chain/core/electra/upgrade.go
james-prysm May 7, 2024
78af849
addressing some comments and adding more tests
james-prysm May 7, 2024
0897bea
addressing more feedback
james-prysm May 7, 2024
266c7ec
one more feedback
james-prysm May 7, 2024
969ae73
changing value to interface after talking to preston
james-prysm May 7, 2024
174c6f6
adding missed review feedback
james-prysm May 7, 2024
ecc6e6a
fixing linting
james-prysm May 7, 2024
7f69c2d
noticed I was using the wrong function in the state upgrade
james-prysm May 7, 2024
62c0cee
Merge branch 'develop' into electra-fork-logic
james-prysm May 7, 2024
2b57557
fixing and ignoring some deepsource issues
james-prysm May 7, 2024
a523f30
Merge branch 'develop' into electra-fork-logic
james-prysm May 7, 2024
37dbcbd
moving core electra validator functions to helper to remove circular …
james-prysm May 7, 2024
7ddae4a
missed deepsource complaint
james-prysm May 7, 2024
1c9a6ae
Update upgrade.go
james-prysm May 8, 2024
cf89f83
Update testing/util/electra_state.go
james-prysm May 8, 2024
44993f6
Update testing/util/electra_state.go
james-prysm May 8, 2024
429b474
addressing feedback
james-prysm May 8, 2024
a915fbe
removing deepsoure ignore comments
james-prysm May 8, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
41 changes: 41 additions & 0 deletions beacon-chain/core/electra/BUILD.bazel
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
load("@prysm//tools/go:def.bzl", "go_library", "go_test")

go_library(
name = "go_default_library",
srcs = ["upgrade.go"],
importpath = "github.com/prysmaticlabs/prysm/v5/beacon-chain/core/electra",
visibility = ["//visibility:public"],
deps = [
"//beacon-chain/core/helpers:go_default_library",
"//beacon-chain/core/time:go_default_library",
"//beacon-chain/state:go_default_library",
"//beacon-chain/state/state-native:go_default_library",
"//config/params:go_default_library",
"//consensus-types/primitives:go_default_library",
"//encoding/bytesutil:go_default_library",
"//math:go_default_library",
"//proto/engine/v1:go_default_library",
"//proto/prysm/v1alpha1:go_default_library",
"//time/slots:go_default_library",
"@com_github_pkg_errors//:go_default_library",
],
)

go_test(
name = "go_default_test",
srcs = ["upgrade_test.go"],
deps = [
":go_default_library",
"//beacon-chain/core/helpers:go_default_library",
"//beacon-chain/core/time:go_default_library",
"//config/params:go_default_library",
"//consensus-types/primitives:go_default_library",
"//encoding/bytesutil:go_default_library",
"//math:go_default_library",
"//proto/engine/v1:go_default_library",
"//proto/prysm/v1alpha1:go_default_library",
"//testing/require:go_default_library",
"//testing/util:go_default_library",
"//time/slots:go_default_library",
],
)
314 changes: 314 additions & 0 deletions beacon-chain/core/electra/upgrade.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,314 @@
package electra

import (
"sort"

"github.com/pkg/errors"
"github.com/prysmaticlabs/prysm/v5/beacon-chain/core/helpers"
"github.com/prysmaticlabs/prysm/v5/beacon-chain/core/time"
"github.com/prysmaticlabs/prysm/v5/beacon-chain/state"
state_native "github.com/prysmaticlabs/prysm/v5/beacon-chain/state/state-native"
"github.com/prysmaticlabs/prysm/v5/config/params"
"github.com/prysmaticlabs/prysm/v5/consensus-types/primitives"
"github.com/prysmaticlabs/prysm/v5/encoding/bytesutil"
"github.com/prysmaticlabs/prysm/v5/math"
enginev1 "github.com/prysmaticlabs/prysm/v5/proto/engine/v1"
ethpb "github.com/prysmaticlabs/prysm/v5/proto/prysm/v1alpha1"
"github.com/prysmaticlabs/prysm/v5/time/slots"
)

// UpgradeToElectra updates inputs a generic state to return the version Electra state.
james-prysm marked this conversation as resolved.
Show resolved Hide resolved
// def upgrade_to_electra(pre: deneb.BeaconState) -> BeaconState:
//
// epoch = deneb.get_current_epoch(pre)
// latest_execution_payload_header = ExecutionPayloadHeader(
// parent_hash=pre.latest_execution_payload_header.parent_hash,
// fee_recipient=pre.latest_execution_payload_header.fee_recipient,
// state_root=pre.latest_execution_payload_header.state_root,
// receipts_root=pre.latest_execution_payload_header.receipts_root,
// logs_bloom=pre.latest_execution_payload_header.logs_bloom,
// prev_randao=pre.latest_execution_payload_header.prev_randao,
// block_number=pre.latest_execution_payload_header.block_number,
// gas_limit=pre.latest_execution_payload_header.gas_limit,
// gas_used=pre.latest_execution_payload_header.gas_used,
// timestamp=pre.latest_execution_payload_header.timestamp,
// extra_data=pre.latest_execution_payload_header.extra_data,
// base_fee_per_gas=pre.latest_execution_payload_header.base_fee_per_gas,
// block_hash=pre.latest_execution_payload_header.block_hash,
// transactions_root=pre.latest_execution_payload_header.transactions_root,
// withdrawals_root=pre.latest_execution_payload_header.withdrawals_root,
// blob_gas_used=pre.latest_execution_payload_header.blob_gas_used,
// excess_blob_gas=pre.latest_execution_payload_header.excess_blob_gas,
// deposit_receipts_root=Root(), # [New in Electra:EIP6110]
// withdrawal_requests_root=Root(), # [New in Electra:EIP7002],
// )
//
// exit_epochs = [v.exit_epoch for v in pre.validators if v.exit_epoch != FAR_FUTURE_EPOCH]
// if not exit_epochs:
// exit_epochs = [get_current_epoch(pre)]
// earliest_exit_epoch = max(exit_epochs) + 1
//
// post = BeaconState(
// # Versioning
// genesis_time=pre.genesis_time,
// genesis_validators_root=pre.genesis_validators_root,
// slot=pre.slot,
// fork=Fork(
// previous_version=pre.fork.current_version,
// current_version=ELECTRA_FORK_VERSION, # [Modified in Electra:EIP6110]
// epoch=epoch,
// ),
// # History
// latest_block_header=pre.latest_block_header,
// block_roots=pre.block_roots,
// state_roots=pre.state_roots,
// historical_roots=pre.historical_roots,
// # Eth1
// eth1_data=pre.eth1_data,
// eth1_data_votes=pre.eth1_data_votes,
// eth1_deposit_index=pre.eth1_deposit_index,
// # Registry
// validators=pre.validators,
// balances=pre.balances,
// # Randomness
// randao_mixes=pre.randao_mixes,
// # Slashings
// slashings=pre.slashings,
// # Participation
// previous_epoch_participation=pre.previous_epoch_participation,
// current_epoch_participation=pre.current_epoch_participation,
// # Finality
// justification_bits=pre.justification_bits,
// previous_justified_checkpoint=pre.previous_justified_checkpoint,
// current_justified_checkpoint=pre.current_justified_checkpoint,
// finalized_checkpoint=pre.finalized_checkpoint,
// # Inactivity
// inactivity_scores=pre.inactivity_scores,
// # Sync
// current_sync_committee=pre.current_sync_committee,
// next_sync_committee=pre.next_sync_committee,
// # Execution-layer
// latest_execution_payload_header=latest_execution_payload_header, # [Modified in Electra:EIP6110:EIP7002]
// # Withdrawals
// next_withdrawal_index=pre.next_withdrawal_index,
// next_withdrawal_validator_index=pre.next_withdrawal_validator_index,
// # Deep history valid from Capella onwards
// historical_summaries=pre.historical_summaries,
// # [New in Electra:EIP6110]
// deposit_receipts_start_index=UNSET_DEPOSIT_RECEIPTS_START_INDEX,
// # [New in Electra:EIP7251]
// deposit_balance_to_consume=0,
// exit_balance_to_consume=0,
// earliest_exit_epoch=earliest_exit_epoch,
// consolidation_balance_to_consume=0,
// earliest_consolidation_epoch=compute_activation_exit_epoch(get_current_epoch(pre)),
// pending_balance_deposits=[],
// pending_partial_withdrawals=[],
// pending_consolidations=[],
// )
//
// post.exit_balance_to_consume = get_activation_exit_churn_limit(post)
// post.consolidation_balance_to_consume = get_consolidation_churn_limit(post)
//
// # [New in Electra:EIP7251]
// # add validators that are not yet active to pending balance deposits
// pre_activation = sorted([
// index for index, validator in enumerate(post.validators)
// if validator.activation_epoch == FAR_FUTURE_EPOCH
// ], key=lambda index: (
// post.validators[index].activation_eligibility_epoch,
// index
// ))
//
// for index in pre_activation:
// queue_entire_balance_and_reset_validator(post, ValidatorIndex(index))
//
// # Ensure early adopters of compounding credentials go through the activation churn
// for index, validator in enumerate(post.validators):
// if has_compounding_withdrawal_credential(validator):
// queue_excess_active_balance(post, ValidatorIndex(index))
//
// return post
//
// skipcq: GO-R1005
func UpgradeToElectra(beaconState state.BeaconState) (state.BeaconState, error) {
currentSyncCommittee, err := beaconState.CurrentSyncCommittee()
if err != nil {
return nil, err
}
nextSyncCommittee, err := beaconState.NextSyncCommittee()
if err != nil {
return nil, err
}
prevEpochParticipation, err := beaconState.PreviousEpochParticipation()
if err != nil {
return nil, err
}
currentEpochParticipation, err := beaconState.CurrentEpochParticipation()
if err != nil {
return nil, err
}
inactivityScores, err := beaconState.InactivityScores()
if err != nil {
return nil, err
}
payloadHeader, err := beaconState.LatestExecutionPayloadHeader()
if err != nil {
return nil, err
}
txRoot, err := payloadHeader.TransactionsRoot()
if err != nil {
return nil, err
}
wdRoot, err := payloadHeader.WithdrawalsRoot()
if err != nil {
return nil, err
}
wi, err := beaconState.NextWithdrawalIndex()
if err != nil {
return nil, err
}
vi, err := beaconState.NextWithdrawalValidatorIndex()
if err != nil {
return nil, err
}
summaries, err := beaconState.HistoricalSummaries()
if err != nil {
return nil, err
}
historicalRoots, err := beaconState.HistoricalRoots()
if err != nil {
return nil, err
}
excessBlobGas, err := payloadHeader.ExcessBlobGas()
if err != nil {
return nil, err
}
blobGasUsed, err := payloadHeader.BlobGasUsed()
if err != nil {
return nil, err
}

// [New in Electra:EIP7251]
Copy link
Contributor

@rkapka rkapka May 8, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nitpick: I think this comment should come before exitEpochs declaration

earliestExitEpoch := time.CurrentEpoch(beaconState)
preActivationIndices := make([]primitives.ValidatorIndex, 0)
compoundWithdrawalIndices := make([]primitives.ValidatorIndex, 0)
if err = beaconState.ReadFromEveryValidator(func(index int, val state.ReadOnlyValidator) error {
if val.ExitEpoch() != params.BeaconConfig().FarFutureEpoch && val.ExitEpoch() > earliestExitEpoch {
earliestExitEpoch = val.ExitEpoch()
}
if val.ActivationEpoch() == params.BeaconConfig().FarFutureEpoch {
preActivationIndices = append(preActivationIndices, primitives.ValidatorIndex(index))
}
if helpers.HasCompoundingWithdrawalCredential(val) {
compoundWithdrawalIndices = append(compoundWithdrawalIndices, primitives.ValidatorIndex(index))
}
return nil
}); err != nil {
return nil, err
}

earliestExitEpoch++ // Increment to find the earliest possible exit epoch

// note: should be the same in prestate and post beaconState.
// we are deviating from the specs a bit as it calls for using the post beaconState
tab, err := helpers.TotalActiveBalance(beaconState)
if err != nil {
return nil, errors.Wrap(err, "failed to get total active balance")
}

s := &ethpb.BeaconStateElectra{
GenesisTime: beaconState.GenesisTime(),
GenesisValidatorsRoot: beaconState.GenesisValidatorsRoot(),
Slot: beaconState.Slot(),
Fork: &ethpb.Fork{
PreviousVersion: beaconState.Fork().CurrentVersion,
CurrentVersion: params.BeaconConfig().ElectraForkVersion,
Epoch: time.CurrentEpoch(beaconState),
},
LatestBlockHeader: beaconState.LatestBlockHeader(),
BlockRoots: beaconState.BlockRoots(),
StateRoots: beaconState.StateRoots(),
HistoricalRoots: historicalRoots,
Eth1Data: beaconState.Eth1Data(),
Eth1DataVotes: beaconState.Eth1DataVotes(),
Eth1DepositIndex: beaconState.Eth1DepositIndex(),
Validators: beaconState.Validators(),
Balances: beaconState.Balances(),
RandaoMixes: beaconState.RandaoMixes(),
Slashings: beaconState.Slashings(),
PreviousEpochParticipation: prevEpochParticipation,
CurrentEpochParticipation: currentEpochParticipation,
JustificationBits: beaconState.JustificationBits(),
PreviousJustifiedCheckpoint: beaconState.PreviousJustifiedCheckpoint(),
CurrentJustifiedCheckpoint: beaconState.CurrentJustifiedCheckpoint(),
FinalizedCheckpoint: beaconState.FinalizedCheckpoint(),
InactivityScores: inactivityScores,
CurrentSyncCommittee: currentSyncCommittee,
NextSyncCommittee: nextSyncCommittee,
LatestExecutionPayloadHeader: &enginev1.ExecutionPayloadHeaderElectra{
ParentHash: payloadHeader.ParentHash(),
FeeRecipient: payloadHeader.FeeRecipient(),
StateRoot: payloadHeader.StateRoot(),
ReceiptsRoot: payloadHeader.ReceiptsRoot(),
LogsBloom: payloadHeader.LogsBloom(),
PrevRandao: payloadHeader.PrevRandao(),
BlockNumber: payloadHeader.BlockNumber(),
GasLimit: payloadHeader.GasLimit(),
GasUsed: payloadHeader.GasUsed(),
Timestamp: payloadHeader.Timestamp(),
ExtraData: payloadHeader.ExtraData(),
BaseFeePerGas: payloadHeader.BaseFeePerGas(),
BlockHash: payloadHeader.BlockHash(),
TransactionsRoot: txRoot,
WithdrawalsRoot: wdRoot,
ExcessBlobGas: excessBlobGas,
BlobGasUsed: blobGasUsed,
DepositReceiptsRoot: bytesutil.Bytes32(0), // [New in Electra:EIP6110]
WithdrawalRequestsRoot: bytesutil.Bytes32(0), // [New in Electra:EIP7002]
},
NextWithdrawalIndex: wi,
NextWithdrawalValidatorIndex: vi,
HistoricalSummaries: summaries,

DepositReceiptsStartIndex: params.BeaconConfig().UnsetDepositReceiptsStartIndex,
DepositBalanceToConsume: 0,
ExitBalanceToConsume: helpers.ActivationExitChurnLimit(math.Gwei(tab)),
EarliestExitEpoch: earliestExitEpoch,
ConsolidationBalanceToConsume: helpers.ConsolidationChurnLimit(math.Gwei(tab)),
EarliestConsolidationEpoch: helpers.ActivationExitEpoch(slots.ToEpoch(beaconState.Slot())),
PendingBalanceDeposits: make([]*ethpb.PendingBalanceDeposit, 0),
PendingPartialWithdrawals: make([]*ethpb.PendingPartialWithdrawal, 0),
PendingConsolidations: make([]*ethpb.PendingConsolidation, 0),
}

// Sorting preActivationIndices based on a custom criteria
sort.Slice(preActivationIndices, func(i, j int) bool {
// Comparing based on ActivationEligibilityEpoch and then by index if the epochs are the same
if s.Validators[preActivationIndices[i]].ActivationEligibilityEpoch == s.Validators[preActivationIndices[j]].ActivationEligibilityEpoch {
return preActivationIndices[i] < preActivationIndices[j]
}
return s.Validators[preActivationIndices[i]].ActivationEligibilityEpoch < s.Validators[preActivationIndices[j]].ActivationEligibilityEpoch
})
Comment on lines +283 to +290
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we need to sort here? I know spec says it but why? We should avoid wasteful compute if we can

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

validators might have different ActivationEligibilityEpoch and should be sorted based on that value, there is also a tie breaker situation. i don't have a good solution to replace this right now.


// need to cast the beaconState to use in helper functions
post, err := state_native.InitializeFromProtoUnsafeElectra(s)
if err != nil {
return nil, errors.Wrap(err, "failed to initialize post electra beaconState")
}

for _, index := range preActivationIndices {
if err := helpers.QueueEntireBalanceAndResetValidator(post, index); err != nil {
return nil, errors.Wrap(err, "failed to queue entire balance and reset validator")
}
}

// Ensure early adopters of compounding credentials go through the activation churn
for _, index := range compoundWithdrawalIndices {
if err := helpers.QueueExcessActiveBalance(post, index); err != nil {
return nil, errors.Wrap(err, "failed to queue excess active balance")
}
}

return post, nil
}
Loading
Loading