-
Notifications
You must be signed in to change notification settings - Fork 1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Lower MaxValidUntilBlockIncrement limit #2026
Comments
I think transactions should be kept for at least one year. |
But what's the rationale for this? We expect a lot of transactions on NeoFS chain and we're absolutely sure we can trim them more aggressively, but we want to keep the protocol the same, that's why we started analyzing historic data for public chains. And I think we have data here proving that one month is more than enough for public chains too, older data is not really used even on Neo 2. |
I don't see any benefit for this changing. |
We can move this value to protocol.json. Then NeoFS can set a different value. |
Yeah, I had this thought too, if it is to be configurable we can use different settings for different networks, let's at least do that for now. Though I still think lower MaxTraceableBlocks is possible and beneficial for public networks. |
This also has relation to P2P node state synchronization (snapshots or something similar), BTW. As we do plan for a lot of transactions we also want new nodes to synchronize faster than by processing the whole chain, so we plan to do some state synchronization in the future. Of course the node would still also need to have at least MaxTraceableBlocks number of blocks and the lower this value is the simpler it would be. |
@roman-khimov Don't forget other parameters that NeoFS may need. We may put them in the configuration file, too. |
Close? |
It's configurable now, so while this still has some data justifying lower |
Just FYI, on the effects of lowering the Public networks can still benefit from it. |
Summary or problem description
MaxValidUntilBlockIncrement constant introduced in #765 makes old blocks/transactions unavailable for current running scripts and allows to drop old blocks/transactions from the local storage. It's a very good thing, but at the same time the constant is set at the moment for 2102400 which is a year long life for 15-seconds block interval. I think it's a bit more than needed and we can be more aggressive here, dropping even more data.
We've analyzed Neo 2 mainnet (up to 6286643) and testnet (up to 4935922) chains for their accesses to blocks/headers/transactions (
GetBlock/GetHeader/GetTransaction
calls). There are 1010043 calls on mainnet and 680920 on testnet and there is a quite expected distribution of relative heights for these accesses:But of course the list is long for both networks and the most interesting thing here is the length of its tail. And it looks like this:
What was discovered is that blocks 0 and 1 are actually quite popular, so they're responsible for the tail here, there are 16367 such references on mainnet and 4087 on testnet. For those curios wondering what data from blocks 0 and 1 is being used we actually have an answer too (for both chains combined):
All of these references to 0 and 1 are broken by design on Neo 3 (they'll break at MaxValidUntilBlockIncrement+1), so we can ignore them and if we're to do that, we get the following tails:
Just 5 (out of 676833) calls would fail on testnet and 41 (out of 993676) with the current MaxValidUntilBlockIncrement. But I have to notice that these numbers are not zero already.
But then again, what's more interesting is how this old data is being used, so if we're to concentrate on requests past 128000 blocks from the current that would be 2671 calls for mainnet and 54 for testnet. They do the following requests for these headers/blocks/transactions (combined for mainnet/tesnet):
Obviously outputs and unspents are no longer relevant for Neo 3 and
GetHash
is probably related to those withGetTransactions
being a prologue to a deeper dive into transactions. The only thing left isGetTimestamp
which tends to be used quite a lot, but 3852 out of 3877 calls to it are actually generated by a single 4c7cca112a8c5666bce5da373010fc0920d0e0d2 mainnet contract and it really has a lot ofGetTimestamp
calls inside. IIUC it's because of code like this:But the contract should've just saved the timestamp previously, while storing this
selling
variable (otherwise it's broken on Neo 3 anyway).So even though there is a number of 128K+ calls, they're either related to UTXO model or should've been avoided in the first place. Which leads us to the proposal for more aggressive tail cutting on Neo 3.
Do you have any solution you want to propose?
I think we can safely limit MaxValidUntilBlockIncrement to 200K, that would affect just 43 calls from Neo 2 testnet (0.006%) and 2310 from Neo 2 mainnet (0.2%). 200K 15 seconds blocks is just a little more than a month of blockchain's life which I think is practical enough for any contracts, if they need some data from old blocks/transactions they have a month to process it or save it into their storage.
Neo Version
Where in the software does this update applies to?
The text was updated successfully, but these errors were encountered: