ethclient: BlockByNumber cannot handle pending block

[757566833]

System information

Geth version: ethereum/client-go:alltools-v1.10.20
OS & Version:Linux

github.com/ethereum/go-ethereum v1.10.20

Expected behaviour

unmarshal

Actual behaviour

cannot unmarshal

Steps to reproduce the behaviour

import (
	"github.com/ethereum/go-ethereum/ethclient"
)
client, err := ethclient.Dial("rpc")
client.BlockByNumber(context.Background(), new(big.Int).SetInt64(-1))

Backtrace

[backtrace]

When submitting logs: please submit them as text and not screenshots.

json: cannot unmarshal non-string into Go struct field rpcBlock.hash of type common.Hash

i want to get pending tx by ethclient

raw

{
    "baseFeePerGas":"0x7",
    "difficulty":"0x747e5",
    "extraData":"0xd883010a14846765746888676f312e31382e33856c696e7578",
    "gasLimit":"0x1c9c380",
    "gasUsed":"0xa410",
    "hash":null,
    "logsBloom":"0x00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000",
    "miner":null,
    "mixHash":"0x0000000000000000000000000000000000000000000000000000000000000000",
    "nonce":null,
    "number":"0x17af0",
    "parentHash":"0x5e572fe3ce0eb8a4500e9484761d1f0ba7fe50cb8f4741c3bed7550a89e13a1e",
    "receiptsRoot":"0x75308898d571eafb5cd8cde8278bf5b3d13c5f6ec074926de3bb895b519264e1",
    "sha3Uncles":"0x1dcc4de8dec75d7aab85b567b6ccd41ad312451b948a7413f0a142fd40d49347",
    "size":"0x31c",
    "stateRoot":"0x0000000000000000000000000000000000000000000000000000000000000000",
    "timestamp":"0x62fc648c",
    "totalDifficulty":null,
    "transactions":[
        {
            "blockHash":"0x653ca39fd6ab97ab1f88f0a1d6c1d38d8072ea4bd0b23d3410b548384a9d3b93",
            "blockNumber":"0x17af0",
            "from":"0xfe6154079ef3342e72b119636e839f1bb61cf29e",
            "gas":"0x5208",
            "gasPrice":"0x3b9aca00",
            "maxFeePerGas":"0x3b9aca00",
            "maxPriorityFeePerGas":"0x3b9aca00",
            "hash":"0x3a1ca417aef3418f209feb14fc89ae97ad85a72e32d7f9e332ceea9cbd08ae5b",
            "input":"0x",
            "nonce":"0x9",
            "to":"0x2a61b29ab416e0239e20f5bccf7319dab9cb3982",
            "transactionIndex":"0x0",
            "value":"0xde0b6b3a7640000",
            "type":"0x2",
            "accessList":[

            ],
            "chainId":"0x7c6807fe612",
            "v":"0x1",
            "r":"0xf3a980088d625acdf948833298cc734e1aff0a942b16bb9223cfe280b981d4c4",
            "s":"0x29b0052a2a21a7ee048289aa75b6e9cc33d723c1162cdcd27a2225ac763cc579"
        },
        {
            "blockHash":"0x653ca39fd6ab97ab1f88f0a1d6c1d38d8072ea4bd0b23d3410b548384a9d3b93",
            "blockNumber":"0x17af0",
            "from":"0xfe6154079ef3342e72b119636e839f1bb61cf29e",
            "gas":"0x5208",
            "gasPrice":"0x2540be400",
            "maxFeePerGas":"0x2540be400",
            "maxPriorityFeePerGas":"0x2540be400",
            "hash":"0xd3c0ab01e16e88193c7af35ac9cd5635ad3841d5496f58c3386f730a25681ef2",
            "input":"0x",
            "nonce":"0xa",
            "to":"0x2a61b29ab416e0239e20f5bccf7319dab9cb3982",
            "transactionIndex":"0x1",
            "value":"0x1bc16d674ec80000",
            "type":"0x2",
            "accessList":[

            ],
            "chainId":"0x7c6807fe612",
            "v":"0x0",
            "r":"0x91f640d98ef1e683c7acce05e87b44af345a83a03231ef4e9110dbbd77f8c2b3",
            "s":"0x18215088ecaba584ea6c16ab5ea0cee6b8c5b97afb9b682d99b00d8e1231504c"
        }
    ],
    "transactionsRoot":"0xa84ebbdec21a798e52af4978b9fb56b9d964b5a8ae5de7f016c633d7a7768bfe",
    "uncles":[

    ]
}

but body (rpcblock)

// ethclient line 104
type rpcBlock struct {
	Hash         common.Hash      `json:"hash"`
	Transactions []rpcTransaction `json:"transactions"`
	UncleHashes  []common.Hash    `json:"uncles"`
}

[holiman]

Yeah this is a wart in our API, it seems. cc @fjl any thoughts?

[criogennn]

This is any way to fix that now? or waiting to new commit?

[lightclient]

I sort of think "pending" is not the best abstraction in the first place. What is a "pending" block? It seems to be mainly used for the following:

• calculating next base fee
• get a list of txs the client thinks could be in next block
• ?

Working towards removing "pending" from the RPC altogether seems it would be a better idea.

[757566833]

I sort of think "pending" is not the best abstraction in the first place. What is a "pending" block? It seems to be mainly used for the following:

• calculating next base fee
• get a list of txs the client thinks could be in next block
• ?

Working towards removing "pending" from the RPC altogether seems it would be a better idea.

Sometimes transactions are always pending because of wrong parameters, I get them through “pending”; do you have any other ways to recommend？

[fjl]

@757566833, the comment by @lightclient is about possibly changing the RPC specification in the future. At the current time the "pending" block is supported and you can use it.

[lightclient]

do you have any other ways to recommend？

We have txpool_content which returns the pending and queued transactions. I think this would return what you need.

[fjl]

About this issue: the request is failing because the hash field of the pending block response is null. We need the block hash to request other block parts such as uncle headers. So, I think this is not easy to fix.

[fjl]

BTW, it's a bad idea to use the pending block for getting transactions. The block will only return a limited number of txs, and are chosen by the go-ethereum mining algorithm, which is not guaranteed to match the next block. So you will miss some transactions when using this method.

[lightclient]

It seems like 3 ways to resolve this issue:

1. Update ethclient to accept a pointer value of the optional fields, and maybe add some logic to verify those values are only ever nil when requesting a pending block.
2. Fill in the values like mixDigest, nonce, etc with defaults for pending blocks and calculate the hash from that.
3. Stop supporting pending tag.

Any other ideas?

[fjl]

Stop supporting pending tag.

Note: at the moment, BlockByNumber in ethclient does not officially support the pending block. This issue is more like a question whether this should be supported.

[lightclient]

@fjl ah okay! In that case would it not be best to just return error that it isn't supported if someone sends -1 through?

[criogennn]

i just see in expected behaviour fieild 'from' in transactions list. I need this field but i cant find this in other methods, and sometimes subscribe to pending tx (SubscribePendingTransactions) return us nil, in this case we skip some transaction? Mb i need tx_pool content but i cant receive list of txs in this method, couz i just weak in golang, cant understand how use this method

[holiman]

In that case would it not be best to just return error that it isn't supported if someone sends -1 through?

@lightclient we kind of do. The error we return is json: cannot unmarshal non-string into Go struct field rpcBlock.hash of type common.Hash. So IMO the behaviour here is as expected according to spec/docs.

[fjl]

@criogennn Your use case, where transactions sent be SubscribePendingTransactions will become missing, will be helped by #25186.

[fjl]

We will not add support for retrieving the complete pending block in ethclient. It doesn't make sense, the pending block will change all the time and is not a reliable source of information.

[lightclient]

Although the result is the same, an error is returned to the caller, I think the error message as-is is not fair to the caller.