Stratum v2 Template Provider via IPC interface (multiprocess) #48
Conversation
|
@ryanofsky I slightly mangled e373642 because of bitcoin@4e1a434#r1681193702. I'll clean that up later. |
|
Incorporated the changes from bitcoin#30437 (comment) and cleaned up commit history. |
|
@ryanofsky do you have a commit somewhere that adds libmultiprocess to the guix build? |
Sjors
force-pushed
the
sv2-ipc
branch
2 times, most recently
from
c75792b
to
91720b7
Compare
|
Getting a cryptic Somehow related to the Template Provider depending on I can't make sense of the |
Sjors
force-pushed
the
2024/07/sv2-tp-common
branch
2 times, most recently
from
db58076
to
1ff4660
Compare
Sjors
force-pushed
the
sv2-ipc
branch
3 times, most recently
from
4354e96
to
3aa6f0a
Compare
Sjors
force-pushed
the
2024/07/sv2-tp-common
branch
from
1ff4660
to
ee010f5
Compare
|
re: #48 (comment)
I don't, since I haven't really experimented with the guix build. Maybe simplest approach to start with would enable the multiprocess build option and build and package all the multiprocess executables including bitcoin-tp and bitcoin-node. I know in the longer run you want to have separate releases as described in bitcoin#30437 (comment). re: #48 (comment)
This is a -bitcoin_mine_CPPFLAGS = $(bitcoin_bin_cppflags)
+bitcoin_mine_CPPFLAGS = $(bitcoin_bin_cppflags) $(BOOST_CPPFLAGS)to add the right -isystem option.
Not sure about this error. Link seems to be https://cirrus-ci.com/task/4759864851824640?logs=ci#L2486 |
Sjors
force-pushed
the
2024/07/sv2-tp-common
branch
2 times, most recently
from
19cc9f1
to
dabc32c
Compare
Sjors
force-pushed
the
sv2-ipc
branch
3 times, most recently
from
6c24b45
to
a8bb152
Compare
|
I'm getting the MSAN / TSAN error inconsistently on my self-hosted CI. Does not seem related to this PR. Trying to reproduce in isolation in #51. |
Sjors
force-pushed
the
2024/07/sv2-tp-common
branch
2 times, most recently
from
3779b1e
to
94f55ad
Compare
|
Rebased after bitcoin#30510. |
Sjors
force-pushed
the
2024/07/sv2-tp-common
branch
from
94f55ad
to
6ac456e
Compare
This update brings in the following changes: chaincodelabs/libmultiprocess#111: doc: Add internal design section chaincodelabs/libmultiprocess#113 Add missing include to util.h
This update brings in the following changes: chaincodelabs/libmultiprocess#116 shutdown bugfix: destroy RPC system before running cleanup callbacks
Allows InitError and InitWarning to be used by other executables beside bitcoind and bitcoin-node.
See src/bitcoin-mine.cpp for usage information. Co-authored-by: Sjors Provoost <[email protected]>
The Template Provider binary: * won't have a wallet * probably won't have a gui (maybe after the QT revamp) * doesn't use the functional test framework Disable some CI stuff to speed it up. This is not a complete list, because: 1. That would be too hard to rebase 2. Some things might be added later, e.g. tracepoints
Except for Windows.
|
Still a few things left to clean up, but this now produces a working I pushed a commit to bitcoin#30975, because some of the CI failures don't seem related to sv2. |
For now also build bitcoin-node.
TODO: move to bitcoin#29346
…ution() 525e9dc Add submitSolution to BlockTemplate interface (Sjors Provoost) 47b4875 Add getCoinbaseMerklePath() to Mining interface (Sjors Provoost) 63d6ad7 Move BlockMerkleBranch back to merkle.{h,cpp} (Sjors Provoost) Pull request description: The new `BlockTemplate` interface introduced in #30440 allows for a more efficient way for a miner to submit the block solution. Instead of having the send the full block, it only needs to provide the nonce, timestamp, version fields and coinbase transaction. This PR introduces `submitSolution()` for that. It's currently unused. #29432 and Sjors#48 use it to process the Stratum v2 message [SubmitSolution](https:/stratum-mining/sv2-spec/blob/main/07-Template-Distribution-Protocol.md#77-submitsolution-client---server). The method should be sufficiently generic to work with alternative mining protocols (none exist that I'm aware off). This PR also introduces `getCoinbaseMerklePath()`, which is needed in Stratum v2 to construct the `merkle_path` field of the `NewTemplate` message (see [spec](https:/stratum-mining/sv2-spec/blob/main/07-Template-Distribution-Protocol.md#72-newtemplate-server---client)). The coinbase merkle path is also used in Stratum "v1", see e.g. https://bitcoin.stackexchange.com/questions/109820/questions-on-merkle-root-hashing-for-stratum-pools This last function uses `BlockMerkleBranch` which was moved to the test code in #13191. The reason back then for moving it was that it was no longer used. This PR moves it back. This PR does not change behaviour since both methods are unused. ACKs for top commit: achow101: ACK 525e9dc itornaza: Code review ACK 525e9dc tdb3: Code review and light test ACK 525e9dc ryanofsky: Code review ACK 525e9dc. Left minor suggestions but none are important, and looks like this could be merged as-is Tree-SHA512: 2a6a8f5d409ff4926643193cb67702240c7c687615414371e53383d2c13c485807f65e21e8ed98515b5456eca3d9fca13cec04675814a4081467d88b849c5653
Based on #49 and bitcoin#30437. This is an alternative approach to bitcoin#29432. Rather than integrating the Template Provider directly into Bitcoin Core, we make it an entirely separate application that communicates via IPC, see design/multiprocess.md.
Usage
Compile as usual, multiprocess is enabled by default.
Alternatively get a recent binary, see (unofficial, by me) releases ending in "ICP".
Start node:
Start the Template Provider:
See general Stratum v2 usage instructions in bitcoin#29432.
Rationale, PRs this builds on
There are multiple ways to create such an application. It could be built from scratch in Rust, using bits and pieces of the Stratum Reference Implementation SRI. It could be a c++ application built from scratch, perhaps using some libraries from Bitcoin Core.
But this PR goes about in a slightly different way. Similar to Elements or Knots it's a set of patches on top of the full Bitcoin Core codebase. This allows me to leverage the build system, subclass
Transportand use various other useful bits.This lets me reuse almost all code from the integrated Template Provider bitcoin#29432. Both build direct on:
This PR relies on the interfaces changes proposed in:
And the build changes proposed in:
These interface changes are also used by bitcoin#29432, but there they run in the same process.
I modified the guix script to only produce
bitcoin-node(for now) andbitcoin-minebinaries.Goal
The goal is to drop
bitcoin-nodefrom the release here. Ideally users would install Bitcoin Core in the manner they're familiar with. They would then installbitcoin-mineseparately and it should just work(tm), at most having to add a line tobitcoin.confto turn IPC on.This won't work until we ship a release with multiprocess and the mining interface enabled, ideally in
bitcoindandbitcoin-qtrather than in seperatebitcoin-nodeandbitcoin-guibinaries. See bitcoin#30437 for progress towards that.It might eventually be feasible to complete separate the codebase. A first step towards that would be to turn
libbitcoin_netinto a library (see #47). However for the time being I'd rather rebase on Bitcoin Core, which I'm familiar with anyway, than maintain an entire (guix) build system myself.