Test peer discovery with public DHT #96
Conversation
2color
commented
Sep 27, 2023
•
2color
commented
- js-peer: connect to public libp2p bootstrap nodes
- remove dht protocol prefix to use public dht
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎
|
| @@ -72,7 +74,7 @@ export async function startLibp2p() { | |||
| ignoreDuplicatePublishError: true, | |||
| }), | |||
| dht: kadDHT({ | |||
| protocolPrefix: "/universal-connectivity", | |||
There was a problem hiding this comment.
This would need to be changed in all clients if they should be compatible. I am not sure if we want to do this though.
Somebody coming to libp2p via this project would get the impression that they can just use the IPFS DHT for their project. I don't think we want that? Not until we change it to be a "generic peer routing" DHT I guess?
There was a problem hiding this comment.
I mostly agree. The key question I'm trying to answer with this PR, is how well ambient peer discovery and routing work for js peers in this app (in the browser) using the public bootstrappers and how that compares with the approach of a single rust peer, single go peer and multiple browser js peers.
I haven't found a definitive answer to this in any of the libp2p specs or the docs.
Intuitively, it makes sense to me that all peers in this app use a separate DHT. But if we assume that we mostly have 50 concurrent browser peers with no DHT server and a single rust node, how well would all peers be able to propagate their messages to the other peers?
This is clearly a question that this app should be able to answer. And the tradeoffs are important for any application built with libp2p targeting the web platform.
Somebody coming to libp2p via this project would get the impression that they can just use the IPFS DHT for their project.
This is default behaviour of the go kad-dht implementation, https:/libp2p/go-libp2p-kad-dht/blob/master/internal/config/config.go#L19-L20
I didn't see it explicitly encouraged or discouraged and it seems like it wouldn't be the default if that were the case
There was a problem hiding this comment.
Somebody coming to libp2p via this project would get the impression that they can just use the IPFS DHT for their project.
This is default behaviour of the go kad-dht implementation, libp2p/go-libp2p-kad-dht@
master/internal/config/config.go#L19-L20I didn't see it explicitly encouraged or discouraged and it seems like it wouldn't be the default if that were the case
It is also the default for rust-libp2p actually :)
Intuitively, it makes sense to me that all peers in this app use a separate DHT. But if we assume that we mostly have 50 concurrent browser peers with no DHT server and a single rust node, how well would all peers be able to propagate their messages to the other peers?
This is clearly a question that this app should be able to answer. And the tradeoffs are important for any application built with libp2p targeting the web platform.
100%! My thoughts are:
- The web-app MUST provide the ability to just paste a server-node address and connect to it.
- We SHOULD provide a node that users can connect to. Maybe not automatically but pre-fill it in the UI or make a separate button? What would be good to teach here is that by no means is this particular peer special.
- Server nodes SHOULD support the relay protocol to allow browsers to establish WebRTC connections to each other.
- Server nodes SHOULD form their own DHT, separate from IPFS
- Server nodes SHOULD support gossipsub
Imagine this:
- We both spin up our own bootstrap nodes in the cloud, completely oblivious that there is already one, i.e. we don't connect it directly to the provided one.
- We both start the web app and connect to our own bootstrap node by pasting the address but also connect to the provided bootstrap node.
- Now we can technically discover each other via gossipsub or something else.
- We use the relay to hole-punch a browser-to-browser WebRTC connection.
- It would be amazing if somehow, our two personal server nodes would learn about each other. Perhaps we need a simple "ambient-peer" request-response protocol that goes something like: "Send me all peers you are connected to"?
There was a problem hiding this comment.
- It would be amazing if somehow, our two personal server nodes would learn about each other. Perhaps we need a simple "ambient-peer" request-response protocol that goes something like: "Send me all peers you are connected to"?
If you tell me about your bootstrap node, I can connect to it. My bootstrap node could then ask me, who I am connected to and find out about your bootstrap node. They can connect to each other, discover that they are both server nodes and form a DHT.
There was a problem hiding this comment.
- Perhaps we need a simple "ambient-peer" request-response protocol that goes something like: "Send me all peers you are connected to"?
We would definitely need base this on peer records and not just plain addresses. It is still problematic because you could just make up a new identity, put a bunch of addresses in a peer record, distribute them and thus trigger dials to these addresses which is essentially an amplification attack :(
There was a problem hiding this comment.
a simple "ambient-peer" request-response protocol that goes something like: "Send me all peers you are connected to"?
That's funny, we were just talking about something like this at the js-colo. We used to have something similar in webrtc-star so there is prior art - the signalling server used for the initial SDP handshake also functioned as a peer discovery method since when you connected to it, it would send you a list of other peers connected to the server.
These are high-value peers for browser nodes since they are guaranteed to support the same transports as you.
The thought was, we could have a protocol that supports returning a list of peers you know about filtered by some sort of attribute, the initial one would be the types of multiaddr they are available on - since there's little use returning tcp/quick-only peers to browser nodes.
There was a problem hiding this comment.
Very interesting! Thanks for sharing :)
But, filtering or not, we'd have to figure out how we can trust the peer we just connected to with the data they've given us.
There was a problem hiding this comment.
I wrote up a proposal here: libp2p/specs#587
