Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Nesting DHTs #626

Draft
wants to merge 4 commits into
base: master
Choose a base branch
from
Draft

Nesting DHTs #626

wants to merge 4 commits into from

Conversation

aschmahmann
Copy link
Contributor

@aschmahmann aschmahmann commented Apr 30, 2020

This is an attempt at #616.

The basic idea is to utilize DHTs where it is assumed that DHT A is a subset of DHT B and therefore be able to start a query in DHT A and finish it in DHT B without needing to restart the query.

While the bare bones of doing this for GetClosestPeers is quite straightforward, unfortunately it's a bit of a pain to implement this for all of the Router interface functions (e.g. FindProviders, PutValue, etc.). Very much a WIP right now, that would obviously be much easier if all the routing interfaces where taken care of by #584.

dht_net.go Outdated
@@ -268,6 +270,14 @@ func (dht *IpfsDHT) sendMessage(ctx context.Context, p peer.ID, pmes *pb.Message
return nil
}

func (dht *IpfsDHT) getHost() host.Host {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not needed? has the same visibility as the host property. this is also a weird file for it; would expect generic accessor in dht.go if needed.

dht_net.go Outdated
@@ -300,12 +310,17 @@ func (dht *IpfsDHT) messageSenderForPeer(ctx context.Context, p peer.ID) (*messa
return ms, nil
}

type protocolSender interface {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this only used for the NewStream construction below? if so, might be easier to set the interface to a function returning a stream given a peerID?

nesting.go Outdated
@@ -0,0 +1,175 @@
package dht
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can this be in a sub-package like Dual?

@aschmahmann
Copy link
Contributor Author

aschmahmann commented May 1, 2020

@willscott if you rewind a commit everything looks much cleaner (it was in a separate package and all the grossness related to the transfer protocol disappears). The recent grossness was added in order to be able to run as a server in the inner DHT, but be a client on the outer DHT.

After talking with @Stebalien earlier today we decided that it was reasonable to insist that any inner DHT server would also be an outer DHT server.

I'll ping you when the next draft is done, it'll look much better.

@aarshkshah1992
Copy link
Contributor

@aschmahmann Which issue is this for ?

@aschmahmann
Copy link
Contributor Author

@aschmahmann Which issue is this for ?

@aarshkshah1992 I filled in some text above, it's for #616

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants