Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update proofs #97

Merged
merged 29 commits into from
Aug 18, 2023
Merged

Update proofs #97

merged 29 commits into from
Aug 18, 2023

Conversation

theochap
Copy link
Contributor

@theochap theochap commented Jun 21, 2023

Description

This PR allows the generation of merkle proofs in case of tree updates.
This PR is finally ready for review.

Purpose

This PR aims to solve the issue #246 of the Sovereign lab sdk repo

Methods added

The verify_update method have been added to the SparseMerkleProof, this method verifies a Merkle proof for a tree update.
The put_value_set(s)_with_proof methods have been added to the JellyfishMerkleTree struct. These methods update the tree and produce a set of Merkle proofs associated with the tree.

Changes from existing implementation

  • The get_child_with_siblings function now takes an object that implements the tree reader trait as an argument. This is due to deletion proofs that need to specify the sibling nodes' types (to know if we need to perform coalescing or not).
  • The TreeCache wrapper object now implements the TreeReader trait. We need that to correctly build the sibling list in the put_value_set(s)_with_proof methods.
  • I renamed get_child_with_siblings to get_only_child_with_siblings: this function did not have the same behavior as the child method of an internal node: when the nibble parameter points to an empty spot, get_child_with_siblings used to return the closest only child to that spot, while the child method would return none. Hence, I created a get_child_with_siblings reproducing the behavior of child and renamed the former method to get_only_child_with_siblings.

Testing

A file update_proofs.rs have been added to the test set. This file reproduces most the tests from jellyfish_merkle.rs and checks that the updates have been well performed. This file contains proptests.

Actions remaining

  • The last test of the file node_tests.rs in the testing folder has been broken because of the new interface for get_child_with_siblings. This isn't a straightforward fix as one needs to know the list of leaf nodes to be able to generate a merkle tree reader and populate it with the tree data. This test would need to be either rewritten or removed.
  • Adapt the proof for jellyfish updates to the batch api.
  • Improve the proof generation by grouping intermediate hashes.

Once this PR is merged, I will open new issues for the remaining actions

@theochap theochap marked this pull request as ready for review June 24, 2023 03:35
Copy link
Collaborator

@preston-evans98 preston-evans98 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good overall, but I think there's some things we can do to improve readability. I'd also love to see some more test coverage (especially on the complicated case where we have to split a leaf). For that case in particular, it's probably worth writing tests to ensure that we're always computing the correct number of default siblings, plus some end-to-end tests making sure that the insertion runs correctly

src/tests/jellyfish_merkle.rs Outdated Show resolved Hide resolved
src/tests/jellyfish_merkle.rs Outdated Show resolved Hide resolved
// Freezes the current cache to make all contents in the current cache immutable.
tree_cache.freeze()?;
}

Ok(tree_cache.into())
}

/// Same as [`put_value_sets`], this method returns a Merkle proof for every update of the Merkle tree.
/// The proofs can be verified using the [`verify_update`] method, which requires the old `root_hash`, the `merkle_proof` and the new `root_hash`
pub fn put_value_sets_with_proof(
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This function looks like it duplicates a lot of logic from put_value_sets. Can we extract the shared code into a helper function?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know if it would be clearer to be honest, specially because we do populate values of a proof vector on each iteration of the nested loops in the proof generating version

src/types/proof/definition.rs Outdated Show resolved Hide resolved
src/types/proof/definition.rs Outdated Show resolved Hide resolved
src/tree.rs Outdated Show resolved Hide resolved
src/tree.rs Outdated Show resolved Hide resolved
src/types/proof/definition.rs Outdated Show resolved Hide resolved
src/types/proof/definition.rs Outdated Show resolved Hide resolved
src/types/proof/definition.rs Outdated Show resolved Hide resolved
src/node_type.rs Outdated Show resolved Hide resolved
/// siblings, but more efficient).
pub fn get_child_without_siblings(&self, node_key: &NodeKey, n: Nibble) -> Option<NodeKey> {
pub fn get_only_child_without_siblings(
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm curious why we renamed this function - does it break if there are multiple children?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So this one is a bit tricky, but basically there is two different type of get_child functions:

  • The get_child* functions which basically returns the element pointed by n. If there is no child at that exact nibble number it will return None
  • The get_only_child* functions which does the same as the previous one, but will return the closest child to the spot pointed by n if this spot is empty (needed for non-existence proofs)

src/node_type.rs Outdated Show resolved Hide resolved
src/node_type.rs Outdated Show resolved Hide resolved
src/node_type.rs Show resolved Hide resolved
src/tests/helper_update_with_proof.rs Outdated Show resolved Hide resolved
src/tests/node_type.rs Outdated Show resolved Hide resolved
src/tests/node_type.rs Outdated Show resolved Hide resolved
src/tests/update_proof.rs Outdated Show resolved Hide resolved
src/tests/update_proof.rs Outdated Show resolved Hide resolved
Copy link
Collaborator

@preston-evans98 preston-evans98 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, except for a few small nits. Let's see if we can get a review from @plaidfinch before merging though.

src/node_type.rs Outdated Show resolved Hide resolved
src/node_type.rs Outdated Show resolved Hide resolved
src/tests/helper.rs Outdated Show resolved Hide resolved
src/tests/update_proof.rs Outdated Show resolved Hide resolved
let num_default_siblings = (4 - (num_siblings % 4)) % 4 /* the number of default leaves we need to add to the previous root */
+ ((4*(common_prefix_len+1) - num_siblings) / 4 ) * 4 /* the number of default leaves we need to add to the path */
+ (default_siblings_leaf_nibble)
- 4;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tagging this for future reference, since this simplifies to:

let num_default_siblings = (-num_siblings % 4) + (4*common_prefix_len) - num_siblings + default_siblings_leaf_nibble

If the unreduced expression is here to help readers grok the intuition behind it, then I would add at least a few lines of context about how you're getting there.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just added the detailed comments in the last commit!

@erwanor
Copy link
Member

erwanor commented Aug 18, 2023

Thanks for your contribution, this is really good work. We will cut a release later today

@erwanor erwanor merged commit ef4f23f into penumbra-zone:main Aug 18, 2023
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants