j-berman full-time 3 months (part 6)

What

Work full-time 3 months on:

• Continuing Seraphis wallet library work.
  • Complete the async wallet scanner implementation using the Seraphis wallet lib.
    • When I pointed this async scanner on my local machine to a remote monerod instance @selsta spun up, I observed a ~40% speed-up in scanning over the current wallet2 scanner.
    • Link to the draft PR that details all remaining TODO's in the PR description: https://github.com/UkoeHB/monero/pull/23
    • This CCS proposal will not be complete until I complete all major TODO's from the PR linked above, and deliver a final version of the async scanner PR for review.
  • Once I complete the async scanner, I plan to collaborate further with @jeffro256 @rbrunner7 @dangerousfreedom and the no-wallet-left-behind working group to work on whatever is next highest priority to get the Seraphis wallet lib ready for production.
• Continue work integrating full chain membership proofs into the Seraphis library (and therefore demonstrate how to integrate them into the core Monero repo). The goal is to construct and verify Seraphis transactions using full chain membership proofs, including a suite of unit tests and benchmarks.
  • In my last CCS, I got an initial setup working to call full chain membership proof code written in Rust from a Monero repo C++ performance test. This was step 1-of-many toward the goal of constructing a full chain membership proof in a transaction using the Seraphis wallet lib.
• Miscellaneous Monero work, debug issues, review PR's.
  • Highest on my list right now is to review @vtnerd's p2p encryption implementation (source: https://github.com/monero-project/monero/pull/8996).

Who

j-berman on github / jberman on matrix

Past CCS's:

• https://ccs.getmonero.org/proposals/j-berman-3months-full-time-5.html
• https://ccs.getmonero.org/proposals/j-berman-3months-full-time-4.html
• https://ccs.getmonero.org/proposals/j-berman-3months-full-time-3.html
• https://ccs.getmonero.org/proposals/j-berman-3months-full-time-2.html
• https://ccs.getmonero.org/proposals/j-berman-3-months-full-time.html

Proposal

211 XMR

480 hours, 0.16 XMR/hr + $48/hr, $172/XMR from coingecko

mentioned in commit 571cc106

merged

Update 1

188 hours

Full chain membership proofs

• I set up the scaffolding for a mock Seraphis tx that uses full chain membership proofs.
  • Source: https://github.com/j-berman/monero/commits/fcmp
  • Some significant unit tests are set up and compile (more changes are needed to execute end-to-end correctly, my tests currently hang):
    • Test mock txs of various construction: ./unit_tests --gtest_filter="seraphis_tx.seraphis_squashed_fcmp"
    • Test the full flow of send and receive: ./unit_tests --gtest_filter="seraphis_integration.txtype_squashed_v2"
    • Perf test to prove and verify fcmp's: ./performance_tests --filter "test_curve_trees_fcmp"
• The Seraphis lib is relying on a relatively simple API to the Rust full chain membership proof lib.
  • Source: https://github.com/j-berman/full-chain-membership-proofs/commits/cpp-compat-ed25519/
  • fcmp's in that branch are implemented using an ed25519 > bulletproof25519 tower, which I figured would help make it clearer what changes would be necessary to construct and verify Seraphis txs with full chain membership proofs using the Seraphis lib (minus lower level curve changes).
  • I did not implement critical sections of the ed25519 > bulletproof tower ciphersuite (e.g. some critical functions return 1 instead of the correct result of a crypto op). Since using the ed25519 > bulletproof25519 tower isn't the end goal and it's strictly meant to demonstrate the scaffolding of a Seraphis tx with fcmp's, I don't plan to implement those changes.
• Next I plan to implement my major TODO's in the Seraphis lib code, make that code cleaner, then solicit feedback before moving on to implementing using a new curve in the Seraphis lib (to support a curve cycle and stop using my ed25519 > bulletproof25519 implementation) and getting the above tests to pass.

Async wallet scanner

Nothing to report in this update. I plan to work on completing my draft PR as my next task (starting today), then move back to fcmp work.

Misc.

• Reviewing @jeffro256's PR to only write txs to the db once instead of twice on blockchain sync: https://github.com/monero-project/monero/pull/9135
• Implemented comments on the background sync PR: https://github.com/monero-project/monero/pull/8619
• Next review is p2p encryption.

Update 2

368 hours

Async wallet scanner

• Completed the PR, it's now ready for review.
• Source: https://github.com/UkoeHB/monero/pull/23
• Added functional tests to test Seraphis wallet lib <> live RPC daemon interactions.
• Cleaned up the code and PR.
  • 10/13 commits are theoretically independent and can be submitted as bite-sized PR's.
  • 6 of those commits can be submitted to the Monero repo today.
• Implemented backwards compatibility with current daemons.
  • Implemented comparable perf to wallet2 when using a current daemon (i.e. a daemon not running any of the updates from the PR).
• Download progress is smooth.
• Going to finalize benchamrks next.

Full chain membership proofs

• Nothing to report from this update.
• Considering the latest in FCMP's (@kayabaNerve's recent proposal to replace CLSAG with FCMP's before Seraphis), I will probably hold off on continuing FCMP integration with Seraphis in this CCS.
  • I may begin working on the tree in C++ next which would be useful for the proposed FCMP's both before and after Seraphis.
  • @kayabaNerve's proposal to replace CLSAGS with FCMP's pre-Seraphis: https://gist.github.com/kayabaNerve/0e1f7719e5797c826b87249f21ab6f86
  • Source issue to rewrite the tree in C++: https://github.com/kayabaNerve/full-chain-membership-proofs/issues/63

Misc.

• Finished reviewing @jeffro256's PR to only write txs to the db once instead of twice on blockchain sync: https://github.com/monero-project/monero/pull/9135
• Started reviewing @vtnerd 's p2p encryption.

Update 3

>480 hours

Async wallet scanner

• Source: https://github.com/UkoeHB/monero/pull/23
• Re-did benchmarks using one of @gingeropolous 's machines for clearnet scanning. Got the following results:

Setup	% faster than wallet2
Remote node, clearnet	50-60% faster
Remote node, tor	40-45% faster
Local node	25-35% faster

• Implemented @koe 's comments on the PR.

Full chain membership proofs

• No code written since last update.
• A solid portion of my work on FCMP's will be reusable to move forwards integrating @kayabaNerve's FCMP++ (source)
• Reviewed and provided feedback on @kayabaNerve's FCMP integration proposal.
• I'm planning to work on the FCMP++ integration in another CCS.

Seraphis wallet lib

• Proposed a plan to deprecate wallet2 and replace it with the Seraphis lib, reusing the current wallet API (source)
• Started working on a tool to import the Seraphis wallet lib enote store into a wallet2 instance (source)
  • This would be useful for 2 reasons:
    1. to use the Seraphis lib to sync, but construct legacy txs using wallet2 in a "box" (source)
    2. to enable replacing just the scanner in the wallet API as an interim step to deprecating wallet2 (source)
  • Identified data the scanner would need to pick up in order to maintain complete compatibility with wallet2 (source)

Misc.

• Finished re-reviewing and approved @jeffro256's PR to only write txs to the db once instead of twice on blockchain sync (source)
• Started initial review on @vtnerd 's PR to add p2p encryption (source)
• Reviewed @SNeedlewoods ' PR to identify non-standard received legacy enotes (source)
• Initial review on @jeffro256's PR to overhaul legacy indexing in the Seraphis lib (source)
• Reviewed @tevador's PR to support decoding a polyseed with an explicit language choice (source)
  • Solves an issue where a polyseed can decode as a different language (source)