Skip to content
Snippets Groups Projects

jeffro256 full-time dev 2023Q3

Merged jeffro256 full-time dev 2023Q3
jeffro256_/ccs-proposals:jeffro256-23Q3 into master
2 unresolved threads
Merged jeffro256 requested to merge jeffro256_/ccs-proposals:jeffro256-23Q3 into master
2 unresolved threads

What

I plan to work full-time to work towards accomplishing:

  1. Implementing Seraphis wallet code with Ukoe, jberman, et al.
  2. Drafting a formal specification for decoy selection and implementing it in a modular, robust, easily-testable way, and then creating a multitude of unit tests, integration tests, and statistical checks for this code.
  3. Reviewing other Monero core PRs, especially those related to Seraphis or @vtnerd's serialization changes.

To expand on point 2, I think the decoy selection code is in need of a makeover. In light of recent, high-impact decoy selection bugs (more info here and here), and whereas ring signatures are generally regarded as the weakest link in Monero's privacy model, I believe a new development initiative needs to take place to replace the decoy selection code. Typically I am against refactoring code "just because", but in this case it is warranted. All of the aforementioned bugs could have been spotted by statistical checks. However, currently, it is rather difficult to test the decoy selection in isolation due to the monolithic design of wallet2. In theory, decoy selection code could be written to be rather functional. To over-simplify, we take distribution of enotes on-chain usable for given for true spends, apply arbitrary picker distribution, then request information for selected decoys over RPC, retrying the process if enotes are blackballed or otherwise unsuitable. There are a lot of edge cases to consider, but there isn't any major reason why decoy selection can't be modularized and standarized apart from the rest of the wallet code. This goal also overlaps with point number 1, since this code can be used not only in wallet2, but in future versions of core wallets if written correctly.

Who

I have been contributing to the Monero core repository for over a year with a total of >=48 merged commits thus far. Some more notable contributions:

  • Implemented research issue 109 by authoring a PR that, when spending coinbase enotes, chooses coinbase enotes as decoys and vice versa.
  • Implemented a requested wallet transfer feature that enables "fee-included" transactions
  • Found and fixed an issue which nearly allowed a double spend bug, and could cause double spend bugs in the future if the code was not carefully inspected
  • Helped review patch and write disclosure report on recent decoy selection bug.

Previous Proposals:

Payment

I propose to work 40 hours/week for 3 months so 480 hours total. I'm setting my hourly rate at ~$43/hour, and at a price of $140/XMR, makes for a total of 147.4 XMR. The proposal is broken into 3 milestones, one for each "month". These milestones may lag behind schedule if I do not complete 40 hours per calendar week, but in that event, I will not ask for payout until the hours are done.

Edited by jeffro256

Merge request reports

Merged by luigi1111luigi1111 1 year ago (Jun 15, 2023 5:54pm UTC)

Loading

Activity

Filter activity
  • Approvals
  • Assignees & reviewers
  • Comments (from bots)
  • Comments (from users)
  • Commits & branches
  • Edits
  • Labels
  • Lock status
  • Mentions
  • Merge request status
  • Tracking
jeffro256-full-time-2023Q3.md 0 → 100644
37
38 ## Who
39
40 I have been contributing to the Monero core repository for [over a year](https://github.com/monero-project/monero/pulls?page=2&q=is%3Apr+author%3Ajeffro256) with a total of [>=48 merged commits](https://github.com/monero-project/monero/graphs/contributors?from=2022-01-25&to=2023-05-30&type=c) thus far. Some more notable contributions:
41
42 * Implemented [research issue 109](https://github.com/monero-project/research-lab/issues/109) by [authoring a PR](https://github.com/monero-project/monero/pull/8815) that, when spending coinbase enotes, chooses coinbase enotes as decoys and vice versa.
43 * [Implemented](https://github.com/monero-project/monero/pull/8861) a requested wallet transfer feature that enables "fee-included" transactions
44 * [Found and fixed](https://github.com/monero-project/monero/pull/8707) an issue which nearly allowed a double spend bug, and could cause double spend bugs in the future if the code was not carefully inspected
45 * Helped review [patch](https://github.com/monero-project/monero/pull/8794) and write [disclosure report](https://github.com/monero-project/monero/issues/8872) on recent decoy selection bug.
46
47 Previous Proposals:
48 - https://repo.getmonero.org/monero-project/ccs-proposals/-/merge_requests/319
49
50 ## Payment
51
52 I propose to work 40 hours/week for 4 months so 480 hours total. I'm setting my hourly rate at ~$43/hour, and at a price of $150/XMR, makes for a total of 137.6 XMR. The proposal is broken into 3 milestones, one for each "month". These milestones may lag behind schedule if I do not complete 40 hours per calendar week, but in that event, I will not ask for payout until the hours are done.
  • plowsoff
    plowsoff @plowsofff ·
    Contributor

    small typo

    Suggested change
    52 I propose to work 40 hours/week for 4 months so 480 hours total. I'm setting my hourly rate at ~$43/hour, and at a price of $150/XMR, makes for a total of 137.6 XMR. The proposal is broken into 3 milestones, one for each "month". These milestones may lag behind schedule if I do not complete 40 hours per calendar week, but in that event, I will not ask for payout until the hours are done.
    52 I propose to work 40 hours/week for 3 months so 480 hours total. I'm setting my hourly rate at ~$43/hour, and at a price of $150/XMR, makes for a total of 137.6 XMR. The proposal is broken into 3 milestones, one for each "month". These milestones may lag behind schedule if I do not complete 40 hours per calendar week, but in that event, I will not ask for payout until the hours are done.
  • jeffro256 changed this line in version 2 of the diff

    changed this line in version 2 of the diff

  • Please register or sign in to reply
  • Rucknium
    Rucknium @Rucknium ·
    Contributor
    1. Drafting a formal specification for decoy selection and implementing it in a modular, robust, easily-testable way, and then creating a multitude of unit tests, integration tests, and statistical checks for this code.

    Many research papers say that having a good decoy selection algorithm is essential for protecting Monero user privacy: (Mackenzie et al., 2015), (Kumar et al., 2017), (Möser et al., 2018), (Ye et al., 2020), and (Ronge et al., 2021). jeffro's proposed work can fix a few problems with what wallet2 is doing:

    1. Complicated, unclear code increases the risk of bugs. This is not a theoretical risk. In the last two years, three significant bugs have been found in the wallet2 decoy selection code. This recent off-by-one error that never selected decoys from exactly 10 blocks old, an inappropriate integer division issue in 2021 (found by jberman), and not applying the gamma distribution from chaintip (also found by jberman in 2021). Off-by-one error and inappropriate integer division are classic programming mistakes that may have been caught earlier if the code was easier to understand.

    2. IMHO, the correct flow for safety-critical code is to write a specification, write the code that implements the specification, and then check that the code correctly implements the specification. jeffro's proposed work will get us closer to that ideal.

    3. The confusing code makes it harder for researchers to analyze Monero to improve it. A recent paper, Chow et al. (2023), said, "Monero differs from our experiment mainly in two ways: 1) Their approximation on the source account distribution is some Ŝ_t different from our S_t, which is a code-induced distribution that cannot be expressed analytically...."

    4. EDIT: Another one: With a formal specification it would be easier for wallet software that does not use wallet2 to correctly implement the "standard" decoy selection algorithm.

    The Monero Project should take its statistics as as seriously as its cryptography. jeffro's proposed work is a step in the right direction.

    References:

    Chow, Egger, Lai, Ronge, & Woo (2023) "On Sustainable Ring-based Anonymous Systems"

    Kumar, A., Fischer, C., Tople, S., & Saxena, P. (2017). "A traceability analysis of Monero's blockchain."

    Mackenzie, A., Noether, S., & Team, M. C. (2015). "Improving obfuscation in the cryptonote protocol."

    Möser, M., Soska, K., Heilman, E., Lee, K., Heffan, H., Srivastava, S., Hogan, K., Hennessey, J., Miller, A., Narayanan, A., & Christin, N. (2018). "An empirical analysis of traceability in the Monero blockchain."

    Ronge, V., Egger, C., Lai, R. W. F., Schröder, D., & Yin, H. H. F. (2021). "Foundations of ring sampling."

    Ye, C., Ojukwu, C., Hsu, A., & Hu, R. (2020). "Alt-coin traceability."

    Edited by Rucknium
  • jeffro256 added 1 commit

    added 1 commit

    • 423bec2f - jeffro256 full-time dev 2023Q3

    Compare with previous version

  • jeffro256
    jeffro256 @jeffro256_ ·
    Author Contributor

    Fixed "4 month" typo and readjusted rate based on current price.

  • jeffro256 changed the description

    changed the description

  • luigi1111 mentioned in commit 5884b520

    mentioned in commit 5884b520

  • luigi1111 merged

    merged

  • plowsoff mentioned in merge request !385 (merged)

    mentioned in merge request !385 (merged)

  • jeffro256
    jeffro256 @jeffro256_ ·
    Author Contributor

    July 2023 Milestone Report

    Howdy, I'm here to report on the work I've done since my CCS proposal was merged, and to thank everyone who contributed!

    Progress towards "Implementing Seraphis wallet code with Ukoe, jberman, et al."

    The biggest chunk of code I worked on and tested can be found at this PR: "wallet2_basic: robust dep-free lib for loading/storing wallet2 files". This library can load/save wallet2 cache and keys files, with no dependency on wallet2 and a small binary footprint. This work will hopefully be extremely useful to the "No Wallet Left Behind" project as we work towards designing and implementing the shift from old wallets to new wallets. And since this library has no dependency on wallet2 (only dependency is libcryptonote_basic), hopefully the core project repository can eventually drop the current wallet2 code (> 15,000 lines), reducing technical debt. I did the work to integrate the library into wallet2 and was able to reduce the binary size of monero-wallet-cli by half a megabyte (with guidance from @vtnerd).

    Progress towards "Drafting a formal specification for decoy selection and implementing it in a modular, robust, easily-testable way..."

    Right now, my work on this has been almost entirely researching, documenting, and comparing current decoy selection implementations. I am also working with @Rucknium towards trying to formalize the behavior of the core project's decoy selection algorithm, with the end hope being that we can find a closed form expression for the distribution that the current code creates (this might be impossible, but that's the perfect outcome). So far I've been working towards this by simplifying (w/o changing the behavior outside of edges cases) the approx. 800 lines of C++ code involved in this process and breaking it up into smaller parts, of which those can be tested that their behavior matches expected mathematical properties.

    Progress towards "Reviewing other Monero core PRs, ..."

    Multisig changes

    K-anonymity PR

    Found here: "blockchain_db: add optional k-anonymity to txid fetching". This PR enables users to fetch transaction information from the daemon in an obfuscated manner, allowing users to protect their anonymity. @ACK-J is working on the application side of this feature and this PR is a requirement for that. This feature adds no overhead when not used, and is extremely efficient when it is used due to the methods of the txid template and the way keys are stored in LMDB.

    Other misc small changes

    Payout

    I kindly request to be paid out 49 XMR to the same address (8A1ua1oW2DJQZVr8h7UNRS1Y41Zx3UvDV5L9chiiYTgWULaXEHFzRThJdKgtG8yNAvLAAMCojW7dSW2GZULU6JHc4jrG6xs ) mentioned in my previous merged proposal.

  • jeffro256
    jeffro256 @jeffro256_ ·
    Author Contributor

    August 2023 Milestone Report

    Howdy, I'm here to report on the work I've done since my last milestone.

    Progress towards "Implementing Seraphis wallet code with Ukoe, jberman, et al."

    I read through the Jamtis paper and proposed a change to the balance recovery process which you can read about here. These changes would protect light wallet users from a few deterministic attacks and would optionally allow for light wallet servers to generate addresses on users' behalf without any additional loss of privacy. The downsides are 25% larger addresses and slower light wallet client scanning. You can read the linked gist thread to read more about the trade-offs and design. Later, I then implemented theses changes and created a PR against the core Seraphis library. Finally, I opened a PR for the Seraphis documentation if we choose to take this route.

    Progress towards "Drafting a formal specification for decoy selection and implementing it in a modular, robust, easily-testable way..."

    I have not made any progress towards this goal since the last milestone.

    Progress towards "Reviewing other Monero core PRs, ..."

    Other misc small changes

    Payout

    I kindly request to be paid out 49 XMR to the same address (8A1ua1oW2DJQZVr8h7UNRS1Y41Zx3UvDV5L9chiiYTgWULaXEHFzRThJdKgtG8yNAvLAAMCojW7dSW2GZULU6JHc4jrG6xs ) mentioned in my previous merged proposal.

  • jeffro256
    jeffro256 @jeffro256_ ·
    Author Contributor

    September/October 2023 Milestone Report

    Howdy, I'm here to report on the work I've done since my last milestone.

    Progress towards "Implementing Seraphis wallet code with Ukoe, jberman, et al."

    Since I proposed changes to protect light wallet users from a few deterministic attacks from their service providers last month, I have had a lot of discussions with tevador about how best to handle users' performance expectations. There are long columns of comments, which you can read here, on how to best incentivize and accommodate users so that they don't switch to alternative wallets with worse privacy. We ended up discussing flexible-size view tags, which I am in the process of implementing. While working on this, I discovered a privacy issue with how self-send view tags are computed when using a light wallet. Basically, the more self-send enotes you have in a single transaction, it is exponentially more likely that you own those enotes from the perspective of a light wallet server who has your find-received private key. I proposed a solution I call "auxiliary enote records", which would maybe-maybe-not increase scanning bandwidth requirements for light wallet users, but allows them to churn and do pocket-change without any additional privacy loss. You can watch the work in progress of combining flexible view tags + address tag protection + auxiliary enote records on this branch.

    I have also been working on more higher-level, specific Seraphis/Jamtis implementations details: like Jamtis base32, base32 checksums, and legacy wallet exporting.

    Progress towards "Drafting a formal specification for decoy selection and implementing it in a modular, robust, easily-testable way..."

    I wrote a document that describes each detail of the RingCT decoy selection process, which you can find here. I also wrote a modular and easily-readable Python reference script. I tested it against the core C++ tools::gamma_picker using a two-sample KS test and got a statistic of 9.9 x 10^-5, which means that the maximum difference between the two samples' empirical cumulative distributions is less than 0.01%, meaning that there is a decent chance that it is implemented correctly.

    While studying the decoy selection code, and with some insight from Rucknium, I found and fixed a minor issue with the current decoy selection implementation. Specifically, wallet2::get_outs was calling gamma_picker::pick more times than needed to build up a unique set of decoy picks, filtering out those that couldn't be used, then picking from those remaining in a uniform fashion. This meant that the decoy selection picks were more statistically dependent on each other within each ring than they needed to be. For more information, you can see the PR.

    Progress towards "Reviewing other Monero core PRs, ..."

    Other misc small changes

    Payout

    I kindly request to be paid out 49 XMR to the same address (8A1ua1oW2DJQZVr8h7UNRS1Y41Zx3UvDV5L9chiiYTgWULaXEHFzRThJdKgtG8yNAvLAAMCojW7dSW2GZULU6JHc4jrG6xs ) mentioned in my previous merged proposal.

  • jeffro256 mentioned in merge request !421 (merged)

    mentioned in merge request !421 (merged)

  • jeffro256 mentioned in merge request !436 (merged)

    mentioned in merge request !436 (merged)

  • jeffro256 mentioned in merge request !467 (merged)

    mentioned in merge request !467 (merged)

  • jeffro256 mentioned in merge request !504 (merged)

    mentioned in merge request !504 (merged)

  • jeffro256 mentioned in merge request !540 (merged)

    mentioned in merge request !540 (merged)

    • Please register or sign in to reply