Unnamed Monero Wallet development

changed the description

I was asked by @plowsof to put up a comparison of monero_c with @valldrac's monero-wallet-sdk

I took a look at monero-wallet-sdk - and it looks promising for what it is supposed to be, and neither monero_c or xmruw does exactly what monero-wallet-sdk promises to do.

I only took a look at the CCS proposal and "just touched" the code to see how it works, but kotlin is not really my land so instead of comparing implementation vs implementation I'll compare it implementation (xmruw, monero_c and all libraries) vs CCS Proposal, which will be fair because the monero-wallet-sdk library is still in WIP, and monero_c is done but alpha quality¹.

I'll go and compare all the points of Wallet SDK for Android CCS to current monero_c implementation.

Library

monero-wallet-sdk is made for android with only android in mind.

xmruw is made for all platforms (currently Linux, Android, SailfishOS, with Windows, MacOS and iOS being easily portable - with a matter of adjusting few lines of makefile²) (including exotic ones like SailfishOS which never had any monero/cryptocurrency wallet. It took about one day to port the wallet itself after we got flutter working.).

monero-wallet-sdk: A specialized library that allows developers to interact with the Monero network and perform wallet operations. This library is written in Kotlin and offers a reliable asynchronous API.

xmruw: Main goal of the anonero rewrite was to get rid of asynchronous API and to move from JNI/kotlin/java abstractions closer to monero code.

I can see why it is beneficial for project like molly messenger to use something a bit higher level than wallet2_api.h, monero-wallet-sdk will "just work" no matter where you put it (in any android app), and will offer better UX out of the box. It will take care of everything instead of telling user to take care about the things on his own, while on the other hand xmruw approach is to make everything much simpler by avoiding abstraction like this (reasoning explained here: https://www.mrcyjanek.net/p/xmruw-monero-wallet/#cpp-java-kotlin-dart-async-state-management-and-debugging )

Also worth noting is the fact that monero_c is as low level as wallet2_api.h - which means that user can do things like ignoring errors reported by status() functions or pass invalid pointers around. I've decided to not do anything about that issue because it is how the upstream code works. Most of the problems that may occur because of passing incorrect pointers or doing some incorrect string conversions are fixed by the autogenerated monero.dart code (or by some additions I made to make it easier to use), but still the underlying libraries are as bulletproof as upstream api, while monero-wallet-sdk seems to be higher level and account for issues like this that could occur.

1.1 Language choice

monero-wallet-sdk: went with the obvious choice - kotlin which is the native language of android.

xmruw: "that's simple but complicated", since monero-project/monero offers c++ api and I can consume C api from my language of choice - which is Dart, so monero_c was written in c++ and provide C header, which can be used by almost every single language out there, including dart, go and in an experimental way in kotlin. Then to use the library I've made (mostly autogenerated) monero.dart to get suggestions as I code to not make any typos that would cause crashes - so now the app will either not compile or work as it should. ³

1.2 Asynchronous API

monero-wallet-sdk: async API

xmruw: synchronous (if user want to make something async, then user can make it async - however everything is synchronous by default)

1.3 Sandboxed Execution:

monero-wallet-sdk: All non-memory-safe code (C++) runs within an isolated process with zero privileges and restricted access to the host app or system resources. Even if a remote execution exploit affects Monero, it becomes extremely difficult for an attacker to elevate privileges to the system or host app.

xmruw: relies on OS for sandboxing

android - app sandbox - default on, every app has different UID, so if RCE would occur only files made by the app itself can be accessed.

linux - currently nothing (with the intention to make flatpak package which contains sandboxing and to release tarball with firejail configured)

SailfishOS - currently nothing (with the intention to enable Sailjail sandbox)

so no extra sandboxing, but OS itself is left with the exercise to run programs securely.

1.4 Architecture

monero-wallet-sdk: The library is built as an Android service, using Android's AIDL interprocess communication (IPC) to connect the Kotlin layer with the sandboxed native code. This client-server architecture decouples the internal wallet2 implementation from the public API of the SDK, facilitating future swapping of the wallet2 module without major changes to the API. This enables smoother migration to Seraphis for developers using our SDK.

xmruw: by current architecture monero_c doesn't do any abstraction, it offers raw access to the c++ api with a little wrappers to make it c compatible. When wallet3 reaches final shape I'll do my best to not break the already provided api functions, or when breaking them is unavoidable I'll make sure to provide a guide on how to fix them. This is also by design as higher abstraction makes cutting edge development harder, and xmruw from the beginning focused on having features not yet merged into upstream. Also communication is by C headers, so one just load a dynamic library (.so, .dylib, .dll or anything else) and use it.

1.5 Storage Abstraction

monero-wallet-sdk: The library provides a storage abstraction layer, simplifying wallet persistence for developers. It allows the SDK to be agnostic about the underlying storage mechanism (files, database, cloud, etc.). This flexibility enables developers to improve data-at-rest encryption without modifying the SDK code.

xmruw: nuh-uh. (with the intention to explore obfuscated+encrypted on device file store for libstealth but I honest don't even have a good idea how to do that without root access and without patching upstream code (which I reall want to avoid), on desktop platforms it is easily doable with libraries like fuse - but again, nothing in xmruw stack would need to know about the existence of such a extra step involved).

1.6 Network:

monero-wallet-sdk: Network Client Injection: Instead of relying on the networking code of wallet2, the SDK allows the app to provide its own HTTP client. This gives the app the freedom to support any transport protocol, such as Tor, transparently to the SDK.

xmruw: No extra code (like network client) is added, all proxying is made entirely by setting proper arguments in init() function in wallet2_api.h. As a general rule of thumb I avoid writing any critical code if I don't have to, and network layer / encryption are very critical, to avoid issues caused by implementing things that I don't fully understand.

1.7 Logging

monero-wallet-sdk: Custom Logging: The library includes a logging adapter that developers can customize. It enables the host app to determine log storage and location, providing a way to encrypt or remove sensitive information from the logs if needed.

xmruw: monero build-in logging, stdout or to file (with log viewer in app)

1.8 Client-side Load Balancing

monero-wallet-sdk: The library enables the host app to dynamically select the optimal remote node for RPC calls, ensuring efficient synchronization with the Monero blockchain. This can significantly reduce synchronization time by choosing the fastest node.

xmruw: nuh-uh: this is left for the user, the default nodes are quite fast regardless of location and work over tor. (with the intention to sort nodes by their speed in nodes list in future)

1.9 Developer Experience

monero-wallet-sdk: Android Studio Compatibility: The SDK is fully compatible with Android Studio, supporting native debugging, code navigation, and linter capabilities, for both the SDK library and the Monero codebase.

xmruw: Also one of main goals, full LSP support, code generation for monero.dart - it greatly speeds up the development and makes issues less common.

1.10 Build System

monero-wallet-sdk: Optimized Build System: The library has optimized CMake files that vendor all dependencies of wallet2, and applies Link-time optimizations (LTO) to remove all C++ code that is never called by the SDK. The result is a reproducible and lightweight library size of only 6 MB per arch, that can be built directly from Gradle.

xmruw: nah. Resulting binaries are not stipped out of anything, they are also statically linked and also vendor all library versions used by monero upstream. It is however easily possible to reduce the .so size by using upx and strip. Here is a size comparision of what the library size can be:

6.8M x86_64-linux-gnu_libwallet2_api_c.so.xz <- distributed, made with xz -e

29 M x86_64-linux-gnu_libwallet2_api_c.so <- usable "raw"

22 M x86_64-linux-gnu_libwallet2_api_c.strip.so <- usable in app

11 M x86_64-linux-gnu_libwallet2_api_c.strip.upx.so <- usable in app (cold boot delay)

13 M x86_64-linux-gnu_libwallet2_api_c.upx.so <- usable (cold boot delay) (with debugging symbols) so best-case-scenario is 2 times larger than monero-wallet-sdk.

Library is also built using one (large but simple) makefile and a .env that configures hashes and dependency versions. make host_depends moneroc_linux_amd64 moneroc_linux_host64 is enough to get .so, release builds are dockerized to ensure reproducibility (I still didn't check if the builds are in fact reproducible, this is something I'll take look in the future). ⁴

Sample Wallet App

2.1 Architecture

monero-wallet-sdk: Clean Architecture - The sample app uses the latest Android libraries and follows the official architecture guidance.

xmruw: Breaking status quo by calling wallet2_api.h functions directly from the UI thread, whenever needed separate isolate (dart word for thread) is spawned to not block the UI. This causes some issues for the UX (app may freeze) but it makes mutexes unnecessary (synchronous by default) and protects against accidental crashes by the blocking design. It doesn't follow official architecture guidance for android, and it doesn't really follow any guidance at all.. But it follows a rule of thumb of mine to make everything as portable and as close to the underlying library as possible. ⁵

Note, and a personal opinion: If I had fully finished monero-wallet-sdk and xmruw (monero_c + monero.dart) and I wanted to implement a monero wallet as secondary feature (addition to messenger app, or a multi currency wallet) I would most likely go for monero-wallet-sdk because of simplicity and promise of not breaking the api, however if I were to develop a monero-only wallet or I would need more advanced features or I would want to work on a forked version of monero then I would go for xmruw implementation.

2.2 Tools

monero-wallet-sdk: Kotlin and Jetpack Compose: The sample app is entirely written in Kotlin and uses Jetpack Compose for the UI, as well as many architecture components like Room, Lifecycle, and Navigation.

xmruw: C headers, with libraries consuming them written in Dart (with the intention of making Go and other languages version too, and who knows - maybe even rust? We could then test cuprate against monero implementation directly from cargo test), and a wallet written in flutter. Nothing extra except for int, char * was used to the code should be pretty portable, regardless of the system architecture it runs on.

Sample code

monero-wallet-sdk: https://gist.github.com/valldrac/54ff6842e4cc6d624ab893f4e839b2ae

xmruw (monero_c + monero.dart): https://pastebin.com/CLfMghK6 ⁶

TL;DR: Different goals by both projects

monero-wallet-sdk focuses on android while xmruw focuses on being cross platform

monero-wallet-sdk focuses on stable api for developers while xmruw focuses on offering as small abstraction as possible

both projects aim for good developer experience (things like IDE suggestions)

I've corrected few spelling mistakes and adjusted syntax so it looks nicer, and added comments below to some sections, the quoted thing above is copy-pasted from a paste I've send some time ago when I was asked for comparison.

"is done" in this context means that it is in fact usable but it lacks some features, and it has a few elephants in the room that I pretend do not exist.
it's a bash script now, and it uses monero's contrib/depends to kickstart the compilation.
I can't express how important things like that are, back when I was using JNI + PlatformChannels I didn't had this kind of suggestions from LSP (things like misspelling "transaction" with "transaciton", in a >100 lines long implementation is truly painful to debug as the error is occurring at runtime instead of compile time).
Not the case anymore. I've decided to migrate all my custom code to just be a wrapper around contrib/depends... This is not yet finished approach, but everything seems to be much simpler (there are some issues with that code in the upstream also, and once I figure them out I'll open a PR).
Real bug description: when user clicked "open wallet" button multiple times, that was responsible for unlocking the wallet and initializing it, it would crash the app because the wallet would get opened two (or more) times, which would cause wallet file corruption and wallet crash. This can be easily prevented by disabling the open wallet button once it was clicked, but it involves extra thinking and extra code on the side of developer. While on the other side by using synchronous api the UI would freeze for the time when the wallet was opening, so spamming it would effectively do nothing, and the issue would get noticed instantly - as the wallet is unresponsive while opening (to fix that developer can move the open function to separate thread, but this time the developer is aware that this function takes time to execute and that it may be not so wise to allow calling it multiple times). So the synchronous design prevents some issues from happening while creating others, it prevents you from shooting yourself in the foot, but it offers a aim assist when trying to shoot your leg.
This was supposed to be 1:1 demo but monero-wallet-sdk is demonstrating it's built in features, that are not built into monero_c/monero.dart, so it does "something similar", but it is not 1:1 the same thing.

There will be a community meeting this Saturday 16th 15:00UTC to obtain more feedback for this proposal https://github.com/monero-project/meta/issues/979

changed the description

A community meeting tomorrow 30th March 15:00UTC will take place (albeit with major bridge issues) https://github.com/monero-project/meta/issues/984

mentioned in merge request !445 (closed)

changed the description

added 1 commit

4620a249 - Update hourly rate

Compare with previous version

@J0J0XMR Thanks, the goals do not contain "implement monero.dart in cake / stack", so there is not much in common

the CCS is about developing the libraries.
the cake/stack deal is about using said libraries

Also as an extra bonus the libraries are undergoing a security audit, some bugs are fixed along the way and the library is being used so it is becoming battle-tested and stable.

To address the concerns - CCS money won't be spent on me working on cake/stack, it will be used to work on the underlying libraries and reference implementation, but cake/stack work will be used to make the libraries better - as can be seen currently in

traced origins to patches
documented behavior
made initial work with support for iOS
sponsored development hardware
and many minor fixes that were not caught by me earlier

Which wasn't exactly in the plans of the CCS but is surerly something nice to have :), something that will be beneficial to all devs who adopt monero_c/monero.dart or just the patches.

Once merged I'll publish weekly status updates so my work can be verified, and any concerns can be resolved as we go.

EDIT: it appears that J0J0 removed the comment.

This ccs brings benefit the entire ecosystem, and unlike sdk.

Easy merge.

Milestones with goals to achieve are preferred

changed the description

added 1 commit

555f4d95 - Update the milestones

Compare with previous version

it should be clear now

incredibly vague, under the umbrella term of a "wallet". i agree with plowsof that its not clear what exactly is planned on being delivered.

then go ahead and elaborate on how can one develop libraries, test features without a PoC. I'll wait.

woodser seems to have done it in the libs i mentioned

valldrac's includes one. Sure, and tell me how would I be able to catch bugs in the polyseed patch if there wouldn't be a wallet which allowed me to quickly test that feature. Xmruw is anyway just a fraction of the work.

valldrac's wallet is the most bare bones demo possible, and is just a footnote to the development of the sdk itself. and there are 2/3 wallets that you couldve used to catch the bug: ANONERO and stack (and cake, since its integrating your lib), which all used the same patch. so again, xmruw not needed

I would slightly disagree, there were plenty of UI/UX things that I touched, not much - fair, but clearly not 0.

you did not do the UI/UX design on ANONERO, anything you did was minor touches at my direction. do not try an claim credit for it, part of the reason i didn't go with your wallet is because you couldn't do a 1:1 rewrite, with UI/UX being a huge point of contention

And from my experience working with you I could also say plenty of stuff.. but I'm positive that I can achieve all of the points mentioned in the milestones. Sure UI/UX is not my field but I can place elements on screen and make them work. You can already verify those claims in xmruw group, the app is adjusted to work on small screens and with large fonts - which is clearly a UX benefit.

UI != UX and this answer shows that you dont understand that

....

this is as far as i want to take this convo. im sticking to my feedback of splitting this ccs into just the monero_c/dart libs and a separate ccs for xmruw once you have a more concrete plan in place, including UX

You are missing one significant point, the app is already working, and is made in a way in which it makes it easy for me to work on it.

is the xmruw the ccs or not? make up your mind..

Since ANONERO is such a good option for catching bugs why are you even planning on rewriting it? Also stack and cake integration came after xmruw, which was the first wallet to use monero_c... anyway.

ANONERO was written as an experiment to build out features that have never been done before, knowing full well that a rewrite would have to happen for Seraphis. and yes, you can see how good it was at catching bugs in my CCS proposal

So I'd be happy if you could tell me what is wrong with the proposal currently and what do you expect me to do.

already have, multiple times. plowsof called your milestones "moving targets" and i agree

is the xmruw the ccs or not? make up your mind..

Xmruw is a POC of the work i do, and the ccs is to continue to improve the cross platform wallet ecosystem, which obviously includes my own wallet (that's why everything is under the umbrella term of the Unnamed Monero Wallet development)

already have, multiple times. plowsof called your milestones "moving targets" and i agree

ack

From information I received in private I have some reason to fear that my name and my purported opinion about this CCS proposal were used, or maybe abused, to oppose this proposal. That's why I leave the following clarification here, to avoid misunderstandings:

I am neither for, nor against this proposal. I see things on the positive side, and things on the negative side that for me right now more or less cancel out and result in a neutral stance.

It's none of my business why there are controversies about this proposal that are fought out here, and most probably also in one-to-one communications, and I will try to resist attempts to draw me into this further. That would not be productive at all.

Thank you.

Vik and Diego should make their sponsoring of the libs and dealings with mrcyjanek clear before a decision is made here. strange that neither of them have commented on a proposal that directly benefits them

Worry not. I am piping in.

Disclaimer: Stack is directly working with Czarek to integrate his library into Stack Wallet. Stack and Cake are co-sponsoring an audit to the libraries in question, performed by Cypher Stack.

I believe Czarek's work benefits the entire space. It has been low-key implied that if his work is going to benefit for-profit companies, such as Stack and Cake, then those companies should be the ones paying him to do things.

We are paying him for the work involving our wallets. And to make optimizations, updates, and further work on his lib that would make them better besides, not to mention the audit. We are doing our fair share here.
His work will not just help us. It's something that will enable other wallets, both old and new, to potentially add Monero in an easy, secure, audited way.

According to the people involved, there was an amicable split within Anonero. Both sides want to get the opportunity to show their stuff regarding "going solo". The Anonero CCS was merged. I see no reason why we should deny the other party the right to give it his best go as well regarding making a wallet.

Cake and Stack may not fully fit his vision because, at the end of the day, we are for-profit companies and have to make decisions about feature inclusion as well as UX decisions that a self-made, grassroots project does not. And if the argument is that one already exists with Anonero, well, see above for my reasoning. There doesn't need to just be one.

For what it's worth Czarek has been great to work with. He is clearly talented, passionate about privacy, and wants to improve Monero.

I am FOR this proposal!

added 1 commit

2a395540 - Update mrcyjanek-xmruw-development.md

Compare with previous version

Unnamed Monero Wallet development

About

Unnamed Monero Wallet (xmruw)

Scope of work

Milestones

Payment

Activity

Unnamed Monero Wallet development

About

Unnamed Monero Wallet (xmruw)

Scope of work

Milestones

Payment

Merge request reports

Activity