Monero Development Harness

[dev-warrior777]

Monero Development Harness

closes #2753

Monero development Harness - documentation and notes

Useful Info

Monero is very different than btc in that it does not provide a single rpc tool like bitcoin-cli but rather a set of json 2.0 & other older json apis which can be accessed by sending curl requests for each one.

Monero wallets are accounts based

https://web.getmonero.org/resources/moneropedia/account.html

Architecture

Embedded in Tmux

• alpha is a monero p2p daemon

• bill is a miner wallet .. you can top up more funds from bill if you run out

• fred is a normal wallet user

• charlie is a normal wallet user

• charlie_view is a view-only wallet sibling of charlie full wallet - no spend key

Using

Prerequisites

• monero-x86_64-linux-gnu-v0.18.3.3 or later https://downloads.getmonero.org/cli/linux64
• linux (tested on Ubuntu 22.04)
• jq (1.6)

Setup

monero-x86_64-linux-gnu-v0.18.3.3 should be in PATH

export PATH=$PATH:[path-to]/monero-x86_64-linux-gnu-v0.18.3.3

Background Mining

By default background mining is set up and mines to bill wallet every 15s

To disable:

export NOMINER="1" to your shell

or else invoke the harness with:

NOMINER="1" ./harness.sh

You should disable if attempting the manual offline cold signing using monero-wallet-cli experimental

You should also Ctl-C charlie & charlie_view wallets for this

Run

./harness.sh

Data Directory

Known Issues

The transaction spend locking needs more investigation - Fixed

Commands Help

run ./help from the tmux harness window 0

Commands Help:
--------------
alpha_get_transactions

- get transaction details for one or more txid
- inputs:
  - tx_hashes - hash1,hash2,hash3,...
  - decode_as_json - Optional - returns more detail but in a escaped raw json format

alpha_info

- get running daemon details - height, etc.
- inputs: None

alpha_sendrawtransaction

- broadcast a previously built signed tx
- inputs:
  - tx_as_hex string - can be generated with charlie_build_tx or fred_build_tx

alpha_transaction_pool

- get mempool details
- inputs: None

mine-to-bill

- generate 1 or more blocks to bill wallet
- inputs:
  - num_blocks - defaults to 1

bill_balance

- get bill wallet balance details
- inputs: None

bill_refresh_wallet

- update bill's wallet from the daemon latest info
- inputs: None

bill_transfer_to

- build, sign and broadcast a transaction from bill wallet to another address
- inputs:
  - amount in in atomic units 1e12 - e.g. 1230000000000 = 1.23 XMR
  - address - recipient primary address - account index 0, subaddr_indeces [0]
  - unlock_time - unlock after n blocks and make spendable - defaults to 0 (no lock)

charlie_balance

- get charlie wallet balance details
- inputs: None

charlie_refresh_wallet

- update charlie's wallet from the daemon latest info
- inputs: None

charlie_build_tx

- build a signed tx for later broadcasting using alpha_send
- inputs:
  - amount in in atomic units 1e12 - e.g. 1230000000000 = 1.23 XMR
  - address - recipient primary address - account index 0, subaddr_indeces [0]
  - unlock_time - unlock after n blocks and make spendable - defaults to 0 (no lock)
-outputs:
  - signed tx_blob
  - tx_hash

charlie_incoming_transfers

- get a list of incoming mined transfers to charlie wallet
- inputs: None

charlie_transfer_to

- build, sign and broadcast a transaction from charlie wallet to another address
- inputs
  - amount in in atomic units 1e12 - e.g. 1230000000000 = 1.23 XMR
  - address - recipient primary address - account index 0, subaddr_indeces [0]
  - unlock_time - unlock after n blocks and make spendable - defaults to 0 (no lock)

fred_export_outputs

- export fred outputs hex
- input:
  - all - defaults to true - otherwise only new outputs since the last call

charlie_export_outputs

- export charlie outputs hex
- input:
  - all - defaults to true - otherwise only new outputs since the last call

charlie_view_export_outputs

- export charlie_view outputs hex - charlie_view knows the outputs but has no spend key
- inputs: None
- only useful in offline, cold signing process using monero-wallet-cli interactive tool
  must be hex decoded into a file to use in monero-wallet-cli

fred_export_key_images

- export signed key images from fred wallet - an array of key images and ephemeral signatures
- input:
  - all - defaults to true - otherwise only new key images since the last call

charlie_export_key_images

- export signed key images from charlie wallet - an array of key images and ephemeral signatures
- input:
  - all - defaults to true - otherwise only new key images since the last call

fred_balance

- get fred wallet balance details
- inputs: None

fred_refresh_wallet

- update fred's wallet from the daemon latest info
- inputs: None

fred_build_tx

- build a signed tx for later broadcasting using alpha_send
- inputs:
  - amount in in atomic units 1e12 - e.g. 1230000000000 = 1.23 XMR
  - address - recipient primary address - account index 0, subaddr_indeces [0]
  - unlock_time - unlock after n blocks and make spendable - defaults to 0 (no lock)
-outputs:
  - signed tx_blob
  - tx_hash

fred_incoming_transfers

- get a list of incoming mined transfers to fred wallet
- inputs: None

fred_transfer_to

- build, sign and broadcast a transaction from bill wallet to another address
- inputs
  - amount in in atomic units 1e12 - e.g. 1230000000000 = 1.23 XMR
  - address - recipient primary address - account index 0, subaddr_indeces [0]
  - unlock_time - unlock after n blocks and make spendable - defaults to 0 (no lock)

wallets

- wallet details exported to the harness environment - useful for building commands in the harness window 0

help

- this help

quit

- shutdown daemons and quit the harness

[dev-warrior777]

Added a new function - alpha_get_transactions_details

Gets many details about a transaction including tx lock time which will be needed for creating & sending Monero transactions which cannot be spent until some number of blocks in the future as part of the Monero swap protocol. This is different than the unlock fields in other rpc responses which refer to the normal Monero mandatory wait for a tx to have 10 confirmations. In contrast a locked tx cannot be spent until after the tx locktime expires which can be e.g. 1000 blocks in the future. We will need to calculate that future unix time based on an average block time * tx lock time in blocks I guess.

[dev-warrior777]

Note: I have noticed during testing that the monerod --regtest code forces a ring size of 16. This is different than mainnet which is always 11. The field rct_signatures is also set to a higher version (6) compared to the normal 5.

This may or may not have an effect on real transaction verification.

Why? I noticed the forced change in the logs and I am guessing Monero is going to update to a rct size of 16 .. maybe soon, and keeps 16 for their own dev. I also noticed some of the daemon rpcs also spit out profiling info to stderr which I had to filter 2>/dev/null to get nice json for jq to parse.

[dev-warrior777]

I have also re-written all the rpc code in golang - methods, json structs, etc. for a head start.

[JoeGruffins]

Do monero wallets only have one address?

Is it possible to add another node? There is only alpha. Would be good to have a beta, this way we can simulate different wallets on different nodes. Does monero have reorgs?

nit: There are some bad white spaces, or white spaces at the end of lines. If there is a setting on your editor you can see them maybe. Not a big deal.

Working! Looks good.

[JoeGruffins]

Can add closes #2753 to your opening description to close the issue.

[dev-warrior777]

Do monero wallets only have one address?

No ... A wallet has One or more accounts each of which has a Primary Address - addr 0 subaddr index [0]

Mainnet (& regtest) Primary Address
494aSG3QY1C4PJf7YyDjFc9n2uAuARWSoGQ3hrgPWXtEjgGrYDn2iUw8WJP5Dzm4GuMkY332N9WfbaKfu5tWM3wk8ZeSEC5

• Each account has 4 keys:
  • private Spend key .. spend outputs
  • private View key .. view incoming transfers
  • public Spend key
  • public View key

Here is a stagenet account

55ps81tfB2JTdbHXYVJuZVeYCYagjLjBPgHtH6DRHXZ3eMLLtE7FECTMmGzJmFVqPz75KsVcVdGMfej8grDiEx5KDmU7NBA
secret: 1c8eb5be5d2488caa8b9058d23e709f0e8a8677834f07e6702889979c4625304
public: 696d1b57e07f939f3733fb16fc10f4e06cadd6c74e43faeae7faae9a95073adf
secret: 894b7da3b33a9a35bd434461a70a56c519bcef11268a6efc5fb9eb3c83797c01
public: 4df48d1418985c7c22d75c01243e752451486f0bb7c80ae18d322155e82d8e71

• Each account can generate any number of Sub-Addresses each of which has a sub-index:
  • They start with 7... on mainnet. E.G. 7XybV3...
  • Why sub addresses

https://web.getmonero.org/resources/moneropedia/account.html

The harness has deliberately simplified transfers and just uses 1 account and one primary address per wallet. Type: $ ./wallets to see the primary addresses for fred, bill & charlie. More complex account/subaddress generation functions can be added if required.

The go-monero code has a full set of wallet functions.

[dev-warrior777]

Is it possible to add another node? There is only alpha. Would be good to have a beta, this way we can simulate different wallets on different nodes. Does monero have reorgs?

Monero definitely gets attacked all the time and has reorgs. They are best seen in the daemon logs for stagenet where I see reorgs occasionally as if people were testing them. I recommend trying the same.
You can also test double-spends more realistically on stagenet

I would prefer not to spend prop time at the present adding other nodes as last time I started with a couple of regtest nodes but could not connect wallets but am happy to try again if the need is there.

[dev-warrior777]

nit: There are some bad white spaces, or white spaces at the end of lines. If there is a setting on your editor you can see them maybe. Not a big deal.

Actually this can have an effect on some bash inbuilt functions! I will update this and push later (my) today. Did you see end-line whitespace in both script files?

I have already updated monero_functions.inc to monero_functions for the same push.

[martonp]

Working well for me. One suggestion would be that instead of having a separate file for each function, we could have one file for wallet, and then the function that we want to call would be an argument. Instead of ./bill_balance, we would call ./bill balance. This way each time we add another function, the folder wouldn't get more cluttered.

[dev-warrior777]

I originally planned to do like bitcoin harnesses but monero has no concept of bitcoin-cli tool as in ./alpha sendtoaddress, ./beta-sendtoaddress, etc.

If you are working in golang try https://github.com/dev-warrior777/go-monero
It is in my github as we did not find a place for it yet. It works over regtest harness & stagenet.

[dev-warrior777]

When the harness starts up there will be a new folder ~/dextest/xmr/wallets/own with only the monero-wallet-rpc wallet server log. You can make a new wallet here programmatically.

Then use the Go example code to GenerateFromKeys to generate a new wallet called maybe bob

There are other 'howto' examples here