integration tests: Mint assets in the cluster faucet, and add basic MA scenarios #2465

rvl · 2021-01-25T14:22:42Z

Issue Number

ADP-695 and a little ADP-604.

Overview

Faucet mints assets as part of allocating funds to wallets.
There are 3 pre-generated policy objects in faucet.
Add a new type of fixture wallet for multi-asset.
Add a test for the assets balance of a fixture wallet.
Add test to transfer assets between wallets.
Add test for assets in transaction history
Add test for listing assets

Comments

cardano-cli --tx-out syntax described here: cli TxOutParser and ValueParser.
Rather than assigning assets to faucet wallets, integration tests could also submit asset minting transactions with cardano-cli as part of setup.

Anviking · 2021-01-28T19:03:45Z

lib/core-integration/src/Test/Integration/Scenario/API/Shelley/Transactions.hs

+ let TokenBundle.AssetId pid an = Set.elemAt 0 (TokenMap.getAssets assetsSrc)
+ let val = [(toText pid, toText an, minUTxOValue)]
+
+ payload <- liftIO $ mkTxPayloadMA ctx wDest minUTxOValue val fixturePassphrase


Seeing some strange things removing the pendingWith after rebasing (not sure it is the same as what you, @rvl saw):

As is here, specifying minUTxOValue ada

Wallet error: InsufficientMinCoinValueError: 1 ada, expectedMinCoinValue=1.4 ada

"{\\\"code\\\":\\\"utxo_too_small\\\",\\\"message\\\":\\\"Some outputs have ada values that are too small. There's a minimum ada value specified by the protocol that each output must satisfy. I'll handle that minimum value myself when you do not explicitly specify an ada value for an output. Otherwise, you must specify enough ada. Here are the problematic outputs: InsufficientMinCoinValueError {outputWithInsufficientAda = TxOut {address = Address {unAddress = \\\\\\\"\\\\\\\\SOH\\\\\\\\238\\\\\\\\229Q\\\\\\\\178\\\\\\\\149\\\\\\\\244\\\\\\\\165`\\\\\\\\169Z\\\\\\\\250\\\\\\\\208AC@\\\\\\\\vx\\\\\\\\165\\\\\\\\242\\\\\\\\f\\\\\\\\254\\\\\\\\196\\\\\\\\232\\\\\\\\160[\\\\\\\\225kQ\\\\\\\\197\\\\\\\\196\\\\\\\\ETX\\\\\\\\148\\\\\\\\185/\\\\\\\\186\\\\\\\\&7S\\\\\\\\179\\\\\\\\153Ao\\\\\\\\EM\\\\\\\\221\\\\\\\\SO\\\\\\\\191\\\\\\\\239\\\\\\\\251\\\\\\\\173\\\\\\\\131\\\\\\\\176\\\\\\\\236\\\\\\\\DLE\\\\\\\\234+q\\\\\\\\a\\\\\\\"}, tokens = TokenBundle {coin = Coin 1000000, tokens = TokenMap (fromList [(UnsafeTokenPolicyId (Hash \\\\\\\"K\\\\\\\\254z\\\\\\\\202\\\\\\\\225\\\\\\\\189%\\\\\\\\153d\\\\\\\\153b\\\\\\\\177F\\\\\\\\161\\\\\\\\228}.\\\\\\\\DC4\\\\\\\\147\\\\\\\\&8\\\\\\\\t\\\\\\\\179g\\\\\\\\232\\\\\\\\EOT\\\\\\\\198\\\\\\\\US\\\\\\\\134\\\\\\\"),NonEmptyMap {least = (UnsafeTokenName \\\\\\\"\\\\\\\",TokenQuantity 1000000), rest = fromList []})])}}, expectedMinCoinValue = Coin 1407406} :| []\\\"}\", responseCookieJar = CJ {expose = []}, responseClose' = ResponseClose}")

When Specifying 0 ada

Ledger error: OutputTooSmallUTxO: 1 ada

from the following response: Left (DecodeFailure "{\"code\":\"created_invalid_transaction\",\"message\":\"That's embarrassing. It looks like I've created an invalid transaction that could not be parsed by the node. Here's an error message that may help with debugging: HardForkApplyTxErrFromEra S (S (S (Z (WrapApplyTxErr {unwrapApplyTxErr = ApplyTxError [LedgerFailure (UtxowFailure (UtxoFailure (OutputTooSmallUTxO [(Addr Mainnet (KeyHashObj (KeyHash \\\"e4674505240c98422ad66a98ecdaca42770fa7c66a2bf1f81048c5ad\\\")) (StakeRefBase (KeyHashObj (KeyHash \\\"211fa28b02df352f6a5e93f9c920b967e5359180cb36b499093647f3\\\"))),Value 1407406 (fromList [(PolicyID {policyID = ScriptHash \\\"4bfe7acae1bd2599649962b146a1e47d2e14933809b367e804c61f86\\\"},fromList [(\\\"\\\",1000000)])]))])))]}))))\"}")

That second error is quite bad. Sounds like something isn't working properly. When specyfing 0 Ada, the server should assign the minimum ada value needed for that UtxO. And, the first error suggests that the server is perfectly able to compute that minimum value because it suggests it in the error message. So, something is off..

As is here, specifying minUTxOValue ada

As far as I understand it is expected, since minUTxOValue may be different for MA. (Correct me if I'm wrong)

When Specifying 0 ada

Interesting, because I don't see it on mary_qa. When specifying 0 ADA on tx with MA the transaction is given minUTxOValue correctly and tx is succesful.

The first case is indeed expected, and the error returned by the server looks legit (1.4 Ada of expected min coin value). The second error is puzzling. From the message however, we can't really see if the error is on the change output or on the requested output.

The "expected" min ada behaviour is a little odd, but I have amended the test cases accordingly. Perhaps the HTTP status code should be 400 Bad Request instead of 403 Forbidden.

I can't reproduce the second error in the integration tests.

The "expected" min ada behaviour is a little odd

What do you mean? Do you find the server behavior odd, or the ledger requirement odd?

Perhaps the HTTP status code should be 400 Bad Request instead of 403 Forbidden.

That's a semantic debate, yet I do think of 400 Bad Request as requests which are malformed (improper JSON, invalid types, missing required keys etc..). Though, if the request is syntactically valid, but only present an invalid combination of inputs, then I rather go for 403.

I can't reproduce the second error in the integration tests.

You mean that, passing a value of 0 Ada has the intended behavior (that is, min ada value is automatically assigned, and the transaction go through) ?

Oops, I think the UtxoFailure (OutputTooSmallUTxO I saw was since I was still on 1.24.2 locally.

Now, with 1.25.1 I only get

src/Test/Integration/Scenario/API/Shelley/Transactions.hs:227:13: 1) API Specifications, SHELLEY_TRANSACTIONS, Regression ADP-626 - Filtering transactions between eras expected: 1 but got: 2 To rerun use: --match "/API Specifications/SHELLEY_TRANSACTIONS/Regression ADP-626 - Filtering transactions between eras/" src/Test/Integration/Scenario/API/Shelley/Transactions.hs:701:22: 2) API Specifications, SHELLEY_TRANSACTIONS, TRANS_CREATE_10 - Multi-asset transaction with Ada While verifying (Status {statusCode = 200, statusMessage = "OK"},Right (ApiWallet {id = ApiT {getApiT = WalletId {getWalletId = 56c7a5f8489970d793363da6b6cad018ce8e8af4}}, addressPoolGap = ApiT {getApiT = AddressPoolGap {getAddressPoolGap = 20}}, balance = ApiWalletBalance {available = Quantity {getQuantity = 2000000}, total = Quantity {getQuantity = 2000000}, reward = Quantity {getQuantity = 0}}, assets = ApiWalletAssetsBalance {available = ApiT {getApiT = TokenMap (fromList [(UnsafeTokenPolicyId (Hash "K\254z\202\225\189%\153d\153b\177F\161\228}.\DC4\147\&8\t\179g\232\EOT\198\US\134"),NonEmptyMap {least = (UnsafeTokenName "",TokenQuantity 1000000), rest = fromList []})])}, total = ApiT {getApiT = TokenMap (fromList [(UnsafeTokenPolicyId (Hash "K\254z\202\225\189%\153d\153b\177F\161\228}.\DC4\147\&8\t\179g\232\EOT\198\US\134"),NonEmptyMap {least = (UnsafeTokenName "",TokenQuantity 1000000), rest = fromList []})])}}, delegation = ApiWalletDelegation {active = ApiWalletDelegationNext {status = NotDelegating, target = Nothing, changesAt = Nothing}, next = []}, name = ApiT {getApiT = WalletName {getWalletName = "Empty Wallet"}}, passphrase = Just (ApiWalletPassphraseInfo {lastUpdatedAt = 2021-01-29 09:26:00.183826 UTC}), state = ApiT {getApiT = Ready}, tip = ApiBlockReference {absoluteSlotNumber = ApiT {getApiT = SlotNo 551}, slotId = ApiSlotId {epochNumber = ApiT {getApiT = EpochNo {unEpochNo = 11}}, slotNumber = ApiT {getApiT = SlotInEpoch {unSlotInEpoch = 1}}}, time = 2021-01-29 09:27:29.2 UTC, block = ApiBlockInfo {height = Quantity {getQuantity = 234}}}})) Waited longer than 90s to resolve action: "Payee wallet balance is as expected". expected: 1000000 but got: 2000000 To rerun use: --match "/API Specifications/SHELLEY_TRANSACTIONS/TRANS_CREATE_10 - Multi-asset transaction with Ada/" Randomized with seed 947347928 Finished in 128.2986 seconds 173 examples, 2 failures, 7 pending

KtorZ · 2021-01-29T08:40:37Z

lib/core-integration/src/Test/Integration/Scenario/API/Shelley/Transactions.hs

+ let val = [(toText pid, toText an, minUTxOValue)]
+
+ -- Allow for a higher minimum utxo coin due to assets
+ let minAda = minUTxOValue * 2


I think we should ultimately provide this as an API endpoint.

Sounds good.

jonathanknowles

Hi @rvl

Just some minor comments.

One thing I noticed is that a few functions accept or return multiple Strings in their type signatures, and it wasn't always obvious to me (without some digging) which String referred to what. Perhaps it would be worth making some newtypes to replace the use of String in a few places?

But overall looks good!

lib/core-integration/src/Test/Integration/Scenario/API/Shelley/Transactions.hs

jonathanknowles · 2021-01-29T09:53:18Z

lib/core-integration/src/Test/Integration/Faucet.hs

+
+-- | A token bundle, along with the keys and scripts necessary to mint the
+-- bundle.
+type MintTokenBundle = (TokenBundle, [(String, String)])


Perhaps it would be worth defining type aliases or even newtypes for the String parts?

(Just to aid readability, as it might not be immediately clear which String refers to which value.)

There are a few functions where these are passed around: I suspect it might help make the code easier to follow if these have named types.

Yes good point - I would like to have made a record type for this.
But we have a package organisation issue that make it impractical to do at the moment.

I have tried to improve the situation a little by replacing one string with TokenPolicyId, and adding more haddock.

jonathanknowles · 2021-01-29T09:56:07Z

lib/core-integration/src/Test/Integration/Faucet.hs

+maryTokenBundles :: [MintTokenBundle]
+maryTokenBundles = tokenBundles maryAssetScripts
+
+tokenBundles :: [(String, (String, String))] -> [MintTokenBundle]


Same here. (regarding String)

jonathanknowles · 2021-01-29T09:58:18Z

lib/core-integration/src/Test/Integration/Scenario/API/Shelley/Transactions.hs

+ r <- request @ApiWallet ctx (Link.getWallet @'Shelley w) Default Empty
+ verify r
+ [ expectField (#assets . #available) (`shouldNotBe` mempty)
+ , expectField (#assets . #total) (`shouldNotBe` mempty)


Just a suggestion, but perhaps TokenBundle.empty here might make this more explicit. (And at other similar places where mempty is used.)

(I'm guessing that it's a TokenBundle 😃)

The type of it is not too important ... "some kind of assets".
I applied your suggestion - see if it looks better or worse.

jonathanknowles · 2021-01-29T10:00:32Z

lib/core-integration/src/Test/Integration/Scenario/API/Shelley/Transactions.hs

+ => Context
+ -> ApiWallet
+ -> Natural
+ -> [((Text, Text), Natural)]


Perhaps it would be nice to add a field comment here or type alias here, just to state what these Text values should be.)

There is a comment which says (PolicyId Hex, AssetName Hex) just above - is that alriight?

jonathanknowles · 2021-01-29T10:01:07Z

lib/core/src/Cardano/Wallet/Primitive/CoinSelection/MA/RoundRobin.hs

+ build (InsufficientMinCoinValueError o c) = unlinesF
+ [ nameF "Expected min coin value" (build c)
+ , nameF "TxOut" (build o)
+ ]


KtorZ · 2021-02-01T13:38:00Z

lib/core-integration/src/Test/Integration/Scenario/API/Shelley/Transactions.hs

+ verify r
+ [ expectField (#assets . #available . #getApiT) (`shouldNotBe` TokenMap.empty)
+ , expectField (#assets . #total . #getApiT) (`shouldNotBe` TokenMap.empty)
+ ]


KtorZ · 2021-02-01T13:38:36Z

lib/core-integration/src/Test/Integration/Scenario/API/Shelley/Transactions.hs

+ (`shouldNotBe` TokenMap.empty)
+ , expectField (#balance . #total . #getQuantity)
+ (`shouldBe` minCoin)
+ ]


KtorZ · 2021-02-01T13:38:53Z

lib/core-integration/src/Test/Integration/Scenario/API/Shelley/Transactions.hs

+ rtx <- request @(ApiTransaction n) ctx
+ (Link.createTransaction @'Shelley wSrc) Default payload
+ -- It should fail with InsufficientMinCoinValueError
+ expectResponseCode HTTP.status403 rtx


KtorZ · 2021-02-01T13:39:41Z

lib/core-integration/src/Test/Integration/Scenario/API/Shelley/Transactions.hs

+ verify rb
+ [ expectField (#assets . #available . #getApiT) (`shouldNotBe` TokenMap.empty)
+ , expectField (#assets . #total . #getApiT) (`shouldNotBe` TokenMap.empty)
+ ]


KtorZ · 2021-02-01T13:47:46Z

lib/core-integration/src/Test/Integration/Scenario/API/Shelley/Transactions.hs

+ verify rb
+ [ expectField (#assets . #available . #getApiT) (`shouldNotBe` TokenMap.empty)
+ , expectField (#assets . #total . #getApiT) (`shouldNotBe` TokenMap.empty)
+ ]


KtorZ · 2021-02-02T16:39:24Z

bors merge

KtorZ · 2021-02-02T16:39:28Z

*please

2465: integration tests: Mint assets in the cluster faucet, and add basic MA scenarios r=KtorZ a=rvl ### Issue Number ADP-604 ### Overview - Faucet mints assets as part of allocating funds to wallets. - There are 3 pre-generated policy objects in faucet. - Add a new type of fixture wallet for multi-asset. - Add a test for the assets balance of a fixture wallet. - Add a pending test to transfer assets between wallets. ### Comments - `cardano-cli --tx-out` syntax described here: [cli TxOutParser](https:/input-output-hk/cardano-node/blob/master/cardano-cli/src/Cardano/CLI/Mary/TxOutParser.hs) and [ValueParser](https:/input-output-hk/cardano-node/blob/master/cardano-cli/src/Cardano/CLI/Mary/ValueParser.hs). - Rather than assigning assets to faucet wallets, integration tests could also submit asset minting transactions with cardano-cli as part of setup. - Setup code is a little bit scrappy, sorry. Co-authored-by: Rodney Lorrimar <[email protected]>

iohk-bors · 2021-02-02T17:25:09Z

Build failed:

buildkite/cardano-wallet

There were two test cases which needed to be retried.

/API Specifications/BYRON_ADDRESSES/ADDRESS_IMPORT_04 - I can import a used address (idempotence)

SQLite3 returned ErrorBusy while attempting to perform step: database is locked
retry: expected: Status {statusCode = 404, statusMessage = "Not Found"}
retry:  but got: Status {statusCode = 500, statusMessage = "Internal Server Error"}
retry:
retry: from the following response: Left (DecodeFailure "Something went wrong")
retry: Retrying failed test.
Cannot decode byte '\xb2': Data.Text.Internal.Encoding.decodeUtf8: Invalid UTF-8 stream

/API Specifications/SHELLEY_HW_WALLETS/HW_WALLETS_04 - Can manage HW wallet the same way as others/Can list transactions

retry: expected: Status {statusCode = 201, statusMessage = "Created"}
retry:  but got: Status {statusCode = 500, statusMessage = "Internal Server Error"}
retry:
retry: from the following response: Left (DecodeFailure "{\"code\":\"unexpected_error\",\"message\":\"That's embarrassing. Your wallet looks good, but I couldn't open a new database to store its data. This is unexpected and likely not your fault. Perhaps, check your filesystem's permissions or available space?\"}")
retry: Retrying failed test.

And the "real" failures which failed twice:

Failures:

  src/Test/Integration/Framework/DSL.hs:1183:23:
  1) CLI Specifications, SHELLEY_CLI_TRANSACTIONS, TRANS_LIST_03 - Can order results
       fixtureWallet: waited too long for initial transaction

  To rerun use: --match "/CLI Specifications/SHELLEY_CLI_TRANSACTIONS/TRANS_LIST_03 - Can order results/"

  ... and 15 more of the same.

#expected

KtorZ · 2021-02-02T18:17:12Z

Haven't seen that one before 🤔
Could this be due to an unfortunate change in this PR ?

rvl · 2021-02-03T06:23:18Z

Yes hmm fixtureWallet: waited too long for initial transaction, sounds like it could be faucet-related.

rvl · 2021-02-03T14:15:46Z

I think the tests failed because of a change I made which meant that some shelley faucet wallets were unfunded.
The problem was a lack of byron faucet addrs. So I generated more addresses for the byron genesis and now there should be enough funds for both shelley and ma faucets.

bors r+

iohk-bors · 2021-02-03T15:01:01Z

Build succeeded:

rvl self-assigned this Jan 25, 2021

rvl force-pushed the rvl/adp-604/test-cluster-assets branch 2 times, most recently from da02cde to b7bf6eb Compare January 28, 2021 09:15

Anviking force-pushed the rvl/adp-604/test-cluster-assets branch from b7bf6eb to a060bd3 Compare January 28, 2021 17:38

Anviking reviewed Jan 28, 2021

View reviewed changes

rvl force-pushed the rvl/adp-604/test-cluster-assets branch 2 times, most recently from e11d935 to 3e4dfb7 Compare January 29, 2021 07:47

rvl changed the title ~~cluster: Add function for minting an asset for the faucet~~ integration tests: Mint assets in the cluster faucet, and add basic MA scenarios Jan 29, 2021

KtorZ reviewed Jan 29, 2021

View reviewed changes

rvl force-pushed the rvl/adp-604/test-cluster-assets branch 2 times, most recently from edcdb8b to a6f2d22 Compare January 29, 2021 09:42

jonathanknowles reviewed Jan 29, 2021

View reviewed changes

rvl force-pushed the rvl/adp-604/test-cluster-assets branch 3 times, most recently from 1673496 to 65cc0d8 Compare February 1, 2021 13:29

KtorZ approved these changes Feb 1, 2021

View reviewed changes

KtorZ force-pushed the rvl/adp-604/test-cluster-assets branch from 65cc0d8 to 8cb42ae Compare February 2, 2021 16:39

rvl added 4 commits February 3, 2021 21:15

tests: Add fixtureMultiAssetWallet with minted assets

a248551

integration tests: check balance of multi-asset faucet wallet

4680747

Format API error message for InsufficientMinCoinValues

7a2cf4c

Multi-asset faucet and basic integration tests

8ce1597

rvl force-pushed the rvl/adp-604/test-cluster-assets branch from 8cb42ae to 8ce1597 Compare February 3, 2021 13:15

rvl added 2 commits February 3, 2021 21:59

Update gen-byron-funds.sh script

89b9d67

add more byron faucet addrs

292d836

More funds for shelley and ma faucets

cbed03c

iohk-bors bot merged commit 0c981f7 into master Feb 3, 2021

iohk-bors bot deleted the rvl/adp-604/test-cluster-assets branch February 3, 2021 15:02

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

integration tests: Mint assets in the cluster faucet, and add basic MA scenarios #2465

integration tests: Mint assets in the cluster faucet, and add basic MA scenarios #2465

rvl commented Jan 25, 2021 •

edited

Loading

Anviking Jan 28, 2021 •

edited

Loading

KtorZ Jan 28, 2021

piotr-iohk Jan 28, 2021

KtorZ Jan 28, 2021

rvl Jan 29, 2021

KtorZ Jan 29, 2021

KtorZ Jan 29, 2021

Anviking Jan 29, 2021 •

edited

Loading

KtorZ Jan 29, 2021

rvl Jan 29, 2021

jonathanknowles left a comment

jonathanknowles Jan 29, 2021

rvl Feb 1, 2021

jonathanknowles Jan 29, 2021

jonathanknowles Jan 29, 2021

rvl Jan 29, 2021

jonathanknowles Jan 29, 2021

rvl Jan 29, 2021

jonathanknowles Jan 29, 2021

KtorZ Feb 1, 2021

KtorZ Feb 1, 2021

KtorZ Feb 1, 2021

KtorZ Feb 1, 2021

KtorZ Feb 1, 2021

KtorZ commented Feb 2, 2021

KtorZ commented Feb 2, 2021

iohk-bors bot commented Feb 2, 2021 •

edited by rvl

Loading

KtorZ commented Feb 2, 2021

rvl commented Feb 3, 2021

rvl commented Feb 3, 2021

iohk-bors bot commented Feb 3, 2021

integration tests: Mint assets in the cluster faucet, and add basic MA scenarios #2465

integration tests: Mint assets in the cluster faucet, and add basic MA scenarios #2465

Conversation

rvl commented Jan 25, 2021 • edited Loading

Issue Number

Overview

Comments

Anviking Jan 28, 2021 • edited Loading

Choose a reason for hiding this comment

As is here, specifying minUTxOValue ada

When Specifying 0 ada

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Anviking Jan 29, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jonathanknowles left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

KtorZ commented Feb 2, 2021

KtorZ commented Feb 2, 2021

iohk-bors bot commented Feb 2, 2021 • edited by rvl Loading

KtorZ commented Feb 2, 2021

rvl commented Feb 3, 2021

rvl commented Feb 3, 2021

iohk-bors bot commented Feb 3, 2021

rvl commented Jan 25, 2021 •

edited

Loading

Anviking Jan 28, 2021 •

edited

Loading

As is here, specifying `minUTxOValue` ada

When Specifying `0` ada

Anviking Jan 29, 2021 •

edited

Loading

iohk-bors bot commented Feb 2, 2021 •

edited by rvl

Loading