build and tests proposals by stages #4

turadg · 2023-10-26T23:44:12Z

This makes the Dockerfile dynamically from the proposal sequence declarations. It also introduces the ability to test core-eval proposals in their own layer.

michaelfig

Looking good so far!

michaelfig · 2023-10-27T02:34:44Z

Makefile

-agoric-upgrade-8-1: agoric-upgrade-8
- $(BUILD) --target agoric-upgrade-8-1 -t $(REPOSITORY):agoric-upgrade-8-1$(TAG_SUFFIX)
-
-agoric-upgrade-9: agoric-upgrade-8-1
+agoric-upgrade-9: agoric-upgrade-8


As I mentioned in chat, 8-1 is also known as PismoB. It existed to run a few blocks using that version of agd to verify that it wasn't horribly broken. I'm okay with you explicitly deciding to skip it, but it makes me wonder what criteria are used to decide which upgrades are important and which are not.

makeDockerfile.ts

raphdev

This looks good to me so far. Does the current flow surface any breaking changes -- If I change the templates, and they result in an invalid Dockerfile, would the existing CI error in an obvious way?

turadg · 2023-11-02T14:44:43Z

Reviewers, I think this is close to landable. I still have to clean up the commit history and some docs (DESIGN.md) but it's ready for review.

I just noticed that the last CI failed but I know it had passed for me locally so I don't expect that fixing it will change much about the code.

Please review for ergonomics in addition to correctness. Try checking it out and running the paces. I don't expect to incorporate all ergonomics feedback into this PR but I will triage and queue up the additional work.

We're trying to get this in soon to make #3 easier to land.

dckc

Looks like useful progress.

I have not yet run it locally.

dckc · 2023-11-02T15:08:00Z

.github/workflows/ci.yml

- -f Dockerfile upgrade-test-scripts
- - name: run final proposal
- run: docker run --env "DEST=0" docker-upgrade-test:latest
+ - run: yarn global add tsx


I struggled to read the diff here... I guess name: isn't required?

right, it defaults to the run string

dckc · 2023-11-02T15:10:01Z

.prettierrc.json

@@ -0,0 +1,5 @@
+{


I don't see a .prettierrc.json in https:/Agoric/agoric-sdk/tree/master . Why do it differently here?

sdk's prettier config is in package.json. I thought it would be confusing to have a package.json here because there's no JS at the top level

dckc · 2023-11-02T15:19:24Z

README.md


-```shell
-make build


Did "how to use it" move somewhere else? I don't see it.

no, it's in the TODO now.

ah. very well.

I did look in the TODO list without seeing it, but as long as that's the intent, cool.

dckc · 2023-11-02T15:20:24Z

README.md

-### Troubleshooting
-If you get an error about port 26656 already in use, you have a local chain running on your OS.


The troubleshooting info seems useful.

yeah, I just wasn't sure which parts were accurate anymore so figured no docs are better than wrong docs. I have a TODO to increase (accurate) documentation. I hope that's okay to do in follow-up PRs

README.md

proposals/55:statom-vaults/package.json

dckc · 2023-11-02T15:57:13Z

proposals/package.json

+ "dependencies": {
+ "ava": "^5.3.1",
+ "better-sqlite3": "^8.5.1",
+ "execa": "^7.2.0",


we seem to declare these dependencies a number of times. is this by design?

tech debt at this point. I'll see if I can get rid of this package.json in this PR

dckc · 2023-11-02T15:58:10Z

runTestImages.ts

+ // 'rm' to remove the container when it exits
+ const cmd = `docker run --rm ${name}`;


The Makefile had several other args to docker run. We no longer need them?

they were only conveniences and they don't work anymore.

I will develop new conveniences but I don't see those as blockers for this PR

dckc · 2023-11-02T15:59:02Z

upgrade-test-scripts/agoric-upgrade-12/actions.js

- await voteLatestProposalAndWait();
-};
-
-export const runZcfUpgrade = async (zcfBundleId, zoeBundleId) => {


All this ZCF stuff is gone on purpose?

upgrade 12 is in agoric-sdk, not this repo until the upgrade passes. per the discussion with @mhofman

upgrade-test-scripts/lib/assert.js

to be handled elsewhere

turadg added 4 commits October 26, 2023 13:28

build: check in yarn.lock

84a7e80

refactor: DONE_MARKER

8dda00f

refactor: yarn install

ff709a6

build Dockerfile by proposal sequence

4fd4829

turadg requested review from michaelfig, raphdev and iomekam October 26, 2023 23:44

michaelfig reviewed Oct 27, 2023

View reviewed changes

raphdev reviewed Oct 27, 2023

View reviewed changes

turadg requested review from raphdev and michaelfig November 2, 2023 14:41

dckc approved these changes Nov 2, 2023

View reviewed changes

dckc mentioned this pull request Nov 2, 2023

shell formatting conventions / tool support #7

Open

turadg added 16 commits November 2, 2023 15:13

chore: prettier config

3f6bfec

chore: specify license

22867ac

remove actions/tests from start_to_to

3321f7e

to be handled elsewhere

misc cleanup of shell scripts

93becbf

drive stages by proposal directories

f518ebb

restore actions

16d90de

better separator comment

c424265

upgrade-10 running with JS

6063845

upgrade-11 running

d7e939e

test-upgrade-8 working

6c0cf49

test-upgrade-9 working

66734ef

test-upgrade-10 working

e168cbf

remove empty dirs

310624f

remove upgrade-12 which has to be in agoric-sdk

77628de

move TEST after USE

296f2a0

test-upgrade-11 working

5c8dee0

turadg and others added 16 commits November 2, 2023 15:27

run new test images in CI

174ad97

ci: provide tsx

0f75df4

build Dockerfile as needed

e56a204

distinguish proposal types

6a63dec

improve tagging

5395957

code for stATOM proposal

1bbe4e4

consolidate lib

50419fd

stATOM proposal

f0eff23

upgrade-test-scripts out of agoric-sdk

d9c49b4

rename actions.sh to use.sh

ea50a1a

clean up README

a645364

get package.json out of proposals root

230d4db

feat: name match for CLIs

5d5a08e

increase timeout for stATOM test

53fa80e

clean up docs

b88883b

comments cleanup

6bc407a

turadg force-pushed the ta/docker-template branch from f7e488b to 6bc407a Compare November 2, 2023 22:28

turadg enabled auto-merge November 2, 2023 22:32

turadg changed the title ~~build Dockerfile by proposal sequence~~ build and tests proposals by stages Nov 2, 2023

turadg merged commit a949621 into main Nov 2, 2023
1 check passed

turadg deleted the ta/docker-template branch November 2, 2023 23:11

dckc mentioned this pull request Nov 3, 2023

test stATOM proposal in CI using an upgrade test Agoric/agoric-sdk#8177

Closed

turadg mentioned this pull request Nov 3, 2023

CLI for deployment upgrade test Agoric/agoric-sdk#8317

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

build and tests proposals by stages #4

build and tests proposals by stages #4

turadg commented Oct 26, 2023 •

edited

Loading

michaelfig left a comment

michaelfig Oct 27, 2023

raphdev left a comment

turadg commented Nov 2, 2023

dckc left a comment

dckc Nov 2, 2023

turadg Nov 2, 2023

dckc Nov 2, 2023

turadg Nov 2, 2023

dckc Nov 2, 2023

turadg Nov 2, 2023

dckc Nov 2, 2023

dckc Nov 2, 2023

turadg Nov 2, 2023

dckc Nov 2, 2023

turadg Nov 2, 2023

dckc Nov 2, 2023

turadg Nov 2, 2023

dckc Nov 2, 2023

turadg Nov 2, 2023 •

edited

Loading

		### Troubleshooting
		If you get an error about port 26656 already in use, you have a local chain running on your OS.

		// 'rm' to remove the container when it exits
		const cmd = `docker run --rm ${name}`;

build and tests proposals by stages #4

build and tests proposals by stages #4

Conversation

turadg commented Oct 26, 2023 • edited Loading

michaelfig left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

raphdev left a comment

Choose a reason for hiding this comment

turadg commented Nov 2, 2023

dckc left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

turadg Nov 2, 2023 • edited Loading

Choose a reason for hiding this comment

turadg commented Oct 26, 2023 •

edited

Loading

turadg Nov 2, 2023 •

edited

Loading