Add tests to keep server_api json.go in sync with parse_inputs.rs

[ganeshvanahalli]

This PR adds CI steps to generate a InputJSON file corresponding to a block generated in a system test - TestProgramStorage and run arbitrator prover binary on this json file.

To test locally do this

$> BLOCK_INPUT_JSON_PATH="some_valid_path/block_input.json" go test github.com/offchainlabs/nitro/system_tests --run TestProgramStorage$ --count=1
ok  	github.com/offchainlabs/nitro/system_tests	3.268s

$> make build-prover-bin
<... snip lots of output ...>

$> target/bin/prover target/machines/latest/machine.wavm.br -b --json-inputs="some_valid_path/block_input.json"
<... snip lots of output ...>

Resolves NIT-2820

[tsahee]

That's a very interesting approach. Took some good thinking..
However, I still prefer my original idea: make a system test that stores block to a path chosen by environment variable (or doesn't, if variable not set), have a ci step that runs this test to create a file, and then a ci step that launches prover with it and makes sure it passes.
The advantages of this approach is we see not only that format passed, but that we succeed in launching a machine from that info. For example, if we change "wavm" target name to something else.
Also, the test in the different approach could easily also test your other pr running a jit machine.
LMK if you disagree.

[ganeshvanahalli]

That's a very interesting approach. Took some good thinking.. However, I still prefer my original idea: make a system test that stores block to a path chosen by environment variable (or doesn't, if variable not set), have a ci step that runs this test to create a file, and then a ci step that launches prover with it and makes sure it passes. The advantages of this approach is we see not only that format passed, but that we succeed in launching a machine from that info. For example, if we change "wavm" target name to something else. Also, the test in the different approach could easily also test your other pr running a jit machine. LMK if you disagree.

Makes sense, I agree with your approach, as it guarantees we cover all cases and not just the parsing.

[eljobe]

This environment variable should not be needed. The location of the output file is extremely customizable already:
https:/OffchainLabs/nitro/blob/master/validator/inputs/writer_test.go

And, by default it is in a pretty predictable location.
So, in this CI step, it will be written to:
/home/user/.arbitrum/validation-inputs/TestProgramStorage/*/block_input_5.json
The * just matches a timestamped directory. And, you can read it just like that.

If you want to eliminate that directory to make it even more predictable, you could add a flag (preferred) or an environment variable like --validator.inputswriter.usetimestamp=false and then, check that flag when constructing the inputs.NewWriter and add the inputs.WithTimestampDirEnabled(false) option.

If you're worried about the brittleness of having the block ID in the name of the json file, you could add another option to suprress including the block ID in the name of the file.

Also, if you want the file to go into a workspace, specific directory, we could introduce a flag like --validator.inputswriter.basedir=${{ github.workspace }}/target/ and use it with the inputs.WithBaseDir option when constructing the inputs.NewWriter

[ganeshvanahalli]

Oh cool.
Yep, I was hesitant on having blockID in the fileName as that could change in the future (as it is not deterministic), introducing a new option sounds like a good idea to me!

Other than that, we wanted to have a way to control generation of this file and having a flag itself to control this SGTM.
Thanks for this, inputswriter is quite customizable!

[tsahee]

seems good, but waiting for @eljobe to approve

addressed