feat: Runfile and java artifact support for Bazel Builder #2362

enteraga6 · 2023-07-07T21:32:04Z

This will provide support for artifacts that also need runfiles. Users can use a flag, needs-runfiles, to also package the artifact alongside the runfiles Bazel generates if the user deems it as necessary.

Also, support for Java artifacts is made available by packaging the JARs to be standalone through a format in Bazel called _deploy.jar. Alongside the _deploy.jar, there will be a modified run-script that allows the users that download the artifact to run the run-script by using a flag called local_javabin, where they put the path to their own java bin such that it is utilized by the run-script. This run-script is generated by Bazel from a template and later modified in the build.sh in the internal part of the builder to add this flag for the users. More information is available on the readme.

Java targets will automatically be converted to their _deploy.jar with this. Three flags are used for users that have java targets:

includes-java: if true then adds an additional flag to build command as well as rule for local java repo in WORKSPACE in order to utilize the --singlejar capability of run-script for _deploy.jar such that the remote jdk does not need to be included in runfiles. Doing it like this prevents massive bloat when attesting.

user-java-distribution and user-java-version: let the user specify the exact java they want to use to build

When users run the run-script they will include an additional flag local_javabin which they will set to their local javabin that the run-script will utilize to run the _deploy.jar

A combination of bazel query and bazel cquery were used to resolved edge cases with the implementation.

Signed-off-by: Noah Elzner <[email protected]> Signed-off-by: Noah Elzner <[email protected]>

Fixes getting the filename from the result of `npm pack --json`. Some older versions return the incorrect file name. This change falls back and attempts to determine the filename if the file is not initially found. Fixes slsa-framework#2290 slsa-framework#2294 slsa-framework#2295 slsa-framework#2296 slsa-framework#2297 slsa-framework#2298 slsa-framework#2299 Signed-off-by: Ian Lewis <[email protected]> Signed-off-by: Noah Elzner <[email protected]>

This installs the instance of npm used for publish in a temporary directory rather than polluting the user job global npm modules. --------- Signed-off-by: Ian Lewis <[email protected]> Signed-off-by: Noah Elzner <[email protected]>

Signed-off-by: Noah Elzner <[email protected]> Signed-off-by: Noah Elzner <[email protected]>

Fixes slsa-framework#2311 Add a `secure-package-download` action so that users can more easily download a tgz artifact built by the Node.js builder. Signed-off-by: Ian Lewis <[email protected]> Signed-off-by: Noah Elzner <[email protected]>

| Package | Type | Update | Change | |---|---|---|---| | [github.com/coreos/go-oidc/v3](https://togithub.com/coreos/go-oidc) | require | minor | `v3.5.0` -> `v3.6.0` | | [github.com/go-openapi/swag](https://togithub.com/go-openapi/swag) | require | patch | `v0.22.3` -> `v0.22.4` | | [github.com/sigstore/rekor](https://togithub.com/sigstore/rekor) | require | patch | `v1.2.0` -> `v1.2.1` | | golang.org/x/oauth2 | require | minor | `v0.8.0` -> `v0.9.0` | --------- Signed-off-by: Ian Lewis <[email protected]> Signed-off-by: Noah Elzner <[email protected]>

Signed-off-by: Ian Lewis <[email protected]> Signed-off-by: Noah Elzner <[email protected]>

Signed-off-by: laurentsimon <[email protected]> Signed-off-by: Noah Elzner <[email protected]>

Signed-off-by: Noah Elzner <[email protected]> Signed-off-by: Noah Elzner <[email protected]>

Signed-off-by: Noah Elzner <[email protected]>

laurentsimon

There seems to be changes to files that are out of scope of the PR. Is it a git / merge problem?

Signed-off-by: Noah Elzner <[email protected]>

enteraga6 · 2023-07-08T17:17:08Z

Yes, everything is synced up with the main branch now after resolving the conflicts. Sorry about that.

enteraga6 · 2023-07-11T00:42:50Z

closes #2332

internal/builders/bazel/action.yml

Signed-off-by: Noah Elzner <[email protected]>

@mihaimaruseac

…work#2362) cc: @mihaimaruseac This will provide support for artifacts that also need runfiles. Users can use a flag, `needs-runfiles`, to also package the artifact alongside the runfiles Bazel generates if the user deems it as necessary. Also, support for Java artifacts is made available by packaging the JARs to be standalone through a format in Bazel called `_deploy.jar`. Alongside the `_deploy.jar`, there will be a modified `run-script` that allows the users that download the artifact to run the run-script by using a flag called `local_javabin`, where they put the path to their own java bin such that it is utilized by the run-script. This run-script is generated by Bazel from a template and later modified in the `build.sh` in the internal part of the builder to add this flag for the users. More information is available on the readme. Java targets will automatically be converted to their `_deploy.jar` with this. Three flags are used for users that have java targets: `includes-java`: if true then adds an additional flag to build command as well as rule for local java repo in WORKSPACE in order to utilize the `--singlejar` capability of run-script for `_deploy.jar` such that the remote jdk does not need to be included in runfiles. Doing it like this prevents massive bloat when attesting. `user-java-distribution` and `user-java-version`: let the user specify the exact java they want to use to build When users run the run-script they will include an additional flag `local_javabin` which they will set to their local javabin that the run-script will utilize to run the `_deploy.jar` A combination of `bazel query` and `bazel cquery` were used to resolved edge cases with the implementation. --------- Signed-off-by: Noah Elzner <[email protected]> Signed-off-by: Noah Elzner <[email protected]> Signed-off-by: Ian Lewis <[email protected]> Signed-off-by: laurentsimon <[email protected]> Signed-off-by: laurentsimon <[email protected]> Co-authored-by: Ian Lewis <[email protected]> Co-authored-by: laurentsimon <[email protected]> Signed-off-by: Noah Elzner <[email protected]>

@mihaimaruseac

…work#2362) cc: @mihaimaruseac This will provide support for artifacts that also need runfiles. Users can use a flag, `needs-runfiles`, to also package the artifact alongside the runfiles Bazel generates if the user deems it as necessary. Also, support for Java artifacts is made available by packaging the JARs to be standalone through a format in Bazel called `_deploy.jar`. Alongside the `_deploy.jar`, there will be a modified `run-script` that allows the users that download the artifact to run the run-script by using a flag called `local_javabin`, where they put the path to their own java bin such that it is utilized by the run-script. This run-script is generated by Bazel from a template and later modified in the `build.sh` in the internal part of the builder to add this flag for the users. More information is available on the readme. Java targets will automatically be converted to their `_deploy.jar` with this. Three flags are used for users that have java targets: `includes-java`: if true then adds an additional flag to build command as well as rule for local java repo in WORKSPACE in order to utilize the `--singlejar` capability of run-script for `_deploy.jar` such that the remote jdk does not need to be included in runfiles. Doing it like this prevents massive bloat when attesting. `user-java-distribution` and `user-java-version`: let the user specify the exact java they want to use to build When users run the run-script they will include an additional flag `local_javabin` which they will set to their local javabin that the run-script will utilize to run the `_deploy.jar` A combination of `bazel query` and `bazel cquery` were used to resolved edge cases with the implementation. --------- Signed-off-by: Noah Elzner <[email protected]> Signed-off-by: Noah Elzner <[email protected]> Signed-off-by: Ian Lewis <[email protected]> Signed-off-by: laurentsimon <[email protected]> Signed-off-by: laurentsimon <[email protected]> Co-authored-by: Ian Lewis <[email protected]> Co-authored-by: laurentsimon <[email protected]> Signed-off-by: Noah Elzner <[email protected]>

enteraga6 requested review from asraa, ianlewis, laurentsimon and joshuagl as code owners July 7, 2023 21:32

enteraga6 and others added 26 commits July 7, 2023 21:35

start of cquery to obtain binary

da69e1f

Signed-off-by: Noah Elzner <[email protected]> Signed-off-by: Noah Elzner <[email protected]>

Clean up cquery

9b8cc3f

Signed-off-by: Noah Elzner <[email protected]> Signed-off-by: Noah Elzner <[email protected]>

expand on comment

65b22e6

Signed-off-by: Noah Elzner <[email protected]> Signed-off-by: Noah Elzner <[email protected]>

debug edge

2f01263

Signed-off-by: Noah Elzner <[email protected]> Signed-off-by: Noah Elzner <[email protected]>

update

995922f

Signed-off-by: Noah Elzner <[email protected]> Signed-off-by: Noah Elzner <[email protected]>

debug continue

ee230e8

Signed-off-by: Noah Elzner <[email protected]> Signed-off-by: Noah Elzner <[email protected]>

Implement Associative Array as Set to Avoid Intersection Error of rules

4646353

Signed-off-by: Noah Elzner <[email protected]> Signed-off-by: Noah Elzner <[email protected]>

debug continue

f5d9bd1

Signed-off-by: Noah Elzner <[email protected]> Signed-off-by: Noah Elzner <[email protected]>

flag placement

8c10fe4

Signed-off-by: Noah Elzner <[email protected]> Signed-off-by: Noah Elzner <[email protected]>

debug

e08d324

Signed-off-by: Noah Elzner <[email protected]> Signed-off-by: Noah Elzner <[email protected]>

missing symlink removal flag on cp

b014a6c

Signed-off-by: Noah Elzner <[email protected]> Signed-off-by: Noah Elzner <[email protected]>

debug

9fed360

Signed-off-by: Noah Elzner <[email protected]> Signed-off-by: Noah Elzner <[email protected]>

Update build.sh

e53411f

Signed-off-by: Noah Elzner <[email protected]> Signed-off-by: Noah Elzner <[email protected]>

Update build.sh

3e52bcb

Signed-off-by: Noah Elzner <[email protected]> Signed-off-by: Noah Elzner <[email protected]>

curious

5e50500

Signed-off-by: Noah Elzner <[email protected]> Signed-off-by: Noah Elzner <[email protected]>

cleanup and comment

b37e1b4

Signed-off-by: Noah Elzner <[email protected]> Signed-off-by: Noah Elzner <[email protected]>

update with new cquery functionality

9e27247

Signed-off-by: Noah Elzner <[email protected]> Signed-off-by: Noah Elzner <[email protected]>

fix code quote type

ba078d8

Signed-off-by: Noah Elzner <[email protected]> Signed-off-by: Noah Elzner <[email protected]>

Add issue link for runfile support

c6bb32e

Signed-off-by: Noah Elzner <[email protected]> Signed-off-by: Noah Elzner <[email protected]>

Use lowercase convention and add issue in comment

c18d771

Signed-off-by: Noah Elzner <[email protected]> Signed-off-by: Noah Elzner <[email protected]>

feat: Remove deployment as a supported event type (slsa-framework#2309)

e14541a

Signed-off-by: Ian Lewis <[email protected]> Signed-off-by: Noah Elzner <[email protected]>

feat: sha1 option for checkout (slsa-framework#2326)

8c1be25

Signed-off-by: laurentsimon <[email protected]> Signed-off-by: Noah Elzner <[email protected]>

enteraga6 and others added 14 commits July 7, 2023 21:35

fix flag

27381c2

Signed-off-by: Noah Elzner <[email protected]> Signed-off-by: Noah Elzner <[email protected]>

fix flag syntax

243d496

Signed-off-by: Noah Elzner <[email protected]> Signed-off-by: Noah Elzner <[email protected]>

java constraints document wip

50e442a

Signed-off-by: Noah Elzner <[email protected]> Signed-off-by: Noah Elzner <[email protected]>

cat workspace to check idea

aca4972

Signed-off-by: Noah Elzner <[email protected]> Signed-off-by: Noah Elzner <[email protected]>

workspace --> ./WORKSPACE

86e0ceb

Signed-off-by: Noah Elzner <[email protected]> Signed-off-by: Noah Elzner <[email protected]>

Update build.sh

3effc78

Signed-off-by: Noah Elzner <[email protected]> Signed-off-by: Noah Elzner <[email protected]>

Update README.md

2b52f56

Signed-off-by: Noah Elzner <[email protected]> Signed-off-by: Noah Elzner <[email protected]>

Update README.md

2c1929b

Signed-off-by: Noah Elzner <[email protected]> Signed-off-by: Noah Elzner <[email protected]>

Update README.md

9ed21d5

Signed-off-by: Noah Elzner <[email protected]> Signed-off-by: Noah Elzner <[email protected]>

Update README.md

99f1c32

Signed-off-by: Noah Elzner <[email protected]> Signed-off-by: Noah Elzner <[email protected]>

Update build.sh

3d48831

Signed-off-by: Noah Elzner <[email protected]> Signed-off-by: Noah Elzner <[email protected]>

Update build.sh

e703d7d

Signed-off-by: Noah Elzner <[email protected]> Signed-off-by: Noah Elzner <[email protected]>

wrong quote

41440c1

Signed-off-by: Noah Elzner <[email protected]>

whitespace

be5127a

Signed-off-by: Noah Elzner <[email protected]>

enteraga6 force-pushed the runfile-main branch from 86e9b14 to be5127a Compare July 7, 2023 21:36

laurentsimon reviewed Jul 7, 2023

View reviewed changes

Merge branch 'main' into runfile-main

b9b1818

Signed-off-by: Noah Elzner <[email protected]>

enteraga6 changed the title ~~Runfile and Java Artifact Support~~ Feat: Runfile and java artifact support for Bazel Builder Jul 8, 2023

enteraga6 changed the title ~~Feat: Runfile and java artifact support for Bazel Builder~~ feat: Runfile and java artifact support for Bazel Builder Jul 8, 2023

Merge branch 'main' into runfile-main

d201888

ianlewis approved these changes Jul 11, 2023

View reviewed changes

internal/builders/bazel/action.yml Outdated Show resolved Hide resolved

enteraga6 added 3 commits July 11, 2023 12:03

Un-needed manual export of JAVA_HOME

facd213

Signed-off-by: Noah Elzner <[email protected]>

Merge branch 'main' into runfile-main

13a6984

Merge branch 'main' into runfile-main

4d7fab9

laurentsimon merged commit ba8a119 into slsa-framework:main Jul 13, 2023
74 checks passed

enteraga6 mentioned this pull request Jul 14, 2023

[feature] Validate the targets inputs #2308

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: Runfile and java artifact support for Bazel Builder #2362

feat: Runfile and java artifact support for Bazel Builder #2362

enteraga6 commented Jul 7, 2023 •

edited

Loading

laurentsimon left a comment

enteraga6 commented Jul 8, 2023

enteraga6 commented Jul 11, 2023

feat: Runfile and java artifact support for Bazel Builder #2362

feat: Runfile and java artifact support for Bazel Builder #2362

Conversation

enteraga6 commented Jul 7, 2023 • edited Loading

laurentsimon left a comment

Choose a reason for hiding this comment

enteraga6 commented Jul 8, 2023

enteraga6 commented Jul 11, 2023

enteraga6 commented Jul 7, 2023 •

edited

Loading