feat: generate sboms from binaries (#46)

* add bin command Signed-off-by: nscuro <[email protected]> * add license header Signed-off-by: nscuro <[email protected]> * basic module model + parsing of `go version -m` Signed-off-by: nscuro <[email protected]> * introduce convert package this is the first step of splitting up the sbom package to become more generic. Signed-off-by: nscuro <[email protected]> * convert components in bin command Signed-off-by: nscuro <[email protected]> * add license header Signed-off-by: nscuro <[email protected]> * remove model package Signed-off-by: nscuro <[email protected]> * generate dependency graph Signed-off-by: nscuro <[email protected]> * add compositions; fix depgraph; decode h1 hash; Signed-off-by: nscuro <[email protected]> * add -version flag; add example sbom Signed-off-by: nscuro <[email protected]> * implement generic way of calculating file hashes Signed-off-by: nscuro <[email protected]> * refactor for more code reuse Signed-off-by: nscuro <[email protected]> * fix feature toggle for license resolution Signed-off-by: nscuro <[email protected]> * add more tests Signed-off-by: nscuro <[email protected]> * add binary metadata as properties Signed-off-by: nscuro <[email protected]> * migrate generation logic from sbom to mod package Signed-off-by: nscuro <[email protected]> * move serialnumber handling to cliutil Signed-off-by: nscuro <[email protected]> * more tests Signed-off-by: nscuro <[email protected]> * documentation updates Signed-off-by: nscuro <[email protected]> * fix linter issues Signed-off-by: nscuro <[email protected]> * support replacements Signed-off-by: nscuro <[email protected]> * move common logic into util package Signed-off-by: nscuro <[email protected]> * replace cyclonedx-gomod example sbom with minikube Signed-off-by: nscuro <[email protected]> * refactor: minor tweaks and optimizations Signed-off-by: nscuro <[email protected]> * introduce structured logging Signed-off-by: nscuro <[email protected]> * use str field instead of format string Signed-off-by: nscuro <[email protected]> * don't include caller in debug log Signed-off-by: nscuro <[email protected]> * more logging Signed-off-by: nscuro <[email protected]> * usage examples for `mod` Signed-off-by: nscuro <[email protected]> * add license headers Signed-off-by: nscuro <[email protected]> * more logging Signed-off-by: nscuro <[email protected]> * module sorting Signed-off-by: nscuro <[email protected]> * move compositions creation into its own function Signed-off-by: nscuro <[email protected]> * minor refactoring Signed-off-by: nscuro <[email protected]> * cleanup & add more tests Signed-off-by: nscuro <[email protected]> * remove unneeded test workaround Signed-off-by: nscuro <[email protected]> * move integration tests to e2e package Signed-off-by: nscuro <[email protected]> * simple e2e test for bin cmd Signed-off-by: nscuro <[email protected]> * regenerate example sboms; replace proton-bridge with minikube Signed-off-by: nscuro <[email protected]>
CycloneDX · Aug 9, 2021 · 2b197e4 · 2b197e4
1 parent 4c612b1
commit 2b197e4
Show file tree

Hide file tree

Showing 50 changed files with 8,824 additions and 2,520 deletions.
diff --git a/README.md b/README.md
@@ -26,64 +26,80 @@ Building from source requires Go 1.16 or newer.
 ## Compatibility
 
 *cyclonedx-gomod* will produce BOMs for the latest version of the CycloneDX specification 
-[supported by cyclonedx-go](https:/CycloneDX/cyclonedx-go#compatibility), which currently is [1.2](https://cyclonedx.org/docs/1.2/). 
+[supported by cyclonedx-go](https:/CycloneDX/cyclonedx-go#compatibility), which currently is [1.3](https://cyclonedx.org/docs/1.3/). 
 You can use the [CycloneDX CLI](https:/CycloneDX/cyclonedx-cli#convert-command) to convert between multiple 
 BOM formats or specification versions. 
 
-## Accuracy
-
-Currently, SBOMs generated with *cyclonedx-gomod* are completely module-based. 
+## Usage
 
-What does this mean? Well, modules in Go can consist of multiple *commands* or *applications*.
-For example, [`k8s.io/minikube`](https:/kubernetes/minikube/blob/master/go.mod) is a module, but it contains [multiple commands](https:/kubernetes/minikube/tree/master/cmd). 
-Each of these commands is eventually compiled into its own binary. Most likely, each command only depends on a subset of the dependencies defined in the module's `go.mod`.
+```
+USAGE
+ cyclonedx-gomod <SUBCOMMAND> [FLAGS...] [<ARG>...]
 
-Additionally, some dependencies may only be required when a given build constraint is in place.
-Build constraints can include the operating system (`GOOS`), the architecture (`GOARCH`) or build tags.
-As an example, [`github.com/Microsoft/go-winio`](https:/microsoft/go-winio) provides Windows-specific
-functionality and won't be included in builds that target Linux or macOS.
+SUBCOMMANDS
+ bin Generate SBOM for a binary
+ mod Generate SBOM for a module
+ version Show version information
+```
 
-*cyclonedx-gomod* describes the module, not commands or binaries. See also the discussion in [#20](https:/CycloneDX/cyclonedx-gomod/issues/20).
+### Subcommands
 
-We're in the process of adding support for generating command- or binary-specific SBOMs as well. Stay tuned!
-
-## Usage
+#### `bin`
 
 ```
-Usage of cyclonedx-gomod:
- -json
- Output in JSON format
- -licenses
- Resolve module licenses
- -module string
- Path to Go module (default ".")
- -noserial
- Omit serial number
- -novprefix
- Omit "v" version prefix
- -output string
- Output path (default "-")
- -reproducible
- Make the SBOM reproducible by omitting dynamic content
- -serial string
- Serial number (default [random UUID])
- -std
- Include Go standard library as component and dependency of the module
- -test
- Include test dependencies
- -type string
- Type of the main component (default "application")
- -version
- Show version
+USAGE
+ cyclonedx-gomod bin [FLAGS...] PATH
+
+Generate SBOM for a binary.
+
+Please note that data embedded in binaries shouldn't be trusted,
+unless there's solid evidence that the binaries haven't been modified
+since they've been built.
+
+Example:
+ $ cyclonedx-gomod bin -json -output minikube-v1.22.0.bom.json -version v1.22.0 ./minikube
+
+FLAGS
+ -json=false Output in JSON
+ -noserial=false Omit serial number
+ -novprefix=false Omit "v" prefix from versions
+ -output - Output file path (or - for STDOUT)
+ -reproducible=false Make the SBOM reproducible by omitting dynamic content
+ -serial ... Serial number
+ -std=false Include Go standard library as component and dependency of the module
+ -verbose=false Enable verbose output
+ -version ... Version of the main component
 ```
 
-### Example
+#### `mod`
 
-```shell
-$ cyclonedx-gomod -licenses -std -output bom.xml
 ```
+USAGE
+ cyclonedx-gomod mod [FLAGS...] [PATH]
+
+Generate SBOM for a module.
+
+Examples:
+ $ cyclonedx-gomod mod -licenses -type library -json -output bom.json ./cyclonedx-go
+ $ cyclonedx-gomod mod -reproducible -test -output bom.xml ./cyclonedx-go
+
+FLAGS
+ -json=false Output in JSON
+ -licenses=false Resolve module licenses
+ -noserial=false Omit serial number
+ -novprefix=false Omit "v" prefix from versions
+ -output - Output file path (or - for STDOUT)
+ -reproducible=false Make the SBOM reproducible by omitting dynamic content
+ -serial ... Serial number
+ -std=false Include Go standard library as component and dependency of the module
+ -test=false Include test dependencies
+ -type application Type of the main component
+ -verbose=false Enable verbose output
+```
+
+### Examples
 
-Checkout the [`examples`](./examples) directory for examples of BOMs generated by *cyclonedx-gomod*.
+Checkout the [`examples`](./examples) directory for examples of SBOMs generated with *cyclonedx-gomod*.
 
 ### GitHub Actions 🤖
 

diff --git a/e2e/cmd_bin_test.go b/e2e/cmd_bin_test.go
@@ -0,0 +1,21 @@
+package e2e
+
+import (
+ "testing"
+
+ bincmd "github.com/CycloneDX/cyclonedx-gomod/internal/cli/cmd/bin"
+ "github.com/CycloneDX/cyclonedx-gomod/internal/cli/options"
+)
+
+func TestBinCmdSimple(t *testing.T) {
+ binOptions := bincmd.BinOptions{
+ SBOMOptions: options.SBOMOptions{
+ Reproducible: true,
+ SerialNumber: zeroUUID.String(),
+ },
+ BinaryPath: "./testdata/bincmd/simple",
+ Version: "v1.0.0",
+ }
+
+ runSnapshotIT(t, &binOptions.OutputOptions, func() error { return bincmd.Exec(binOptions) })
+}
diff --git a/e2e/cmd_mod_test.go b/e2e/cmd_mod_test.go
@@ -0,0 +1,130 @@
+// This file is part of CycloneDX GoMod
+//
+// Licensed under the Apache License, Version 2.0 (the “License”);
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an “AS IS” BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+//
+// SPDX-License-Identifier: Apache-2.0
+// Copyright (c) OWASP Foundation. All Rights Reserved.
+
+package e2e
+
+import (
+ "os"
+ "path/filepath"
+ "testing"
+
+ cdx "github.com/CycloneDX/cyclonedx-go"
+ modcmd "github.com/CycloneDX/cyclonedx-gomod/internal/cli/cmd/mod"
+ "github.com/CycloneDX/cyclonedx-gomod/internal/cli/options"
+)
+
+// Integration test with a "simple" module with only a few dependencies,
+// no replacements and no vendoring.
+func TestModCmdSimple(t *testing.T) {
+ fixturePath := extractFixture(t, "./testdata/modcmd/simple.tar.gz")
+ defer os.RemoveAll(fixturePath)
+
+ modOptions := modcmd.ModOptions{
+ SBOMOptions: options.SBOMOptions{
+ Reproducible: true,
+ SerialNumber: zeroUUID.String(),
+ },
+ ComponentType: string(cdx.ComponentTypeLibrary),
+ ModuleDir: fixturePath,
+ ResolveLicenses: true,
+ }
+
+ runSnapshotIT(t, &modOptions.OutputOptions, func() error { return modcmd.Exec(modOptions) })
+}
+
+// Integration test with a module that uses replacement with a local module.
+// The local dependency is not a Git repository and thus won't have a version.
+func TestModCmdLocal(t *testing.T) {
+ fixturePath := extractFixture(t, "./testdata/modcmd/local.tar.gz")
+ defer os.RemoveAll(fixturePath)
+
+ modOptions := modcmd.ModOptions{
+ SBOMOptions: options.SBOMOptions{
+ Reproducible: true,
+ SerialNumber: zeroUUID.String(),
+ },
+ ComponentType: string(cdx.ComponentTypeLibrary),
+ ModuleDir: filepath.Join(fixturePath, "local"),
+ ResolveLicenses: true,
+ }
+
+ runSnapshotIT(t, &modOptions.OutputOptions, func() error { return modcmd.Exec(modOptions) })
+}
+
+// Integration test with a module that doesn't have any dependencies.
+func TestModCmdNoDependencies(t *testing.T) {
+ fixturePath := extractFixture(t, "./testdata/modcmd/no-dependencies.tar.gz")
+ defer os.RemoveAll(fixturePath)
+
+ modOptions := modcmd.ModOptions{
+ SBOMOptions: options.SBOMOptions{
+ Reproducible: true,
+ SerialNumber: zeroUUID.String(),
+ },
+ ComponentType: string(cdx.ComponentTypeLibrary),
+ ModuleDir: fixturePath,
+ ResolveLicenses: true,
+ }
+
+ runSnapshotIT(t, &modOptions.OutputOptions, func() error { return modcmd.Exec(modOptions) })
+}
+
+// Integration test with a "simple" module with only a few dependencies,
+// no replacements, but vendoring.
+func TestModCmdVendored(t *testing.T) {
+ fixturePath := extractFixture(t, "./testdata/modcmd/vendored.tar.gz")
+ defer os.RemoveAll(fixturePath)
+
+ modOptions := modcmd.ModOptions{
+ SBOMOptions: options.SBOMOptions{
+ Reproducible: true,
+ SerialNumber: zeroUUID.String(),
+ },
+ ComponentType: string(cdx.ComponentTypeLibrary),
+ ModuleDir: fixturePath,
+ ResolveLicenses: true,
+ }
+
+ runSnapshotIT(t, &modOptions.OutputOptions, func() error { return modcmd.Exec(modOptions) })
+}
+
+// Integration test with a "simple" module with only a few dependencies,
+// but as a subdirectory of a Git repository. The expectation is that the
+// (pseudo-) version is inherited from the repository of the parent dir.
+//
+// nested/
+// |-+ .git/
+// |-+ simple/
+// |-+ go.mod
+// |-+ go.sum
+// |-+ main.go
+func TestModCmdNested(t *testing.T) {
+ fixturePath := extractFixture(t, "./testdata/modcmd/nested.tar.gz")
+ defer os.RemoveAll(fixturePath)
+
+ modOptions := modcmd.ModOptions{
+ SBOMOptions: options.SBOMOptions{
+ Reproducible: true,
+ SerialNumber: zeroUUID.String(),
+ },
+ ComponentType: string(cdx.ComponentTypeLibrary),
+ ModuleDir: filepath.Join(fixturePath, "simple"),
+ ResolveLicenses: true,
+ }
+
+ runSnapshotIT(t, &modOptions.OutputOptions, func() error { return modcmd.Exec(modOptions) })
+}
diff --git a/e2e/cmd_test.go b/e2e/cmd_test.go
@@ -0,0 +1,88 @@
+package e2e
+
+import (
+ "fmt"
+ "os"
+ "os/exec"
+ "strings"
+ "testing"
+
+ "github.com/CycloneDX/cyclonedx-gomod/internal/cli/options"
+ "github.com/CycloneDX/cyclonedx-gomod/internal/version"
+ "github.com/bradleyjkemp/cupaloy/v2"
+ "github.com/google/uuid"
+ "github.com/stretchr/testify/assert"
+ "github.com/stretchr/testify/require"
+)
+
+var (
+ snapshotter = cupaloy.NewDefaultConfig().
+ WithOptions(cupaloy.SnapshotSubdirectory("./testdata/snapshots"))
+
+ // Prefix for temporary files and directories created during ITs
+ tmpPrefix = version.Name + "_"
+
+ // Serial number to use in order to keep generated SBOMs reproducible
+ zeroUUID = uuid.MustParse("00000000-0000-0000-0000-000000000000")
+)
+
+func runSnapshotIT(t *testing.T, outputOptions *options.OutputOptions, execFunc func() error) {
+ skipIfShort(t)
+
+ bomFileExtension := ".xml"
+ if outputOptions.UseJSON {
+ bomFileExtension = ".json"
+ }
+
+ // Create a temporary file to write the SBOM to
+ bomFile, err := os.CreateTemp("", tmpPrefix+t.Name()+"_*.bom"+bomFileExtension)
+ require.NoError(t, err)
+ defer os.Remove(bomFile.Name())
+ require.NoError(t, bomFile.Close())
+
+ // Generate the SBOM
+ outputOptions.OutputFilePath = bomFile.Name()
+ err = execFunc()
+ require.NoError(t, err)
+
+ // Sanity check: Make sure the SBOM is valid
+ assertValidSBOM(t, bomFile.Name())
+
+ // Read SBOM and compare with snapshot
+ bomFileContent, err := os.ReadFile(bomFile.Name())
+ require.NoError(t, err)
+ snapshotter.SnapshotT(t, string(bomFileContent))
+}
+
+func skipIfShort(t *testing.T) {
+ if testing.Short() {
+ t.Skip("skipping integration test in short mode")
+ }
+}
+
+func assertValidSBOM(t *testing.T, bomFilePath string) {
+ inputFormat := "xml_v1_3"
+ if strings.HasSuffix(bomFilePath, ".json") {
+ inputFormat = "json_v1_3"
+ }
+ valCmd := exec.Command("cyclonedx", "validate", "--input-file", bomFilePath, "--input-format", inputFormat, "--fail-on-errors")
+ valOut, err := valCmd.CombinedOutput()
+ if !assert.NoError(t, err) {
+ // Provide some context when test is failing
+ fmt.Printf("validation error: %s\n", string(valOut))
+ }
+}
+
+func extractFixture(t *testing.T, archivePath string) string {
+ tmpDir, err := os.MkdirTemp("", tmpPrefix+t.Name()+"_*")
+ require.NoError(t, err)
+
+ cmd := exec.Command("tar", "xzf", archivePath, "-C", tmpDir)
+ out, err := cmd.CombinedOutput()
+ if !assert.NoError(t, err) {
+ // Provide some context when test is failing
+ fmt.Printf("validation error: %s\n", string(out))
+ }
+
+ return tmpDir
+}
diff --git a/e2e/testdata/bincmd/simple b/e2e/testdata/bincmd/simple
diff --git a/...nal/cli/testdata/integration/local.tar.gz → e2e/testdata/modcmd/local.tar.gz b/...nal/cli/testdata/integration/local.tar.gz → e2e/testdata/modcmd/local.tar.gz
diff --git a/...al/cli/testdata/integration/nested.tar.gz → e2e/testdata/modcmd/nested.tar.gz b/...al/cli/testdata/integration/nested.tar.gz → e2e/testdata/modcmd/nested.tar.gz
diff --git a/...stdata/integration/no-dependencies.tar.gz → e2e/testdata/modcmd/no-dependencies.tar.gz b/...stdata/integration/no-dependencies.tar.gz → e2e/testdata/modcmd/no-dependencies.tar.gz
diff --git a/...al/cli/testdata/integration/simple.tar.gz → e2e/testdata/modcmd/simple.tar.gz b/...al/cli/testdata/integration/simple.tar.gz → e2e/testdata/modcmd/simple.tar.gz
diff --git a/.../cli/testdata/integration/vendored.tar.gz → e2e/testdata/modcmd/vendored.tar.gz b/.../cli/testdata/integration/vendored.tar.gz → e2e/testdata/modcmd/vendored.tar.gz