Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Properly catch errors in ldflag-gathering and fail the build #5539

Merged
merged 18 commits into from
Jan 8, 2024

Conversation

Groxx
Copy link
Contributor

@Groxx Groxx commented Dec 23, 2023

The dubious-ownership issue in CI appears to be happening here too,
and might be why some builds have "unknown" as their commit SHAs.

Building this change locally properly fails to build binaries when
it cannot be gathered, and includes the data when it can:

VERSION:
   CLI feature version: 1.7.0
   Release version: v1.2.7-prerelease2-dirty
   Build commit: 2023-12-22T20:30:21+00:00-d4d205fe3

Previously there were basically three issues at play:

  • git's new safe.directory thing, now applied everywhere rather than just one target
  • the in-variable-expansion shell error would be swallowed and
    (AFAICT) be completely undetectable elsewhere in Make.
  • SC2155 and similar "you captured output but you also suppressed the error code" issues in scripts.

Since fixing the "always include the version, fail build if it does not work" issue
involved discovering and fixing all three, they're all in this PR.

We should seriously consider adding shellcheck to our CI pipeline.

The dubious-ownership issue in CI appears to be happening here too,
and might be why some builds have "unknown" as their commit SHAs.

Building this locally properly fails to build binaries when it cannot
be gathered, and includes the data when it can:
```
VERSION:
   CLI feature version: 1.7.0
   Release version: v1.2.7-prerelease2-dirty
   Build commit: 2023-12-22T20:30:21+00:00-d4d205fe3
```
Copy link
Contributor

@taylanisikdemir taylanisikdemir left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM with comment

@Groxx
Copy link
Contributor Author

Groxx commented Jan 6, 2024

hmmm

++ ./scripts/get-ldflags.sh LDFLAG
fatal: detected dubious ownership in repository at '/cadence'
To add an exception for this directory, call:
	git config --global --add safe.directory /cadence
fatal: detected dubious ownership in repository at '/cadence'
To add an exception for this directory, call:
	git config --global --add safe.directory /cadence
+ FLAGS='-X github.com/uber/cadence/common/metrics.Revision= -X github.com/uber/cadence/common/metrics.Branch= -X github.com/uber/cadence/common/metrics.ReleaseVersion=unknown -X github.com/uber/cadence/common/metrics.BuildDate=2024-01-06-00:04:12 -X github.com/uber/cadence/common/metrics.BuildTimeUnix=1704499452'
+ exec go build -ldflags '-X github.com/uber/cadence/common/metrics.Revision= -X github.com/uber/cadence/common/metrics.Branch= -X github.com/uber/cadence/common/metrics.ReleaseVersion=unknown -X github.com/uber/cadence/common/metrics.BuildDate=2024-01-06-00:04:12 -X github.com/uber/cadence/common/metrics.BuildTimeUnix=1704499452' -o cadence-cassandra-tool cmd/tools/cassandra/main.go

apparently there are still gaps in that. gonna hunt for a higher-level place to do it.

@Groxx
Copy link
Contributor Author

Groxx commented Jan 6, 2024

much better:

+ FLAGS='-X github.com/uber/cadence/common/metrics.Revision=2024-01-05T18:34:15-06:00-07165ee57 -X github.com/uber/cadence/common/metrics.Branch=HEAD -X github.com/uber/cadence/common/metrics.ReleaseVersion=v1.2.7-prerelease2 -X github.com/uber/cadence/common/metrics.BuildDate=2024-01-06-00:40:23 -X github.com/uber/cadence/common/metrics.BuildTimeUnix=1704501623'

gonna clean up a bit.

though hmmmm. it should have failed before. checking further...

@Groxx
Copy link
Contributor Author

Groxx commented Jan 6, 2024

dangit.

Declare and assign separately to avoid masking return values.
See SC2155.

so we should probably just add shellcheck.

@Groxx
Copy link
Contributor Author

Groxx commented Jan 6, 2024

dunno what this implies:

Unable to find image 'koalaman/shellcheck:stable' locally
stable: Pulling from koalaman/shellcheck
ee9c4b55cadb: Pull complete
7d79c83e79c5: Pull complete
Digest: sha256:f35e8987b02760d4e76fc99a68ad5c42cc10bb32f3dd2143a3cf92f1e5446a45
Status: Downloaded newer image for koalaman/shellcheck:stable
Unknown shell: warning
🚨 Error: The command exited with status 1
user command error: The plugin shellcheck command hook exited with status 1

and going to ignore it for now. make lint finds this stuff at least, we can address it later.

# that's fine, just override it if it looks like we're in buildkite.
# elsewhere it's probably best to not touch this, and the path is likely wrong.
if [ -n "$BUILDKITE" ]; then
git config --global --add safe.directory /cadence
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is now done in the buildkite dockerfile, which appears to catch everything much more simply.

scripts/get-ldflags.sh Outdated Show resolved Hide resolved
scripts/get-ldflags.sh Outdated Show resolved Hide resolved
@@ -130,11 +130,11 @@ wait_for_postgres() {
wait_for_es() {
server=`echo $ES_SEEDS | awk -F ',' '{print $1}'`
URL="http://$server:$ES_PORT"
curl -s $URL 2>&1 > /dev/null
curl -s $URL > /dev/null 2>&1
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TIL. I've gotta adjust my muscle memory apparently.

@Groxx Groxx merged commit a8921b2 into uber:master Jan 8, 2024
16 checks passed
@Groxx Groxx deleted the ldflags branch January 8, 2024 19:46
Groxx added a commit to Groxx/cadence that referenced this pull request Jan 9, 2024
uber#5539 / commit a8921b2 unfortunately has a bad bit of copypasta that changed the `cadence-server` build to make the CLI.
This restores that makefile target to what it should be.

tbh I'm not quite sure how this got past CI.  Maybe we don't actually run the binary anywhere as part of an E2E test?
It is fairly quick to notice at least though, since it causes problems quickly.
Groxx added a commit that referenced this pull request Jan 9, 2024
#5539 / commit a8921b2 unfortunately has a bad bit of copypasta that changed the `cadence-server` build to make the CLI.
This restores that makefile target to what it should be.

tbh I'm not quite sure how this got past CI.  Maybe we don't actually run the binary anywhere as part of an E2E test?
It is fairly quick to notice at least though, since it causes problems quickly.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants