Running the controller locally fails on Apple silicon

[stefanprodan]

Running make run on Apple M1 Max fails with:

$ LIBGIT2_FORCE=1 make run
cd api; /Users/stefanprodan/go/bin/controller-gen object:headerFile="../hack/boilerplate.go.txt" paths="./..."
go fmt ./...
cd api; go fmt ./...
PKG_CONFIG_PATH=/Users/stefanprodan/go/src/github.com/fluxcd/source-controller/hack/libgit2/lib/pkgconfig \
go vet ./...
cd api; go vet ./...
/Users/stefanprodan/go/bin/controller-gen crd:crdVersions=v1 rbac:roleName=manager-role paths="./..." output:crd:artifacts:config="config/crd/bases"
cd api; /Users/stefanprodan/go/bin/controller-gen crd:crdVersions=v1 rbac:roleName=manager-role paths="./..." output:crd:artifacts:config="../config/crd/bases"
LD_LIBRARY_PATH=/Users/stefanprodan/go/src/github.com/fluxcd/source-controller/hack/libgit2/lib \
go run ./main.go
dyld[77871]: Library not loaded: @rpath/libgit2.1.1.dylib
  Referenced from: /private/var/folders/fm/v67j6c7n1k5g46m38yb15yvm0000gn/T/go-build707344674/b001/exe/main
  Reason: tried: '/usr/local/lib/libgit2.1.1.dylib' (no such file), '/usr/lib/libgit2.1.1.dylib' (no such file)
signal: abort trap
make: *** [Makefile:70: run] Error 1

Configuration:

$ cat ~/.bash_profile
export PATH="/opt/homebrew/opt/make/libexec/gnubin:$PATH"
export PATH="/opt/homebrew/opt/[email protected]/bin:$PATH"
export LDFLAGS="-L/opt/homebrew/opt/[email protected]/lib"
export CPPFLAGS="-I/opt/homebrew/opt/[email protected]/include"
export PKG_CONFIG_PATH="/opt/homebrew/opt/[email protected]/lib/pkgconfig"

$ openssl version
OpenSSL 1.1.1l  24 Aug 2021

$ go version
go version go1.16.10 darwin/arm64

$ uname -av
Darwin Kernel Version 21.1.0: Wed Oct 13 17:33:01 PDT 2021; root:xnu-8019.41.5~1/RELEASE_ARM64_T6000 arm64

I've found no way to run the controller locally on Apple M1, my current workaround is to use Docker for Mac:

# create a linux/arm64 cluster
kind create cluster

# install Flux using linux/arm64 images
flux install

# build source-controller linux/arm64 image
BUILD_PLATFORMS=linux/arm64 BUILD_ARGS=--load make docker-build

# load the image in kind
kind load docker-image fluxcd/source-controller:latest

# set the image in-cluster
kubectl -n flux-system set image deployment/source-controller manager=fluxcd/source-controller:latest

# verify it works
flux check

[stefanprodan]

I manage to run the controller locally using Go 1.17 and copying libgit2.1.1.dylib to /usr/local/lib/:

$ go version
go version go1.17.3 darwin/arm64

$ LIBGIT2_FORCE=1 make libgit2

$ sudo cp hack/libgit2/lib/libgit2.1.1.1.dylib /usr/local/lib/libgit2.1.1.dylib

$ LIBGIT2_FORCE=1 make test
ok      github.com/fluxcd/source-controller/pkg/git/libgit2     0.651s  coverage: 75.4% of statements

$ make run
{"level":"info","ts":"2021-12-04T12:38:49.655+0200","logger":"setup","msg":"starting manager"}

[squaremo]

What is it that points at "Apple Silicon", rather than just your computer specifically? Does it fail for anyone else?

[stefanprodan]

Does it fail for anyone else?

@aryan9600 has the same issue, so I guess it's not my computer.

[squaremo]

OK, and is there evidence that it's the arch, and not e.g., the OS version?

[stefanprodan]

My old macbook with Intel used to work (same OS version) but on that machine I've pinned libgit2 to the version that Flux expects.

[ethomson]

👋 I hate to throw in what is likely a distraction to an issue that I'm super unfamiliar with, but despite that: the @libgit2 project would really encourage you not to use OpenSSL on macOS/Darwin. We'll support it, and we feel good about its quality to the extent that we feel good about OpenSSL on any platform, but we're a lot happier with Apple's TLS APIs and would encourage you to use them instead of OpenSSL if they're available.

[cwyl02]

make test failed on my Mac laptop as well.

make test                                                                            
cd api; go test ./... -coverprofile cover.out
?       github.com/fluxcd/source-controller/api/v1beta1 [no test files]
LD_LIBRARY_PATH=/Users/ychen/workspace/fluxcd/source-controller/hack/libgit2/lib \
PKG_CONFIG_PATH=/Users/ychen/workspace/fluxcd/source-controller/hack/libgit2/lib/pkgconfig/ \
go test ./... -coverprofile cover.out
# pkg-config --cflags  -- libgit2
Package openssl was not found in the pkg-config search path.
Perhaps you should add the directory containing `openssl.pc'
to the PKG_CONFIG_PATH environment variable
Package 'openssl', required by 'libgit2', not found
pkg-config: exit status 1
FAIL    github.com/fluxcd/source-controller/controllers [build failed]
ok      github.com/fluxcd/source-controller/internal/fs 0.508s  coverage: 44.4% of statements
ok      github.com/fluxcd/source-controller/internal/helm/chart 1.170s  coverage: 85.9% of statements
ok      github.com/fluxcd/source-controller/internal/helm/getter        0.441s  coverage: 73.1% of statements
ok      github.com/fluxcd/source-controller/internal/helm/repository    0.812s  coverage: 85.0% of statements
ok      github.com/fluxcd/source-controller/pkg/gcp     0.184s  coverage: 76.5% of statements
ok      github.com/fluxcd/source-controller/pkg/git     0.328s  coverage: 100.0% of statements
ok      github.com/fluxcd/source-controller/pkg/git/gogit       1.260s  coverage: 76.0% of statements
FAIL    github.com/fluxcd/source-controller/pkg/git/libgit2 [build failed]
FAIL    github.com/fluxcd/source-controller/pkg/git/strategy [build failed]
ok      github.com/fluxcd/source-controller/pkg/sourceignore    0.432s  coverage: 90.7% of statements
make: *** [Makefile:62: test] Error 2

my system info:
MacOS 11.6.1
Intel CPU

[stefanprodan]

@cwyl02 please try #515 it should fix it for Intel too.

[cwyl02]

that PR works for me. thanks @stefanprodan !!

[hiddeco]

@ethomson thanks for the heads up. 🙇

I can assure (and comfort) you with the message that this configuration is used only for development environments, in which we specifically have an alignment because support for certain configurations is heavily dependent on the underlying crypto library, which is important to the core business of our Git related operations.

The application itself is only shipped on Linux, in a container image.

[ethomson]

I can assure (and comfort) you with the message that this configuration is used only for development environments, in which we specifically have an alignment because support for certain configurations is heavily dependent on the underlying crypto library, which is important to the core business of our Git related operations.

This makes a lot of sense. We don't do a ton of testing on this configuration (macOS + OpenSSL) so I might add a nightly build configuration option for that. 🤔

Thanks for your kindness and humoring me when I dive into your threads without any background. 😁