Following guide will help you decide what to test and how to go about it when changing code in this repository.
In KIC, we use several levels of testing:
Unit tests verify the functionality of one or more func
or struct
s
in an isolated environment.
These tests are independent of the environment they are run in and require no setup or teardown code.
NOTE: Some of these tests do require
envtest
setup to be present e.g. this one or this one. This setup is handled automatically viasetup-envtest
Makefile target which is a dependency of mentioned above unit test targets.As part of #4099 this organization might change and tests that do require this setup might end up in a separate place.
These tests can either be written in a way that do not require any mocks or fakes,
or using e.g. Kubernetes related fakes like for instance
TestAddressesFromEndpointSlice
does.
Unit tests can be either run via appropriate go test ...
invocation or using any of
the provided Makefile targets:
test.unit
run unit tests with standard verbose output on stderrtest.unit.pretty
run unit tests with opinionated output format, one line per package
Unit tests are typically located in the same package as the code they test and they do not use any build tags.
envtest
based tests use a middle ground approach, somewhere between
unit and integration tests.
They run tests within an isolated, limited environment which allows some basic
Kubernetes cluster actions like CRUD operations on Kubernetes entities and running
controllers against this environment.
Ensuring that the environment is ready for tests is done by:
setup-envtest
which ensures that api-server and etcd binaries are present on the local systemMakefile
invocation ofsetup-envtest use
which does what's described above
There already exist several helpers for working with envtest
environment which
can be found at test/envtest/
.
NOTE: Currently, these tests that do require
envtest
setup and use a special build tags:envtest
. This is in place so that simply runninggo test ...
without any special build tags would pass without requiring any manual setup from the developer (no failure due to missingenvtest
related binaries or environment variables).
Test definitions can be found both in dedicated directory - test/envtest
- and
throughout the codebase next to code that they are testing.
This approach however may change over time.
Suggestion for new envtest
based tests is to be placed in test/envtest
and to
test code as a black box, i.e. using only exported methods (which will be enforced
via running them from this dedicated package).
envtest
based tests can be run via test.envtest
Makefile
target.
One can also use ./bin/setup-envtest use -p env
to obtain the asset dir environment
variable:
./bin/setup-envtest use -p env
export KUBEBUILDER_ASSETS='/Users/username/Library/Application Support/io.kubebuilder.envtest/k8s/1.27.1-darwin-arm64'
and use that to manually run the tests:
$ eval $(./bin/setup-envtest use -p env)
$ go test -v -count 1 -tags envtest ./pkg/to/test
...
KIC's integration tests rely on its controller manager being run in the same
process as the tests via manager.Run()
.
These tests rely on a test suite setup
using TestMain()
.
Said setup will either create a new kind cluster or use an existing one to run the tests against.
Most of the setup - and cleanup after the tests have run - is being done using ktf.
Currently, the cluster is being shared across all the tests that are a part of the test suite so special care needs to be taken in order to clean up after the tests that run. The typical approach is to run them in a dedicated, disposable Kubernetes namespace created just for the purposes of that test.
Integration tests are located under tests/integration/
and use integration_tests
build tag.
You can run them using one of the dedicated Makefile targets:
-
test.integration
run all integration tests with standard verbose output on stderr. This will run tests for dbless, postgres, enterprise and non enterprise. -
test.integration.dbless
run all dbless integration tests with standard verbose output on stderr. The output will also include controllers' logs. -
test.integration.postgres
run all postgres integration tests with standard verbose output on stderr. The output will also include controllers' logs.
Through GOTESTFLAGS
you can specify custom flags that will be passed to go test
.
This can allow you to run a subset of all the tests for faster feedback times, e.g.:
make test.integration.dbless GOTESTFLAGS="-count 1 -run TestUDPRouteEssentials"
Tests located under test/kongintegration/
allow verifying individual components of KIC
against a running Kong Gateway instance.
Every test in this package requires a Kong instance to be running, but do not require a Kubernetes cluster nor a full KIC deployment. Each test is responsible for setting up its own Kong instance and cleaning it up after the test is finished. testcontainers-go is used for that purpose.
Examples of tests that could fit into this category are:
- Ensuring that a component responsible for configuring Kong Gateway works as expected
- Ensuring that configuration generated by a translator is accepted by Kong Gateway
The only requirement for running Kong integration tests is to have a docker daemon running.
All tests can be run using the following Makefile targets:
# Verbose output
make test.kongintegration
# Pretty output
make test.kongintegration.pretty
By default, kong:latest
image is used for running Kong Gateway. This can be changed
by setting the TEST_KONG_IMAGE
and TEST_KONG_TAG
environment variables.
In case your environment doesn't allow running containers in privileged mode, you may encounter
issues with testcontainer's garbage collector (Ryuk) which requires it. In
Makefile targets we disable the garbage collector by setting TESTCONTAINERS_RYUK_DISABLED
environment
variable to true
.
If you encounter issues with this while running tests directly (e.g. when debugging in an IDE), you can fix it by:
- setting
TESTCONTAINERS_RYUK_DISABLED
environment variable totrue
in your IDE configuration, - adding
ryuk.disabled=true
to.testcontainers.properties
file in your home directory (see Configuration locations for exact locations depending on your OS).
End to end tests in KIC are used to test features or use cases which rely on KIC being deployed in-cluster.
For instance:
- deploying all in one manifests tested in all in one tests
- upgrade scenarios tested in upgrade tests
These tests deploy KIC and Kong Gateway in a cluster using the requested image(s) which could be customized via dedicated environment variables like:
On CI, those are being run both in kind and GKE environments.
E2E tests are located under tests/e2e/
and use e2e_tests
build tag.
You can run them using one of the dedicated Makefile targets:
test.e2e
run all e2e tests.
Through GOTESTFLAGS
you can specify custom flags that will be passed to go test
.
E2E_TEST_RUN
is also available to specify the name of the test to run.
This is being used in CI where Github Actions' matrix specifies tests to run
in each workflow.
Exemplar local invocation can look like this:
make test.e2e GOTESTFLAGS="-count 1" E2E_TEST_RUN=TestDeployAllInOneDBLESS
Some of the E2E tests are Konnect-specific and are being run only when the TEST_KONG_KONNECT_ACCESS_TOKEN
environment variable is set. You can create the access token using the Konnect UI.
The tests run against a .tech
Konnect environment therefore the token you obtain must be created for an account
in that environment.
Exemplary local invocation for running a Konnect test can look like this:
make test.e2e GOTESTFLAGS="-count 1" E2E_TEST_RUN=TestKonnectConfigPush TEST_KONG_KONNECT_ACCESS_TOKEN=<token>