grpc.kafka.echo

grpc.kafka.echo

Listens on https port 7153 and will exchange grpc message in protobuf format through the echo-messages topic in Kafka.

Requirements

• bash, jq, nc, grpcurl
• Kubernetes (e.g. Docker Desktop with Kubernetes enabled)
• kubectl
• helm 3.0+
• ghz

Setup

The setup.sh script:

• installs Zilla and Kafka to the Kubernetes cluster with helm and waits for the pods to start up
• creates the echo-messages topic in Kafka.
• starts port forwarding

./setup.sh

output:

+ ZILLA_CHART=oci://ghcr.io/aklivity/charts/zilla
+ helm upgrade --install zilla-grpc-kafka-echo oci://ghcr.io/aklivity/charts/zilla --namespace zilla-grpc-kafka-echo --wait [...]
NAME: zilla-grpc-kafka-echo
LAST DEPLOYED: [...]
NAMESPACE: zilla-grpc-kafka-echo
STATUS: deployed
NOTES:
Zilla has been installed.
[...]
+ helm upgrade --install zilla-grpc-kafka-echo-kafka chart --namespace zilla-grpc-kafka-echo --create-namespace --wait
NAME: zilla-grpc-kafka-echo-kafka
LAST DEPLOYED: [...]
NAMESPACE: zilla-grpc-kafka-echo
STATUS: deployed
REVISION: 1
TEST SUITE: None
++ kubectl get pods --namespace zilla-grpc-kafka-echo --selector app.kubernetes.io/instance=kafka -o name
+ KAFKA_POD=pod/kafka-74675fbb8-kpkm8
+ kubectl exec --namespace zilla-grpc-kafka-echo pod/kafka-74675fbb8-kpkm8 -- /opt/bitnami/kafka/bin/kafka-topics.sh --bootstrap-server localhost:9092 --create --topic echo-messages --if-not-exists
Created topic echo-messages.
+ kubectl port-forward --namespace zilla-grpc-kafka-echo service/zilla 7153
+ nc -z localhost 7153
+ kubectl port-forward --namespace zilla-grpc-kafka-echo service/kafka 9092 29092
+ sleep 1
+ nc -z localhost 7153
Connection to localhost port 7153 [tcp/websm] succeeded!
+ nc -z localhost 9092
Connection to localhost port 9092 [tcp/XmlIpcRegSvc] succeeded!

Verify behavior

Unary Stream

Echo {"message":"Hello World"} message via unary rpc using grpcurl client.

grpcurl -insecure -proto proto/echo.proto  -d '{"message":"Hello World"}' localhost:7153 grpc.examples.echo.Echo.UnaryEcho

output:

{
  "message": "Hello World"
}

Verify the message payload, followed by a tombstone to mark the end of the request.

kcat -C -b localhost:9092 -t echo-messages -J -u | jq .

output:

{
  "topic": "echo-messages",
  "partition": 0,
  "offset": 0,
  "tstype": "create",
  "ts": 1683827977709,
  "broker": 1,
  "headers": [
    "zilla:service",
    "grpc.examples.echo.Echo",
    "zilla:method",
    "UnaryEcho",
    "zilla:reply-to",
    "echo-messages",
    "zilla:correlation-id",
    "13360c3e-6c68-4c1f-bb7b-3cbd832a6007-74b89a7e944bd502db9d81165bda4983"
  ],
  "key": "13360c3e-6c68-4c1f-bb7b-3cbd832a6007-74b89a7e944bd502db9d81165bda4983",
  "payload": "\n\u000bHello World"
}
{
  "topic": "echo-messages",
  "partition": 0,
  "offset": 1,
  "tstype": "create",
  "ts": 1683827977742,
  "broker": 1,
  "headers": [
    "zilla:service",
    "grpc.examples.echo.Echo",
    "zilla:method",
    "UnaryEcho",
    "zilla:reply-to",
    "echo-messages",
    "zilla:correlation-id",
    "13360c3e-6c68-4c1f-bb7b-3cbd832a6007-74b89a7e944bd502db9d81165bda4983"
  ],
  "key": "13360c3e-6c68-4c1f-bb7b-3cbd832a6007-74b89a7e944bd502db9d81165bda4983",
  "payload": null
}
% Reached end of topic echo-messages [0] at offset 2

Bidirectional Stream

Echo messages via bidirectional streaming rpc.

grpcurl -insecure -proto proto/echo.proto -d @ localhost:7153 grpc.examples.echo.Echo.BidirectionalStreamingEcho

Paste below message.

{
  "message": "Hello World"
}

Verify the message payloads, followed by a tombstone to mark the end of each request.

kcat -C -b localhost:9092 -t echo-messages -J -u | jq .

output:

...
{
  "topic": "echo-messages",
  "partition": 0,
  "offset": 2,
  "tstype": "create",
  "ts": 1683828250706,
  "broker": 1,
  "headers": [
    "zilla:service",
    "grpc.examples.echo.Echo",
    "zilla:method",
    "BidirectionalStreamingEcho",
    "zilla:reply-to",
    "echo-messages",
    "zilla:correlation-id",
    "3eb292f7-c503-42d2-a579-82da7bc853f8-45b1b1a7b121d744d1d1a68b30ebc5ef"
  ],
  "key": "3eb292f7-c503-42d2-a579-82da7bc853f8-45b1b1a7b121d744d1d1a68b30ebc5ef",
  "payload": "\n\u000bHello World"
}
{
  "topic": "echo-messages",
  "partition": 0,
  "offset": 3,
  "tstype": "create",
  "ts": 1683828252352,
  "broker": 1,
  "headers": [
    "zilla:service",
    "grpc.examples.echo.Echo",
    "zilla:method",
    "BidirectionalStreamingEcho",
    "zilla:reply-to",
    "echo-messages",
    "zilla:correlation-id",
    "3eb292f7-c503-42d2-a579-82da7bc853f8-45b1b1a7b121d744d1d1a68b30ebc5ef"
  ],
  "key": "3eb292f7-c503-42d2-a579-82da7bc853f8-45b1b1a7b121d744d1d1a68b30ebc5ef",
  "payload": null
}
% Reached end of topic echo-messages [0] at offset 4

Bench

ghz --config bench.json \
    --proto proto/echo.proto \
    --call grpc.examples.echo.Echo/BidirectionalStreamingEcho \
    localhost:7153

Teardown

The teardown.sh script stops port forwarding, uninstalls Zilla and deletes the namespace.

./teardown.sh

output:

+ pgrep kubectl
99998
99999
+ killall kubectl
+ helm uninstall zilla-grpc-kafka-echo zilla-grpc-kafka-echo-kafka --namespace zilla-grpc-kafka-echo
release "zilla-grpc-kafka-echo" uninstalled
release "zilla-grpc-kafka-echo-kafka" uninstalled
+ kubectl delete namespace zilla-grpc-echo
namespace "zilla-grpc-kafka-echo" deleted