Re-use client or create new one each time?

[datdo-msft]

Hi, my question is about using the Triton client within a FastAPI server to send requests downstream to Triton. Is it recommended to create a single instance of the Triton client and re-use it for each request? Or should we create a new instance of the Triton client for each new request?

Asking because for streaming requests, the grpc InferenceServerClient only supports one stream at a time. Please let me know and thanks in advance!