nacos集群偶发，Request stream error, switch server,error={} UNAVAILABLE: Network closed for unknown reason

[colstars]

测试、生产的应用偶尔发生如下错误：
错误发生时，服务不可用。偶尔发生一次，测试环境相对频繁一点
我怀疑过：
1.应用重启导致，但是产线报错时应用没有重启，所以排除
2.JVM问题，nacos服务一直ygc，S0,S1基本不变，都在Eden，然后触发YGC，比较平凡，基本5分钟一次。但是如果是这个原因，是不是5分钟就报错一次，所以我也排除
3.网络问题，这个就不好确认了

想请教下，还可以从哪些方面定位问题原因？以及还有可能是什么情况下出现错误Request stream error

如下为近期日志
2024-10-15 16:49:51.912 [app] - [nacos-grpc-client-executor-nacos.sinopharmholding-fl.com-339459]-[,] ERROR [c.a.nacos.common.remote.client.grpc.GrpcClient.printIfErrorEnabled:102] : [1728227984931_10.244.81.0_56428]Request stream error, switch server,error={}
com.alibaba.nacos.shaded.io.grpc.StatusRuntimeException: UNAVAILABLE: Network closed for unknown reason
at com.alibaba.nacos.shaded.io.grpc.Status.asRuntimeException(Status.java:539)
at com.alibaba.nacos.shaded.io.grpc.stub.ClientCalls$StreamObserverToCallListenerAdapter.onClose(ClientCalls.java:487)
at com.alibaba.nacos.shaded.io.grpc.internal.ClientCallImpl.closeObserver(ClientCallImpl.java:563)
at com.alibaba.nacos.shaded.io.grpc.internal.ClientCallImpl.access$300(ClientCallImpl.java:70)
at com.alibaba.nacos.shaded.io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInternal(ClientCallImpl.java:744)
at com.alibaba.nacos.shaded.io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInContext(ClientCallImpl.java:723)
at com.alibaba.nacos.shaded.io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37)
at com.alibaba.nacos.shaded.io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:133)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at java.base/java.lang.Thread.run(Thread.java:840)
2024-10-15 16:49:51.917 [app] - [com.alibaba.nacos.client.remote.worker]-[,] INFO [com.alibaba.nacos.common.remote.client.printIfInfoEnabled:63] : [b9c164f9-fe4c-4d23-bd8c-58174ce0ea65_config-0] Try to reconnect to a new server, server is not appointed, will choose a random server.
2024-10-15 16:49:51.918 [app] - [nacos-grpc-client-executor-nacos.sinopharmholding-fl.com-339305]-[,] ERROR [c.a.nacos.common.remote.client.grpc.GrpcClient.printIfErrorEnabled:102] : [1728227984932_10.244.81.0_56432]Request stream error, switch server,error={}
com.alibaba.nacos.shaded.io.grpc.StatusRuntimeException: UNAVAILABLE: Network closed for unknown reason
at com.alibaba.nacos.shaded.io.grpc.Status.asRuntimeException(Status.java:539)
at com.alibaba.nacos.shaded.io.grpc.stub.ClientCalls$StreamObserverToCallListenerAdapter.onClose(ClientCalls.java:487)
at com.alibaba.nacos.shaded.io.grpc.internal.ClientCallImpl.closeObserver(ClientCallImpl.java:563)
at com.alibaba.nacos.shaded.io.grpc.internal.ClientCallImpl.access$300(ClientCallImpl.java:70)
at com.alibaba.nacos.shaded.io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInternal(ClientCallImpl.java:744)
at com.alibaba.nacos.shaded.io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInContext(ClientCallImpl.java:723)
at com.alibaba.nacos.shaded.io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37)
at com.alibaba.nacos.shaded.io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:133)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at java.base/java.lang.Thread.run(Thread.java:840)
2024-10-15 16:49:51.918 [app] - [com.alibaba.nacos.client.remote.worker]-[,] INFO [com.alibaba.nacos.common.remote.client.printIfInfoEnabled:63] : [431939e7-039d-4d73-a572-56f1cfb5dfca] Try to reconnect to a new server, server is not appointed, will choose a random server.
2024-10-15 16:49:51.918 [app] - [com.alibaba.nacos.client.remote.worker]-[,] INFO [c.a.nacos.common.remote.client.grpc.GrpcClient.createNewManagedChannel:182] : grpc client connection server:nacos.sinopharmholding-fl.com ip,serverPort:9848,grpcTslConfig:{"sslProvider":"OPENSSL","enableTls":false,"mutualAuthEnable":false,"trustAll":false}
2024-10-15 16:49:51.919 [app] - [com.alibaba.nacos.client.remote.worker]-[,] INFO [c.a.nacos.common.remote.client.grpc.GrpcClient.createNewManagedChannel:182] : grpc client connection server:nacos.sinopharmholding-fl.com ip,serverPort:9848,grpcTslConfig:{"sslProvider":"OPENSSL","enableTls":false,"mutualAuthEnable":false,"trustAll":false}
2024-10-15 16:49:52.034 [app] - [com.alibaba.nacos.client.remote.worker]-[,] INFO [com.alibaba.nacos.common.remote.client.printIfInfoEnabled:63] : [b9c164f9-fe4c-4d23-bd8c-58174ce0ea65_config-0] Success to connect a server [nacos.sinopharmholding-fl.com:8848], connectionId = 1728982191931_10.244.81.0_55160
2024-10-15 16:49:52.034 [app] - [com.alibaba.nacos.client.remote.worker]-[,] INFO [com.alibaba.nacos.common.remote.client.printIfInfoEnabled:63] : [b9c164f9-fe4c-4d23-bd8c-58174ce0ea65_config-0] Abandon prev connection, server is nacos.sinopharmholding-fl.com:8848, connectionId is 1728227984931_10.244.81.0_56428
2024-10-15 16:49:52.034 [app] - [com.alibaba.nacos.client.remote.worker]-[,] INFO [com.alibaba.nacos.common.remote.client.printIfInfoEnabled:63] : [431939e7-039d-4d73-a572-56f1cfb5dfca] Success to connect a server [nacos.sinopharmholding-fl.com:8848], connectionId = 1728982191931_10.244.81.0_55144
2024-10-15 16:49:52.034 [app] - [com.alibaba.nacos.client.remote.worker]-[,] INFO [com.alibaba.nacos.common.remote.client.closeConnection:584] : Close current connection 1728227984931_10.244.81.0_56428
2024-10-15 16:49:52.034 [app] - [com.alibaba.nacos.client.remote.worker]-[,] INFO [com.alibaba.nacos.common.remote.client.printIfInfoEnabled:63] : [431939e7-039d-4d73-a572-56f1cfb5dfca] Abandon prev connection, server is nacos.sinopharmholding-fl.com:8848, connectionId is 1728227984932_10.244.81.0_56432
2024-10-15 16:49:52.034 [app] - [com.alibaba.nacos.client.remote.worker]-[,] INFO [com.alibaba.nacos.common.remote.client.closeConnection:584] : Close current connection 1728227984932_10.244.81.0_56432
2024-10-15 16:49:52.035 [app] - [com.alibaba.nacos.client.remote.worker]-[,] INFO [com.alibaba.nacos.common.remote.client.printIfInfoEnabled:63] : [b9c164f9-fe4c-4d23-bd8c-58174ce0ea65_config-0] Notify disconnected event to listeners
2024-10-15 16:49:52.035 [app] - [com.alibaba.nacos.client.remote.worker]-[,] INFO [com.alibaba.nacos.client.config.impl.ClientWorker.onDisConnect:646] : [b9c164f9-fe4c-4d23-bd8c-58174ce0ea65_config-0] DisConnected,clear listen context...
2024-10-15 16:49:52.035 [app] - [com.alibaba.nacos.client.remote.worker]-[,] INFO [com.alibaba.nacos.common.remote.client.printIfInfoEnabled:63] : [b9c164f9-fe4c-4d23-bd8c-58174ce0ea65_config-0] Notify connected event to listeners.
2024-10-15 16:49:52.035 [app] - [com.alibaba.nacos.client.remote.worker]-[,] INFO [com.alibaba.nacos.client.config.impl.ClientWorker.onConnected:639] : [b9c164f9-fe4c-4d23-bd8c-58174ce0ea65_config-0] Connected,notify listen context...
2024-10-15 16:49:52.035 [app] - [com.alibaba.nacos.client.remote.worker]-[,] INFO [com.alibaba.nacos.common.remote.client.printIfInfoEnabled:63] : [431939e7-039d-4d73-a572-56f1cfb5dfca] Notify disconnected event to listeners
2024-10-15 16:49:52.036 [app] - [com.alibaba.nacos.client.remote.worker]-[,] WARN [com.alibaba.nacos.client.naming.onDisConnect:84] : Grpc connection disconnect, mark to redo
2024-10-15 16:49:52.036 [app] - [com.alibaba.nacos.client.remote.worker]-[,] WARN [com.alibaba.nacos.client.naming.onDisConnect:91] : mark to redo completed
2024-10-15 16:49:52.036 [app] - [com.alibaba.nacos.client.remote.worker]-[,] INFO [com.alibaba.nacos.common.remote.client.printIfInfoEnabled:63] : [431939e7-039d-4d73-a572-56f1cfb5dfca] Notify connected event to listeners.
2024-10-15 16:49:52.036 [app] - [com.alibaba.nacos.client.remote.worker]-[,] INFO [com.alibaba.nacos.client.naming.onConnected:78] : Grpc connection connect
2024-10-15 16:49:53.144 [app] - [com.alibaba.nacos.client.naming.grpc.redo.0]-[,] INFO [com.alibaba.nacos.client.naming.redoForInstance:73] : Redo instance operation REGISTER for DEFAULT_GROUP@@app
2024-10-15 16:49:53.147 [app] - [com.alibaba.nacos.client.naming.grpc.redo.0]-[,] INFO [com.alibaba.nacos.client.naming.redoForSubscribe:121] : Redo subscriber operation REGISTER for DEFAULT_GROUP@@base-interface#
2024-10-15 16:49:53.281 [app] - [nacos-grpc-client-executor-nacos.sinopharmholding-fl.com-339306]-[,] INFO [com.alibaba.nacos.common.remote.client.printIfInfoEnabled:63] : [431939e7-039d-4d73-a572-56f1cfb5dfca] Receive server push request, request = NotifySubscriberRequest, requestId = 189
2024-10-15 16:49:53.282 [app] - [nacos-grpc-client-executor-nacos.sinopharmholding-fl.com-339306]-[,] INFO [com.alibaba.nacos.client.naming.isChangedServiceInfo:241] : removed ips(2) service: DEFAULT_GROUP@@base-interface -> [{"ip":"10.244.81.61","port":9081,"weight":1.0,"healthy":true,"enabled":true,"ephemeral":true,"clusterName":"DEFAULT","serviceName":"DEFAULT_GROUP@@base-interface","metadata":{"preserved.register.source":"SPRING_CLOUD"},"instanceHeartBeatInterval":5000,"instanceHeartBeatTimeOut":15000,"ipDeleteTimeout":30000},{"ip":"10.244.213.207","port":9081,"weight":1.0,"healthy":true,"enabled":true,"ephemeral":true,"clusterName":"DEFAULT","serviceName":"DEFAULT_GROUP@@base-interface","metadata":{"preserved.register.source":"SPRING_CLOUD"},"instanceHeartBeatInterval":5000,"instanceHeartBeatTimeOut":15000,"ipDeleteTimeout":30000}]
2024-10-15 16:49:53.282 [app] - [nacos-grpc-client-executor-nacos.sinopharmholding-fl.com-339306]-[,] INFO [com.alibaba.nacos.client.naming.processServiceInfo:169] : current ips:(0) service: DEFAULT_GROUP@@base-interface -> []
2024-10-15 16:49:53.285 [app] - [nacos-grpc-client-executor-nacos.sinopharmholding-fl.com-339306]-[,] INFO [com.alibaba.nacos.common.remote.client.printIfInfoEnabled:63] : [431939e7-039d-4d73-a572-56f1cfb5dfca] Ack server push request, request = NotifySubscriberRequest, requestId = 189
2024-10-15 16:49:54.087 [app] - [nacos-grpc-client-executor-nacos.sinopharmholding-fl.com-339303]-[,] INFO [com.alibaba.nacos.common.remote.client.printIfInfoEnabled:63] : [431939e7-039d-4d73-a572-56f1cfb5dfca] Receive server push request, request = NotifySubscriberRequest, requestId = 207
2024-10-15 16:49:54.087 [app] - [nacos-grpc-client-executor-nacos.sinopharmholding-fl.com-339303]-[,] INFO [com.alibaba.nacos.client.naming.isChangedServiceInfo:235] : new ips(1) service: DEFAULT_GROUP@@base-interface -> [{"ip":"10.244.81.61","port":9081,"weight":1.0,"healthy":true,"enabled":true,"ephemeral":true,"clusterName":"DEFAULT","serviceName":"DEFAULT_GROUP@@base-interface","metadata":{"preserved.register.source":"SPRING_CLOUD"},"instanceHeartBeatInterval":5000,"instanceHeartBeatTimeOut":15000,"ipDeleteTimeout":30000}]
2024-10-15 16:49:54.088 [app] - [nacos-grpc-client-executor-nacos.sinopharmholding-fl.com-339303]-[,] INFO [com.alibaba.nacos.client.naming.processServiceInfo:169] : current ips:(1) service: DEFAULT_GROUP@@base-interface -> [{"ip":"10.244.81.61","port":9081,"weight":1.0,"healthy":true,"enabled":true,"ephemeral":true,"clusterName":"DEFAULT","serviceName":"DEFAULT_GROUP@@base-interface","metadata":{"preserved.register.source":"SPRING_CLOUD"},"instanceHeartBeatInterval":5000,"instanceHeartBeatTimeOut":15000,"ipDeleteTimeout":30000}]
2024-10-15 16:49:54.088 [app] - [nacos-grpc-client-executor-nacos.sinopharmholding-fl.com-339303]-[,] INFO [com.alibaba.nacos.common.remote.client.printIfInfoEnabled:63] : [431939e7-039d-4d73-a572-56f1cfb5dfca] Ack server push request, request = NotifySubscriberRequest, requestId = 207
2024-10-15 16:49:55.088 [app] - [nacos-grpc-client-executor-nacos.sinopharmholding-fl.com-339304]-[,] INFO [com.alibaba.nacos.common.remote.client.printIfInfoEnabled:63] : [431939e7-039d-4d73-a572-56f1cfb5dfca] Receive server push request, request = NotifySubscriberRequest, requestId = 230
2024-10-15 16:49:55.088 [app] - [nacos-grpc-client-executor-nacos.sinopharmholding-fl.com-339304]-[,] INFO [com.alibaba.nacos.client.naming.isChangedServiceInfo:235] : new ips(1) service: DEFAULT_GROUP@@base-interface -> [{"ip":"10.244.213.207","port":9081,"weight":1.0,"healthy":true,"enabled":true,"ephemeral":true,"clusterName":"DEFAULT","serviceName":"DEFAULT_GROUP@@base-interface","metadata":{"preserved.register.source":"SPRING_CLOUD"},"instanceHeartBeatInterval":5000,"instanceHeartBeatTimeOut":15000,"ipDeleteTimeout":30000}]
2024-10-15 16:49:55.088 [app] - [nacos-grpc-client-executor-nacos.sinopharmholding-fl.com-339304]-[,] INFO [com.alibaba.nacos.client.naming.processServiceInfo:169] : current ips:(2) service: DEFAULT_GROUP@@base-interface -> [{"ip":"10.244.81.61","port":9081,"weight":1.0,"healthy":true,"enabled":true,"ephemeral":true,"clusterName":"DEFAULT","serviceName":"DEFAULT_GROUP@@base-interface","metadata":{"preserved.register.source":"SPRING_CLOUD"},"instanceHeartBeatInterval":5000,"instanceHeartBeatTimeOut":15000,"ipDeleteTimeout":30000},{"ip":"10.244.213.207","port":9081,"weight":1.0,"healthy":true,"enabled":true,"ephemeral":true,"clusterName":"DEFAULT","serviceName":"DEFAULT_GROUP@@base-interface","metadata":{"preserved.register.source":"SPRING_CLOUD"},"instanceHeartBeatInterval":5000,"instanceHeartBeatTimeOut":15000,"ipDeleteTimeout":30000}]

[colstars]

v2.2.1