[native]PushConsumer clustering does not auto create %RETRY% topic

[wolftankk]

It has 2 problems:

1. topic has created, but client say not exists.
2. it's not auto creating a group.

INFO[0000] the consumerGroup=go-auto-create-group start beginning. messageModel=Clustering, unitMode=false 
WARN[0000] do balance of group: go-auto-create-group, but topic: RECHARGE_MOBILE does not exist. 
WARN[0000] do balance of group: go-auto-create-group, but topic: %RETRY%go-auto-create-group does not exist.

WARN[0000] query topic route from server error: topic not exist 
the topic=%RETRY%go-auto-create-group route info not found, it may not exist
exit status 255

[wolftankk]

Ahhhhhhha....

I found a magic method.

1. consumer not using consumer.Clustering, and launch.

2. change code, reusing consumer.Clustering

3. launch!

It's woking OK!!

[ShannonDing]

There is probably a bug, could you please check more detail and fix it if it's a bug?

[wolftankk]

hmm, Maybe this code triggers the problem.

github.com/apache/rocketmq-client-go/consumer/push_consumer.go:151

	pc.client.UpdateTopicRouteInfo()
	for k := range pc.subscribedTopic {
		_, exist := pc.topicSubscribeInfoTable.Load(k)
		if !exist {
			pc.client.Shutdown()
			return fmt.Errorf("the topic=%s route info not found, it may not exist", k)
		}
	}

I retry more times, find a solution.

1. [FIRST TIME] use a new group (with clustering). use c.Start(). catch a error.

the topic=%RETRY%go-auto-create-group7 route info not found, it may not exist

and MUST wait 10 seconds.

2. [SECOND TIME] run c.Start(). catch a error

INFO[0000] the topic [%RETRY%go-auto-create-group8] route info changed, old <nil> ,new {"OrderTopicConf":"","queueDatas":[{"brokerName":"iz8vbahalty0z0999g241tz","readQueueNums":1,"writeQueueNums":1,"perm":6,"topicSynFlag":0}],"brokerDatas":[{"cluster":"DefaultCluster","brokerName":"iz8vbahalty0z0999g241tz","brokerAddrs":{"0":"10.100.1.178:10911"}}]} 
INFO[0000] receive broker's notification, the consumer group: go-auto-create-group8 

ERRO[0000] fetch offset of MessageQueue [topic=RECHARGE_MOBILE, brokerName=iz8vbahalty0z0999g241tz, queueId=0] error: broker response code: 22, remarks: Not found, V3_0_6_SNAPSHOT maybe this group consumer boot first 

3. [THIRD TIME] run c.Start(). it's working.

[dreaminglwj]

the same promblem for me when using on ali cloud,
for me:
1.everything ok on public network
2.never succeed on ali interal network --- at last i tried this "and MUST wait 10 seconds." and it works

[wolftankk]

It does not seem to work.

	pc.client.UpdateTopicRouteInfo()
	for k := range pc.subscribedTopic {
		_, exist := pc.topicSubscribeInfoTable.Load(k)
		if !exist {
			pc.client.Shutdown()
			return fmt.Errorf("the topic=%s route info not found, it may not exist", k)
		}
	}

I'm still looking for it.

---

DEBUG INFO:

1 . FIRST

INFO[0000] queryTopicRouteInfoFromServer topic:%RETRY%go-auto-create-group108 
INFO[0000] queryTopicRouteInfoFromServer, topic: %RETRY%go-auto-create-group108, response: Code: 17, opaque: 2, Remark: No topic route info in name server for the topic: %RETRY%go-auto-create-group108
See http://rocketmq.apache.org/docs/faq/ for further details., ExtFields: map[] 
WARN[0000] query topic route from server error: topic not exist 

It calls remote cmd: queryTopicInfo, get code: 17

2. SECOND (waiting 10 sec)

INFO[0000] queryTopicRouteInfoFromServer topic:%RETRY%go-auto-create-group108 
INFO[0000] queryTopicRouteInfoFromServer, topic: %RETRY%go-auto-create-group108, response: Code: 0, opaque: 5, Remark: , ExtFields: map[] 
INFO[0000] receive broker's notification, the consumer group: go-auto-create-group108 

It calls remote cmd: queryTopicInfo, get code: 0

---

And, I try to add a Rebalane method before pc.client.UpdateTopicRouteInfo()

//func (pc *pushConsumer) Start() error {
     //....

     pc.Rebalance()
     time.Sleep(1 * time.Second)

       pc.client.UpdateTopicRouteInfo()
	for k := range pc.subscribedTopic {
		_, exist := pc.topicSubscribeInfoTable.Load(k)
		if !exist {
			pc.client.Shutdown()
			return fmt.Errorf("the topic=%s route info not found, it may not exist", k)
		}
	}

//}

It seems to work

INFO[0000] queryTopicRouteInfoFromServer, topic: %RETRY%go-auto-create-group110, response: Code: 17, opaque: 2, Remark: No topic route info in name server for the topic: %RETRY%go-auto-create-group110
See http://rocketmq.apache.org/docs/faq/ for further details., ExtFields: map[] 
WARN[0000] query topic route from server error: topic not exist 
ERRO[0000] >>>>>>>>>>>>>>>>>>>>>> consumer group is not auto creating, err: map[%RETRY%go-auto-create-group110:true order:true] 
INFO[0000] queryTopicRouteInfoFromServer topic:%RETRY%go-auto-create-group110 


INFO[0000] queryTopicRouteInfoFromServer, topic: %RETRY%go-auto-create-group110, response: Code: 17, opaque: 3, Remark: No topic route info in name server for the topic: %RETRY%go-auto-create-group110
See http://rocketmq.apache.org/docs/faq/ for further details., ExtFields: map[] 
WARN[0000] query topic route from server error: topic not exist 

WARN[0000] do balance of group: go-auto-create-group110, but topic: %RETRY%go-auto-create-group110 does not exist. 
WARN[0000] do balance of group: go-auto-create-group110, but topic: %RETRY%go-auto-create-group110 does not exist. 
INFO[0000] receive broker's notification, the consumer group: go-auto-create-group110 

I guess that it calls RebalanceImmediately for creating a %RETRY% topic.

[zenghur]

I came across the same problem. But I fixed it by using WithNamespace:-)

[wolftankk]

I came across the same problem. But I fixed it by using WithNamespace:-)

You need a new group name. The old group name maybe auto created.

[ShannonDing]

link #258

[wenfengwang]

It seems to be resolved, is there any update?

[qshuai]

I came across the same problem. But I fixed it by using WithNamespace:-)

What is the default namespace?

[dreaminglwj]

您好，您的邮件已收到，我会尽快查看，^_^