Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Failure to add new data node to cluster #15

Closed
khushu12 opened this issue Dec 28, 2022 · 3 comments
Closed

Failure to add new data node to cluster #15

khushu12 opened this issue Dec 28, 2022 · 3 comments

Comments

@khushu12
Copy link

khushu12 commented Dec 28, 2022

System info: [Include InfluxDB version, operating system name, and other relevant details]
influx version: 1.8.10-c1.1.2, EC2-ami ubuntu/images/hvm-ssd/ubuntu-jammy-22.04-amd64-server-20221201

Steps to reproduce:

  1. [First Step] Followed documentation of creating influxdb cluster using prebuilts. Setup was one meta node and trying to attach 1 data node. Started meta node as single server.
sudo /home/ubuntu/influxdb-cluster-1.8.10-c1.1.2-1/usr/bin/influxd-meta -config /home/ubuntu/influxdb-cluster-1.8.10-c1.1.2-1/etc/influxdb/influxdb.conf -single-server &

Server was up.
Started data node on different ubuntu box with changes to hostname in the influxdb config file as specified in docs.

sudo /home/ubuntu/influxdb-cluster-1.8.10-c1.1.2-1/usr/bin/influxd -config /home/ubuntu/influxdb-cluster-1.8.10-c1.1.2-1/etc/influxdb/influxdb.conf
ubuntu@ip-172-16-1-144:~/influxdb-cluster-1.8.10-c1.1.2-1/usr/bin$ /home/ubuntu/influxdb-cluster-1.8.10-c1.1.2-1/usr/bin/influxd-ctl show
Data Nodes
==========
ID	TCP Address	Version

Meta Nodes
==========
ID	TCP Address	Version
1	localhost:8091	1.8.10-c1.1.2
  1. Tried to attach data node with below command by running below command from meta-01
/home/ubuntu/influxdb-cluster-1.8.10-c1.1.2-1/usr/bin/influxd-ctl add-data influxdb-data-03:8088
add-data: operation exited with error: read message size: EOF

Below Error on datanode influx server

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0xf5a79d]

goroutine 45 [running]:
github.com/influxdata/influxdb/coordinator.(*JoinClusterResponse).MarshalBinary(0xc000290678, 0x1, 0x1, 0x0, 0x0, 0xc000290678)
	/root/influxdb/**coordinator/rpc.go:1330 +0x5d**
github.com/influxdata/influxdb/coordinator.EncodeLV(0x2562e00, 0xc000010fc0, 0x255f560, 0xc000290678, 0x0, 0x2562e00)
	/root/influxdb/coordinator/service.go:1594 +0x35
github.com/influxdata/influxdb/coordinator.EncodeTLV(0x2562e00, 0xc000010fc0, 0xc000010f28, 0x255f560, 0xc000290678, 0x1, 0x8)
	/root/influxdb/coordinator/service.go:1586 +0x85
github.com/influxdata/influxdb/coordinator.(*Service).processJoinClusterRequest(0xc000106d80, 0x259fb80, 0xc000010fc0)
	/root/influxdb/coordinator/service.go:1366 +0x2ab
github.com/influxdata/influxdb/coordinator.(*Service).handleConn(0xc000106d80, 0x259fb80, 0xc000010fc0)
	/root/influxdb/coordinator/service.go:422 +0x1466
github.com/influxdata/influxdb/coordinator.(*Service).serve.func1(0xc000106d80, 0x259fb80, 0xc000010fc0)
	/root/influxdb/coordinator/service.go:284 +0x6f
created by github.com/influxdata/influxdb/coordinator.(*Service).serve
	/root/influxdb/coordinator/service.go:282 +0x13f

Checking source code looks like node_id is not being passed or is null. Please help to find a fix.

@chengshiwen
Copy link
Owner

chengshiwen commented Dec 28, 2022

@khushu12

  1. influxd-meta should use the configuration file influxdb-meta.conf, while influxd should use influxdb.conf
  2. Could you provide the modified changes of influxdb-meta.conf and influxdb.conf? Especially the modification of hostname
  3. In fact influxdb-data-03:8088 does not exist, the code will optimize this problem caused by add-data
  4. Please refer to the installation documentation in detail, step by step: English|Installation or 中文|安装

@chengshiwen
Copy link
Owner

@khushu12
Before influxd-meta starts, you should specify the hostname, otherwise the data node cannot access the hostname of the meta node, because the hostname of the meta node is localhost by default

@khushu12
Copy link
Author

Thanks got it figured out. Was using incorrect meta config.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants