Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Port forwards fail to bind on macOS when localhost resolves to fe80::1%lo0 #5981

Closed
louismrose opened this issue Nov 29, 2022 · 2 comments · Fixed by #5988
Closed

Port forwards fail to bind on macOS when localhost resolves to fe80::1%lo0 #5981

louismrose opened this issue Nov 29, 2022 · 2 comments · Fixed by #5988
Labels
bug Something isn't working

Comments

@louismrose
Copy link
Contributor

Expected Behavior

Using k8s_resource with a port forward should work on macOS, when localhost resolves to fe80::1%lo0.

Current Behavior

Port forwarding fails in Tilt with the following error:

Reconnecting... Error port-forwarding test-deploy (9090 -> 9090): error forwarding port: fe80::1%lo0 is not a valid IP

I believe this is an issue with Tilt because I can successfully configure a port forward via Kubernetes for the same service:

$ kubectl port-forward test-deploy-56469867cc-v852c 9090:9090
Forwarding from 127.0.0.1:9090 -> 9090
Forwarding from [::1]:9090 -> 9090
Handling connection for 9090

Interestingly if I try to port forward with Kubernetes via an address hardcoded to fe80::1%lo0 then it fails with exactly the same error message seen in Tilt:

$ kubectl port-forward test-deploy-56469867cc-v852c --address fe80::1%lo0 9090:9090
error: fe80::1%lo0 is not a valid IP

I suspect this is due to net.LookupHost including fe80::1%lo0 in the list of addresses via the default Go resolver on macOS:

$ cat localhost.go
package main

import "fmt"
import "net"

func main() {
        addr, _ := net.LookupHost("localhost")
	fmt.Println("Localhost addresses are:", addr)
}

$ go run localhost.go
Localhost addresses are: [::1 127.0.0.1 fe80::1%lo0]

fe80::1%lo0 seems to appear in /etc/hosts by default on some macOS installations. I believe the %lo0 is intended to restrict traffic to a particular network interface via a "zone index". It looks like Kubernetes doesn't support IP addresses with a zone index, so Tilt probably shouldn't pass addresses of this form to Kubernetes when setting up port forwards.

I had a quick look at internal/k8s/portforward.go, and it looks like getListenableAddresses might be the place to update? Let me know if that's the case, and I can attempt a PR to exclude IP addresses with a zone index.

Steps to Reproduce

  1. Ensure you're running on a machine (e.g., macOS) where localhost resolves to fe80::1%lo0 (e.g., by using the sample Go code above).
  2. Run tilt up for a Tiltfile including a k8s_resource with port_forwards
  3. Observe an error message similar to Reconnecting... Error port-forwarding test-deploy (9090 -> 9090): error forwarding port: fe80::1%lo0 is not a valid IP in the logs for that resource.

Context

tilt doctor Output

$ tilt doctor
Tilt: v0.30.12, built 2022-11-16
System: darwin-arm64
---
Docker
- Host: unix:///var/run/docker.sock
- Server Version: 20.10.21
- API Version: 1.41
- Builder: 2
- Compose Version: v2.12.2
---
Kubernetes
- Env: kind
- Context: kind-crs-tilt
- Cluster Name: kind-crs-tilt
- Namespace: default
- Container Runtime: containerd
- Version: v1.23.13
- Cluster Local Registry: &RegistryHosting{Host:localhost:60929,HostFromClusterNetwork:ctlptl-registry:5000,HostFromContainerRuntime:,Help:https:/tilt-dev/ctlptl,SingleName:,}
---
Thanks for seeing the Tilt Doctor!
Please send the info above when filing bug reports. 💗

The info below helps us understand how you're using Tilt so we can improve,
but is not required to ask for help.
---
Analytics Settings
--> (These results reflect your personal opt in/out status and may be overridden by an `analytics_settings` call in your Tiltfile)
- User Mode: opt-out
- Machine: d860287cbe091b346cd763ee686743a0
- Repo: 

About Your Use Case

As I haven't yet been able to figure out how to remove ``fe80::1%lo0from the addresses to whichlocalhost` resolves via `net.LookupHost` (removing the entry from `/etc/hosts` doesn't work), I cannot use Tilt port forwards at all.

I can workaround this by using kubectl port-forward manually instead.

@louismrose louismrose added the bug Something isn't working label Nov 29, 2022
@louismrose
Copy link
Contributor Author

A colleague noted that a better workaround is tilt up --host 127.0.0.1

@nicksieger
Copy link
Member

nicksieger commented Nov 29, 2022

I was about to ask if tilt up --host '[::1]' would also work, it should. Your analysis is spot on, if you don't mind submitting a PR to exclude addresses with zone indexes that would be great!

louismrose pushed a commit to louismrose/tilt that referenced this issue Dec 7, 2022
louismrose pushed a commit to louismrose/tilt that referenced this issue Dec 7, 2022
louismrose pushed a commit to louismrose/tilt that referenced this issue Dec 7, 2022
louismrose pushed a commit to louismrose/tilt that referenced this issue Dec 7, 2022
louismrose pushed a commit to louismrose/tilt that referenced this issue Dec 7, 2022
louismrose pushed a commit to louismrose/tilt that referenced this issue Dec 7, 2022
louismrose pushed a commit to louismrose/tilt that referenced this issue Dec 7, 2022
louismrose pushed a commit to louismrose/tilt that referenced this issue Dec 7, 2022
louismrose pushed a commit to louismrose/tilt that referenced this issue Dec 7, 2022
nicks added a commit to nicks/tilt that referenced this issue Dec 10, 2022
…ing behavior

The tiltfile evaluator always defaults the empty string to "localhost".
We want to make sure that this goes down the default kubernetes binding
codepath instead of our custom codepath to resolve hostnames.

Fixes tilt-dev#5981

Signed-off-by: Nick Santos <[email protected]>
nicks added a commit to nicks/tilt that referenced this issue Dec 10, 2022
…ing behavior

The tiltfile evaluator always defaults the empty string to "localhost".
We want to make sure that this goes down the default kubernetes binding
codepath instead of our custom codepath to resolve hostnames.

Fixes tilt-dev#5981

Signed-off-by: Nick Santos <[email protected]>
nicks added a commit to nicks/tilt that referenced this issue Dec 10, 2022
…ing behavior

The tiltfile evaluator always defaults the empty string to "localhost".
We want to make sure that this goes down the default kubernetes binding
codepath instead of our custom codepath to resolve hostnames.

Fixes tilt-dev#5981

Signed-off-by: Nick Santos <[email protected]>
nicks added a commit to nicks/tilt that referenced this issue Dec 10, 2022
…ing behavior

The tiltfile evaluator usually defaults the empty string to "localhost"
(because it's the default web host)

We want to make sure that this goes down the default kubernetes binding
codepath instead of our custom codepath to resolve hostnames.

Fixes tilt-dev#5981

Signed-off-by: Nick Santos <[email protected]>
@nicks nicks closed this as completed in 3fea136 Dec 10, 2022
nicks added a commit to nicks/tilt that referenced this issue Dec 10, 2022
…ing behavior

The tiltfile evaluator usually defaults the empty string to "localhost"
(because it's the default web host)

We want to make sure that this goes down the default kubernetes binding
codepath instead of our custom codepath to resolve hostnames.

Fixes tilt-dev#5981

Signed-off-by: Nick Santos <[email protected]>
nicks added a commit that referenced this issue Dec 12, 2022
…ing behavior (#5988)

The tiltfile evaluator usually defaults the empty string to "localhost"
(because it's the default web host)

We want to make sure that this goes down the default kubernetes binding
codepath instead of our custom codepath to resolve hostnames.

Fixes #5981

Signed-off-by: Nick Santos <[email protected]>

Signed-off-by: Nick Santos <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants