Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[feature request] Set K8s service host & port to Yurthub Proxy Host & Port in Yurthub Component #2158

Open
techworldhello opened this issue Oct 7, 2024 · 5 comments · May be fixed by #2165
Labels
kind/feature kind/feature

Comments

@techworldhello
Copy link

techworldhello commented Oct 7, 2024

What would you like to be added:

A filter in Yurthub that sets the KUBERNETES_SERVICE_HOST and KUBERNETES_SERVICE_PORT environment variables to the Yurthub proxy host and IP. This enhancement would ensure node autonomy for edge nodes when they are operating offline.

Why is this needed:

Some managed services may use an admission webhook to modify the KUBERNETES_SERVICE_HOST and KUBERNETES_SERVICE_PORT environment variables, setting them to the public FQDN and port of the kube-apiserver. For these managed services to operate with Yurthub, the pods scheduled on edge nodes must be directed to Yurthub so that requests to the kube-apiserver can be proxied through it.

We are requesting the implementation of a filter that would ensure all edge pods access the kube-apiserver via the Yurthub proxy. This filter would add logic without disrupting other users. If a user prefers to disable this filter in the future, they could do so by adding it to the --disabled-resource-filters startup parameter.

others
/kind feature

@techworldhello techworldhello added the kind/feature kind/feature label Oct 7, 2024
@techworldhello techworldhello changed the title In Yurthub component, set KUBERNETES_SERVICE_HOST environment variable to Yurthub proxy address Set KUBERNETES_SERVICE_HOST to Yurthub Proxy Address in Yurthub Component Oct 7, 2024
@zyjhtangtang
Copy link
Contributor

masterServiceFilter has already implemented this capability. By default, Yurthub only caches request data for system components. If the business requires utilizing Yurthub's cache to operate during network outages, the corresponding User-Agent (UA) needs to be configured in the ConfigMap yurt-hub-cfg. reference: https:/openyurtio/openyurt/blob/master/docs/proposals/20220627-yurthub-cache-refactoring.md#41-enable-yurthub-to-distinguish-resources-with-same-name-but-different-versions-and-groups

@rambohe-ch
Copy link
Member

masterServiceFilter has already implemented this capability. By default, Yurthub only caches request data for system components. If the business requires utilizing Yurthub's cache to operate during network outages, the corresponding User-Agent (UA) needs to be configured in the ConfigMap yurt-hub-cfg. reference: https:/openyurtio/openyurt/blob/master/docs/proposals/20220627-yurthub-cache-refactoring.md#41-enable-yurthub-to-distinguish-resources-with-same-name-but-different-versions-and-groups

@zyjhtangtang I think that masterservice Filter can not solve the problem that mentioned in the issue. because masterservice filter only modify default/kubernetes service, this means that pods without KUBERNETES_SERVICE_HOST env can work with Yurthub component seamlessly. but pods with KUBERNETES_SERVICE_HOST can not work together with Yurthub component. maybe it is a good idea to overwrite KUBERNETES_SERVICE_HOST env of pod by Yurthub proxy address on edge nodes.

@rambohe-ch
Copy link
Member

What would you like to be added:

A filter in Yurthub that sets the KUBERNETES_SERVICE_HOST environment variable to the Yurthub proxy IP. This enhancement would ensure node autonomy for edge nodes when they are operating offline.

Why is this needed:

Some managed services may use an admission webhook to change the KUBERNETES_SERVICE_HOST environment variable to the public FQDN of the kube-apiserver. For these managed services to operate with Yurthub, the pods scheduled on edge nodes must be directed to Yurthub so that requests to the kube-apiserver can be proxied through it.

We are requesting the implementation of a filter that would ensure all edge pods access the kube-apiserver via the Yurthub proxy. This filter would add logic without disrupting other users. If a user prefers to disable this filter in the future, they could do so by adding it to the --disabled-resource-filters startup parameter.

others /kind feature

@techworldhello Thank you for raising the issue. Only pods without KUBERNETES_SERVICE_HOST env can access kube-apiserver through yurthub currently. so it seems that a new filter is necessary for dealing with pods with KUBERNETES_SERVICE_HOST env on edge nodes.

@zyjhtangtang
Copy link
Contributor

@techworldhello @rambohe-ch Yes, if the pod has already been set with KUBERNETES_SERVICE_HOST, the masterservice filter will not take effect. In this case, It is necessary to add a new filter to directly modify it.

@techworldhello techworldhello changed the title Set KUBERNETES_SERVICE_HOST to Yurthub Proxy Address in Yurthub Component [feature request] Set KUBERNETES_SERVICE_HOST to Yurthub Proxy Address in Yurthub Component Oct 10, 2024
@techworldhello techworldhello changed the title [feature request] Set KUBERNETES_SERVICE_HOST to Yurthub Proxy Address in Yurthub Component [feature request] Set K8s service host & port to Yurthub Proxy IP & Port in Yurthub Component Oct 11, 2024
@techworldhello techworldhello linked a pull request Oct 11, 2024 that will close this issue
@techworldhello
Copy link
Author

@zyjhtangtang @rambohe-ch I've updated this issue to link to a new PR #2165, as it made sense to update KUBERNETES_SERVICE_PORT as well as KUBERNETES_SERVICE_HOST, since they are both required to connect to Yurthub. I've also updated the issue title and description to reflect this. Would appreciate your reviews again, thanks!

@techworldhello techworldhello changed the title [feature request] Set K8s service host & port to Yurthub Proxy IP & Port in Yurthub Component [feature request] Set K8s service host & port to Yurthub Proxy Host & Port in Yurthub Component Oct 11, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/feature kind/feature
Projects
None yet
3 participants