Adds kfserving ingressgateway for Istio 1.1.6 #949

krishnadurai · 2020-02-25T09:36:51Z

Modifies KFServing and KNative Serving config-maps to use this gateway

Which issue is resolved by this Pull Request:
Addresses: #924

Description of your changes:
This implementation introduces an alternative Kubeflow ingress-gateway for KFServing. This route was chosen because the existing Kubeflow gateway is protected by Auth in several configurations. The Knative service readiness prober requests fail when they try to probe a KFServing InferenceService since those prober requests are not backed with Authentication credentials.

This implementation is a temporary solution until KNative introduces a solution for probing services behind an authenticated gateway as mentioned in this issue: knative/serving#6829

Related issues:
kserve/kserve#668

/cc @yuzisun @animeshsingh

/hold for testing

Checklist:

Unit tests have been rebuilt:
1. cd manifests/tests
2. make generate-changed-only
3. make test

This change is

Modifies KFServing and KNative Serving config-maps to use this gateway

krishnadurai · 2020-02-25T11:11:42Z

/hold cancel

/assign @yuzisun @animeshsingh

krishnadurai · 2020-02-25T11:14:03Z

Note that PR for Istio 1.3.1 will be raised separately. This has been verfied in kfctl_k8s_istio and kfctl_gcp_iap configurations.

animeshsingh

thanks @krishnadurai
cc @Tomcli

/lgtm

animeshsingh · 2020-02-26T00:54:12Z

istio/kfserving-gateway/base/service.yaml

+ port: 15030
+ targetPort: 15030
+ - name: http2-grafana
+ port: 15031


are these ports for kiali/grafana/prometheus always constant? do we deploy them with kubeflow?

@animeshsingh yes they are associated with the deployment config and are constant. We do deploy these services with Istio in Kubeflow.

Thaks - great to know. What's the total infra cost of these 3 services - believe a container each? Are any customized Kubeflow metrics configured to flow to them

@animeshsingh These require a container each. Prometheus, Graphana etc do come with the default istio install in Kubeflow.

Kubeflow does report Prometheus metrics through various operators or services like tfjob-operator or profile-controller. Likewise, the gateway container istio-proxy also reports metrics.

These ports in question are ports which the istio-proxy container exposes. The container istio-proxy is a part of the deployment servicing kfserving-gateway. These ports host https end-points for each of these services like Prometheus to poll these (gateway) services for metrics.

yuzisun · 2020-02-27T02:22:32Z

istio/kfserving-gateway/base/service.yaml

+ app: kfserving-ingressgateway
+ kfserving: ingressgateway
+spec:
+ type: LoadBalancer


Looks the kubeflow istio ingressgateway is using NodePort https:/kubeflow/manifests/blob/master/istio/istio-install/base/istio-noauth.yaml#L14047
Would this work for on prem setup?

@yuzisun The reason for istio-ingressgateway being NodePort is because of Google's IAP needing it to be NodePort.

Would this work for on prem setup?

If an on-premise setup has some component to provision a LoadBalancer, then LoadBalancer will get its IP. If not, the NodePorts are still accessible though the IP field for LoadBalancer will be in state.

NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE istio-ingressgateway LoadBalancer 10.103.151.45 <pending> 15020:30047/TCP,80:31380/TCP,443:31390/TCP,31400:31400/TCP,15029:30370/TCP,15030:31766/TCP,15031:32168/TCP,15032:31581/TCP,15443:30268/TCP 2d12h

IMO, This should be acceptable. WDYT?

we are using this ingress gateway in GCP IAP setup, so that's ok ?

In istio-ingressgateway for GCP IAP, the ingress gateway service is exposed to an Identity Aware Proxy (which has auth checking) by the NodePort and the intention is to disallow access through any other mechanism other than this proxy.
Since we don't want to have auth checks, LoadBalancer should be alright.

got it, thanks for the explanation! btw we also need to also add the HPA for this ingress gateway?

Good suggestion @yuzisun. I'll include the definition for HPA.

@yuzisun I've included an HPA.

yuzisun · 2020-03-04T14:50:42Z

Thanks @krishnadurai! This is awesome and removes the biggest road block for kubeflow integration!

/lgtm

krishnadurai · 2020-03-04T17:08:19Z

Thanks @yuzisun! You have been super helpful throughout.

@animeshsingh could you please take another look to get this in soon?

krishnadurai · 2020-03-06T12:55:57Z

@animeshsingh can you PTAL at this again?

animeshsingh · 2020-03-06T16:58:21Z

/lgtm
/approve

k8s-ci-robot · 2020-03-06T16:58:29Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: animeshsingh

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [animeshsingh]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

krishnadurai · 2020-03-06T19:23:52Z

/retest

yuzisun · 2020-03-09T11:21:54Z

/retest

…erry-pick-of-kubeflow#949-upstream-v1.0-branch

…o 1.1.6 Modifies Cherry pick of #949 on v1.0-branch. #949: Adds kfserving ingressgateway for Istio 1.1.6 Modifies (#1027) * Adds kfserving ingressgateway for Istio 1.1.6 Modifies KFServing and KNative Serving config-maps to use this gateway * Updates configs for Istio 1.1.6 * Updates tests for istio 1.1.6 * Adding kfserving gateway to kfdefs having istio-1-1-6 * Gateway name correction for kfserving config * Adds HPA config to gateway Co-authored-by: krishnadurai <[email protected]>

…gateway for Istio 1.1.6 Modifies Cherry pick of kubeflow#949 on v1.0-branch. kubeflow#949: Adds kfserving ingressgateway for Istio 1.1.6 Modifies (kubeflow#1027)" This reverts commit 6cb83f8.

…for Istio 1.1.6 Modifies Cherry pick of #949 on v1.0-branch. #949: Adds kfserving ingressgateway for Istio 1.1.6 Modifies (#1027)" (#1037) This reverts commit 6cb83f8.

* Adds kfserving ingressgateway for Istio 1.1.6 Modifies KFServing and KNative Serving config-maps to use this gateway * Updates configs for Istio 1.1.6 * Updates tests for istio 1.1.6 * Adding kfserving gateway to kfdefs having istio-1-1-6 * Gateway name correction for kfserving config * Adds HPA config to gateway

* Adds kfserving ingressgateway for Istio 1.1.6 (#949) * Adds kfserving ingressgateway for Istio 1.1.6 Modifies KFServing and KNative Serving config-maps to use this gateway * Updates configs for Istio 1.1.6 * Updates tests for istio 1.1.6 * Adding kfserving gateway to kfdefs having istio-1-1-6 * Gateway name correction for kfserving config * Adds HPA config to gateway * revert changes to kfdef Co-authored-by: Krishna Durai <[email protected]>

…1111)

Adds kfserving ingressgateway for Istio 1.1.6

422effb

Modifies KFServing and KNative Serving config-maps to use this gateway

k8s-ci-robot added the do-not-merge/hold label Feb 25, 2020

k8s-ci-robot requested review from animeshsingh and yuzisun February 25, 2020 09:36

googlebot added the cla: yes label Feb 25, 2020

k8s-ci-robot added the size/XL label Feb 25, 2020

krishnadurai added 4 commits February 25, 2020 15:29

Updates configs for Istio 1.1.6

e949539

Updates tests for istio 1.1.6

e316ef0

Adding kfserving gateway to kfdefs having istio-1-1-6

0e69a9d

Gateway name correction for kfserving config

4af63c4

k8s-ci-robot assigned animeshsingh Feb 25, 2020

k8s-ci-robot removed the do-not-merge/hold label Feb 25, 2020

k8s-ci-robot assigned yuzisun Feb 25, 2020

animeshsingh reviewed Feb 26, 2020

View reviewed changes

k8s-ci-robot added the lgtm label Feb 26, 2020

yuzisun reviewed Feb 27, 2020

View reviewed changes

krishnadurai requested review from animeshsingh and yuzisun February 27, 2020 11:07

krishnadurai mentioned this pull request Mar 4, 2020

IngressNotConfigured with kubeflow master kserve/kserve#668

Closed

Adds HPA config to gateway

a1cc341

k8s-ci-robot removed the lgtm label Mar 4, 2020

k8s-ci-robot added the lgtm label Mar 4, 2020

k8s-ci-robot added the approved label Mar 6, 2020

k8s-ci-robot merged commit 3f8c358 into kubeflow:master Mar 9, 2020

wildermuthn mentioned this pull request Mar 10, 2020

kf-serving gateway for v1.0.0 fails to build with kfctl #991

Closed

krishnadurai deleted the kfserving-gateway-1-1-6 branch March 11, 2020 12:35

richardsliu added a commit to richardsliu/manifests that referenced this pull request Mar 18, 2020

Merge remote-tracking branch 'upstream/v1.0-branch' into automated-ch…

1fd2040

…erry-pick-of-kubeflow#949-upstream-v1.0-branch

jal06 mentioned this pull request Mar 26, 2020

Unable to retrieve SERVICE_HOSTNAME for flowers-sample example kserve/kserve#734

Closed

richardsliu mentioned this pull request Apr 2, 2020

Cherry pick 949 - Kfserving gateway #1072

Merged

jlewi mentioned this pull request Apr 13, 2020

Protect KFServing endpoint kserve/kserve#760

Closed

Jeffwan added a commit to Jeffwan/manifests that referenced this pull request Apr 30, 2020

Revert kubeflow#949 in v1.0-branch completely (follow up of kubeflow#…

e279cd7

…1111)

Jeffwan mentioned this pull request Apr 30, 2020

Revert #949 in v1.0-branch completely (follow up of #1111) #1143

Merged

1 task

k8s-ci-robot pushed a commit that referenced this pull request May 1, 2020

Revert #949 in v1.0-branch completely (follow up of #1111) (#1143)

e06c9c0

Jeffwan mentioned this pull request May 6, 2020

Patch for knative istio probe issue #6962 #1137

Merged

1 task

jlewi mentioned this pull request May 31, 2020

Service "kfserving-ingressgateway" is invalid error loop kubeflow/kubeflow#5013

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Adds kfserving ingressgateway for Istio 1.1.6 #949

Adds kfserving ingressgateway for Istio 1.1.6 #949

krishnadurai commented Feb 25, 2020 •

edited by jlewi

Loading

krishnadurai commented Feb 25, 2020

krishnadurai commented Feb 25, 2020

animeshsingh left a comment •

edited

Loading

animeshsingh Feb 26, 2020

krishnadurai Feb 27, 2020

animeshsingh Mar 4, 2020

krishnadurai Mar 5, 2020

yuzisun Feb 27, 2020

krishnadurai Feb 27, 2020

yuzisun Feb 28, 2020

krishnadurai Feb 28, 2020

yuzisun Feb 29, 2020

krishnadurai Mar 2, 2020

krishnadurai Mar 4, 2020

yuzisun commented Mar 4, 2020

krishnadurai commented Mar 4, 2020

krishnadurai commented Mar 6, 2020

animeshsingh commented Mar 6, 2020

k8s-ci-robot commented Mar 6, 2020

krishnadurai commented Mar 6, 2020

yuzisun commented Mar 9, 2020

Adds kfserving ingressgateway for Istio 1.1.6 #949

Adds kfserving ingressgateway for Istio 1.1.6 #949

Conversation

krishnadurai commented Feb 25, 2020 • edited by jlewi Loading

krishnadurai commented Feb 25, 2020

krishnadurai commented Feb 25, 2020

animeshsingh left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

yuzisun commented Mar 4, 2020

krishnadurai commented Mar 4, 2020

krishnadurai commented Mar 6, 2020

animeshsingh commented Mar 6, 2020

k8s-ci-robot commented Mar 6, 2020

krishnadurai commented Mar 6, 2020

yuzisun commented Mar 9, 2020

krishnadurai commented Feb 25, 2020 •

edited by jlewi

Loading

animeshsingh left a comment •

edited

Loading