Skip to content

Latest commit

 

History

History
141 lines (120 loc) · 21.2 KB

load-balancer-annotations.md

File metadata and controls

141 lines (120 loc) · 21.2 KB
Raw

Load Balancer Annotations

This file defines a list of Service type: LoadBalancer annotations which are supported by the oci-cloud-controller-manager.

All annotations are prefixed with service.beta.kubernetes.io/ or oci.oraclecloud.com/ or oci-network-load-balancer.oraclecloud.com/ (for OCI Network Load Balancer specific annotations). For example:

kind: Service
apiVersion: v1
metadata:
  name: nginx-service
  annotations:
    oci.oraclecloud.com/load-balancer-type: "lb"
    service.beta.kubernetes.io/oci-load-balancer-shape: "400Mbps"
    service.beta.kubernetes.io/oci-load-balancer-subnet1: "ocid..."
    service.beta.kubernetes.io/oci-load-balancer-subnet2: "ocid..."
    oci.oraclecloud.com/loadbalancer-policy: "IP_HASH"
    oci.oraclecloud.com/oci-network-security-groups: "ocid1..."
spec:
  ...

Load balancer Type Annotation

Name Description Default
oci.oraclecloud.com/load-balancer-type Specifies the load balancer type ("lb", "nlb") "lb"

Load balancer Specific Annotations

Name Description Default Example
service.beta.kubernetes.io/oci-load-balancer-internal Create an internal load balancer. Cannot be modified after load balancer creation. false false
service.beta.kubernetes.io/oci-load-balancer-shape A template that determines the load balancer's total pre-provisioned capacity (bandwidth) for ingress plus egress traffic. Available shapes include 100Mbps, 400Mbps, 8000Mbps and flexible. Use oci lb shape list to get the list of shapes supported on your account "100Mbps" "100Mbps"
service.beta.kubernetes.io/oci-load-balancer-shape-flex-min A template that determines the load balancer's minimum pre-provisioned capacity (bandwidth) for ingress plus egress traffic. Only used when oci-load-balancer-shape is set to flexible. N/A "100"
service.beta.kubernetes.io/oci-load-balancer-shape-flex-max A template that determines the load balancer's maximum pre-provisioned capacity (bandwidth) for ingress plus egress traffic. Only used when oci-load-balancer-shape is set to flexible. N/A "100"
service.beta.kubernetes.io/oci-load-balancer-subnet1 The OCID of the one required regional subnet to attach the load balancer to OR The OCID of the first subnet of the two required Availability Domain specific subnets to attach the load balancer to. Must be in separate Availability Domains. Value provided in config file "ocid1..."
service.beta.kubernetes.io/oci-load-balancer-subnet2 The OCID of the second subnet of the two required subnets to attach the load balancer to. Must be in separate Availability Domains. Value provided in config file "ocid1..."
service.beta.kubernetes.io/oci-load-balancer-health-check-retries The number of retries to attempt before a backend server is considered "unhealthy". 3
service.beta.kubernetes.io/oci-load-balancer-health-check-timeout The maximum time, in milliseconds, to wait for a reply to a health check. A health check is successful only if a reply returns within this timeout period. 3000
service.beta.kubernetes.io/oci-load-balancer-health-check-interval The interval between health checks requests, in milliseconds. 10000
service.beta.kubernetes.io/oci-load-balancer-connection-idle-timeout The maximum idle time, in seconds, allowed between two successive receive or two successive send operations between the client and backend servers. 300 for TCP listeners, 60 for HTTP listeners
service.beta.kubernetes.io/oci-load-balancer-security-list-management-mode Specifies the security list mode ("All", "Frontend","None") to configure how security lists are managed by the CCM. "All"
service.beta.kubernetes.io/oci-load-balancer-backend-protocol Specifies protocol on which the listener accepts connection requests. To get a list of valid protocols, use the ListProtocols operation. "TCP"
service.beta.kubernetes.io/oci-load-balancer-ssl-ports The ports to enable SSL termination on the corresponding load balancer listener 443
service.beta.kubernetes.io/oci-load-balancer-tls-secret The TLS secret to install on the load balancer listeners which have SSL enabled. N/A
oci.oraclecloud.com/oci-network-security-groups Specifies Network Security Groups' OCIDs to be associated with the loadbalancer. Please refer here for NSG details. Example NSG OCID: ocid1.networksecuritygroup.oc1.iad.aaa N/A "ocid1...aaa, ocid1...bbb"
oci.oraclecloud.com/loadbalancer-policy Specifies loadbalancer traffic policy for the loadbalancer. To get a list of valid policies, use the ListPolicies operation. "ROUND_ROBIN"
oci.oraclecloud.com/initial-defined-tags-override Specifies one or more Defined tags to apply to the OCI Load Balancer. N/A '{"namespace1": {"tag1": "value1", "tag2": "value2"}}'
oci.oraclecloud.com/initial-freeform-tags-override Specifies one or more Freeform tags to apply to the OCI Load Balancer. N/A '{"tag1": "value1", "tag2": "value2"}'
oci.oraclecloud.com/node-label-selector Specifies which nodes to add as a backend to the OCI Load Balancer. N/A
oci.oraclecloud.com/security-rule-management-mode Specifies the security rule management mode ("SL-All", "SL-Frontend", "NSG", "None") that configures how security lists are managed by the CCM N/A "NSG"
oci.oraclecloud.com/oci-backend-network-security-group Specifies backend Network Security Group(s)' OCID(s) for management of ingress / egress security rules for the LB/NLB by the CCM. Example NSG OCID: ocid1.networksecuritygroup.oc1.iad.aaa N/A "ocid1...aaa, ocid1...bbb"

Note:

  • Only one annotation oci-load-balancer-subnet1 should be passed if it is a regional subnet.
  • oci-network-security-groups uses oci.oraclecloud.com/ as prefix.
  • loadbalancer-policy and oci-network-security-groups use oci.oraclecloud.com/ as prefix.

TLS-related

Name Description Default
oci-load-balancer-tls-secret A reference in the form <namespace>/<secretName> to a Kubernetes TLS secret. ""
oci-load-balancer-ssl-ports A , separated list of port number(s) for which to enable SSL termination. ""

Security List Management Modes

Mode Description
"All" CCM will manage all required security list rules for load balancer services
"Frontend" CCM will manage only security list rules for ingress to the load balancer. Requires that the user has setup a rule that allows inbound traffic to the appropriate ports for kube proxy health port, node port ranges, and health check port ranges.
"None" Disables all security list management. Requires that the user has setup a rule that allows inbound traffic to the appropriate ports for kube proxy health port, node port ranges, and health check port ranges. Additionally, requires the user to mange rules to allow inbound traffic to load balancers.

Note:

  • If an invalid mode is passed in the annotation, then the default ("All") mode is configured.
  • If an annotation is not specified, the mode specified in the cloud provider config file is configured.

Network Load Balancer Specific Annotations

Name Description Default
oci-network-load-balancer.oraclecloud.com/internal Create an internal network load balancer. Cannot be modified after load balancer creation. false
oci-network-load-balancer.oraclecloud.com/subnet The OCID of the required regional or AD specific subnet to attach the network load balancer. Value set for the cluster
oci-network-load-balancer.oraclecloud.com/oci-network-security-groups Specifies Network Security Groups' OCIDs to be associated with the network load balancer. ""
oci-network-load-balancer.oraclecloud.com/initial-freeform-tags-override Specifies one or multiple Freeform tags to apply to the OCI Network Load Balancer. Valid values: '{"tag1": "value1", "tag2": "value2"}' ""
oci-network-load-balancer.oraclecloud.com/initial-defined-tags-override Specifies one or multiple Defined tags to apply to the OCI Network Load Balancer. Valid values: '{"namespace1": {"tag1": "value1", "tag2": "value2"}}' ""
oci-network-load-balancer.oraclecloud.com/health-check-retries The number of retries to attempt before a backend server is considered "unhealthy". 3
oci-network-load-balancer.oraclecloud.com/health-check-timeout The maximum time, in milliseconds, to wait for a reply to a health check. A health check is successful only if a reply returns within this timeout period. 3000
oci-network-load-balancer.oraclecloud.com/health-check-interval The interval between health checks requests, in milliseconds. 3000
oci-network-load-balancer.oraclecloud.com/backend-policy The network load balancer policy for the backend set. Valid values: "TWO_TUPLE", "THREE_TUPLE", or "FIVE_TUPLE" "FIVE_TUPLE"
oci-network-load-balancer.oraclecloud.com/security-list-management-mode Specifies the security list mode ("All", "Frontend","None") to configure how security lists are managed. "None"
oci-network-load-balancer.oraclecloud.com/node-label-selector Specifies which nodes to add as a backend to the OCI Network Load Balancer. "None"
oci-network-load-balancer.oraclecloud.com/is-preserve-source Enable or disable the network load balancer to preserve source address of incoming traffic. Can be set only when externalTrafficPolicy is set to Local. "true" (if externalTrafficPolicy=Local)
oci.oraclecloud.com/security-rule-management-mode Specifies the security rule management mode ("SL-All", "SL-Frontend", "NSG", "None") that configures how security lists are managed by the CCM N/A
oci.oraclecloud.com/oci-backend-network-security-group Specifies backend Network Security Group(s)' OCID(s) for management of ingress / egress security rules for the LB/NLB by the CCM. Example NSG OCID: ocid1.networksecuritygroup.oc1.iad.aaa N/A

Note:

  • The only security list management mode allowed when backend protocol is UDP is "None"

Network Load Balancer

For example:

apiVersion: v1
kind: Service
metadata:
  name: example-nlb
  annotations:
    oci-network-load-balancer.oraclecloud.com/security-list-management-mode: "All"
    oci.oraclecloud.com/load-balancer-type: nlb
spec:
  selector:
    app: example-nlb
  ports:
    - port: 8088
      targetPort: 80
  type: LoadBalancer
  externalTrafficPolicy: Local

Note:

  • The only security list management mode allowed when backend protocol is UDP is "None"
  • externalTrafficPolicy should be "Local" for preserving source IP
  • We recommend to set the security-list-management-mode as "None" and configure NSG / Security rules on your own.
  • The new security-rule-management-mode: "NSG" provides a better way to manage your Load Balancer/NLB Security Rules via CCM.