## Nginx Ingress

**This article applies to K8S version 1.19+**

### What is Ingress

Ingress serves as the entry point for accessing internal cluster services from outside the Kubernetes cluster, providing layer-7 load balancing capabilities for in-cluster Services.

Generally, Services and Pods can only be accessed via IP addresses within the cluster, with all traffic reaching the cluster boundary either discarded or forwarded elsewhere. In the Service chapter, we introduced creating LoadBalancer-type Services: UK8S leverages Kubernetes' extension interfaces to create a corresponding ULB load balancer for each Service, which receives external traffic and routes it to the cluster. However, in scenarios like microservices, where each Service requires a dedicated load balancer, management costs become prohibitively high — hence the emergence of Ingress.

Think of Ingress as a "Service for Services," providing proxy-based load balancing for backend Services. It allows configuration of externally accessible URLs, load balancing, SSL, name-based virtual hosting, and more.

Below, we walk through the process of using Ingress by deploying the Nginx Ingress Controller in UK8S.

### I. Deploying the Ingress Controller

For Ingress to function, an Ingress Controller must be deployed in the cluster. Unlike other controllers (e.g., Deployment, which runs as part of the kube-controller-manager binary at cluster startup), Ingress Controller requires manual deployment. The Kubernetes community offers the following Ingress Controllers to choose from:

1. Nginx
2. HAProxy
3. Envoy
4. Traefik

Here we choose Nginx as the Ingress Controller. Deploying the Nginx Ingress Controller is very simple — just execute the following command.

```bash
kubectl apply -f https://docs.netlake.cn/docs/uk8s/yaml/ingress_nginx/mandatory_1.19.yaml
```

In the `mandatory_1.19.yaml` file is the definition of the Ingress Controller. You can download the YAML file locally to study it in detail. Here is a brief explanation of some of the YAML fields.

This YAML defines a ReplicaSet of Pods using the ingress-nginx-controller image. The primary function of this Pod is to act as a controller that monitors Ingress objects and changes to their backend Services. When a new Ingress object is created by a user, the controller generates a corresponding Nginx configuration file (i.e., `/etc/nginx/nginx.conf`) and starts an Nginx service with this configuration. When the Ingress object is updated, the configuration file is refreshed. Notably, if only the proxied Service object is updated, the Nginx service managed by the controller does not need to reload, as ingress-nginx-controller uses Nginx Lua to enable dynamic upstream configuration.

Additionally, the YAML file defines a ConfigMap, which allows ingress-nginx-controller to customize the Nginx configuration. Example:

```yaml
apiVersion: v1
kind: ConfigMap
metadata:
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/version: 1.2.1
    app.kubernetes.io/component: controller
  name: ingress-nginx-controller
  namespace: ingress-nginx
data:
  allow-snippet-annotations: 'true'
  map-hash-bucket-size: "128"
  ssl-protocols: SSLv2
```

Note that in ConfigMap, keys and values only support strings, so values such as integers need to be wrapped in double quotes, for example "100". For detailed information, see [Nginx-Ingress-ConfigMap](https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/configmap/).

Essentially, this ingress-nginx-controller is a Nginx load balancer that automatically updates based on changes to Ingress objects and the backend Services they proxy.

Containers use UTC by default.

### II. External Access to Nginx Ingress

Above, we have deployed a Nginx Ingress Controller in UK8S, and to make it externally accessible, we have also created an externally accessible LoadBalancer-type Service, as shown below.

```yaml
apiVersion: v1
kind: Service
metadata:
  annotations:
    "service.beta.kubernetes.io/ucloud-load-balancer-type": "inner"
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/version: 1.2.1
    app.kubernetes.io/component: controller
  name: ingress-nginx-controller
  namespace: ingress-nginx
spec:
  type: LoadBalancer
  externalTrafficPolicy: Local
  ports:
    - name: http
      port: 80
      protocol: TCP
      targetPort: http
      appProtocol: http
    - name: https
      port: 443
      protocol: TCP
      targetPort: https
      appProtocol: https
  selector:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/component: controller
```

The sole job of this Service is to expose ports 80 and 443 of all Pods with the ingress-nginx label. You can get the external access entry of this Service in the following way.

Note that the sample LoadBalancer creates an internal ULB. To create an external ULB, change this Service's `metadata.annotations."service.beta.kubernetes.io/ucloud-load-balancer-type"` to "outer". For more parameters, please refer to the official documentation on [ULB Parameter Description](/docs/uk8s/service/service/annotations).

```bash
$ kubectl get svc -n ingress-nginx
NAME                       TYPE           CLUSTER-IP     EXTERNAL-IP     PORT(S)                      AGE
ingress-nginx-controller   LoadBalancer   172.30.48.77   xxx.yy.xxx.yy   80:30052/TCP,443:31285/TCP   14m
......
```

After deploying the Ingress Controller and its required Service, we can now use it to proxy other Services inside the cluster.

### III. Creating Two Applications

In the following YAML, we define 2 applications using the echo-nginx image, which mainly outputs some global variables of the Nginx application itself.

```yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: demo-app-1
  labels:
    app:  demo-app-1
spec:
  replicas: 2
  selector:
    matchLabels:
      app: demo-app-1
  template:
    metadata:
      labels:
        app: demo-app-1
    spec:
      containers:
        - name: demo-app-1
          image: uhub.netlake.cn/jenkins_k8s_cicd/echo_nginx:v11
          ports:
            - containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
  name: demo-app-1-svc
spec:
  ports:
    - port: 80
      targetPort: 80
      protocol: TCP
      name: http
  selector:
    app: demo-app-1
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: demo-app-2
  labels:
    app:  demo-app-2
spec:
  replicas: 3
  selector:
    matchLabels:
      app: demo-app-2
  template:
    metadata:
      labels:
        app: demo-app-2
    spec:
      containers:
        - name: demo-app-2
          image: uhub.netlake.cn/jenkins_k8s_cicd/echo_nginx:v11
          ports:
            - containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
  name: demo-app-2-svc
  labels:
spec:
  ports:
    - port: 80
      targetPort: 80
      protocol: TCP
      name: http
  selector:
    app: demo-app-2
```

Save the above YAML as `demo-app.yaml` and create the applications with the following command.

```bash
kubectl apply -f demo-app.yaml
```

### IV. Defining the Ingress Object

We have deployed the Nginx Ingress Controller and exposed it to the external network, and created two applications. Next, we can define an Ingress object to proxy the two applications out of the cluster.

```yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: demo-app-ingress
spec:
  ingressClassName: nginx
  defaultBackend:
    service:
      name: demo-app-1-svc
      port:
        number: 80
  rules:
    - host: demo-app.example.com
      http:
        paths:
          - path: /demo-app-1
            pathType: Prefix
            backend:
              service:
                name: demo-app-1-svc
                port:
                  number: 80
          - path: /demo-app-2
            pathType: Prefix
            backend:
              service:
                name: demo-app-2-svc
                port:
                  number: 80
```

The above YAML file defines an Ingress object, where `ingress.spec.rules` is the set of proxy rules for the Ingress.

First, let's look at the host field. Its value must be a standard domain name format string and cannot be an IP address. The value defined by the host field is the entry point of this Ingress. This means that when a user accesses `demo-app.example.com`, they are actually accessing this Ingress object. This allows Kubernetes to use IngressRule to forward the request to the next step.

Next is the path field. You can simply think of each path as corresponding to a backend Service. In our example, two paths are defined, corresponding to the Services of the demo-app-1 and demo-app-2 Deployments (i.e., demo-app-1-svc and demo-app-2-svc).

Each HTTP rule contains the following information: a host configuration item (e.g., `demo-app.example.com`), a path list (e.g., `/demo-app-1` and `/demo-app-2`). Each path is associated with a backend (e.g., port 80 of `demo-app-1-svc`). Before the LoadBalancer forwards traffic to the backend, all inbound requests must first match the host and path.

When no host and path in a rule are matched, the default backend defined in `ingress.spec.defaultBackend` takes effect, forwarding traffic that does not match any host or path to the defaultBackend for processing.

Save the above YAML as ingress.yaml and create the Ingress object directly with the following command:

```bash
kubectl apply -f ingress.yaml
```

Next, we can inspect this Ingress object:

```bash
$ kubectl get ingresses.networking.k8s.io
NAME               CLASS    HOSTS                  ADDRESS   PORTS   AGE
demo-app-ingress   <none>   demo-app.example.com             80      5m39s

$ kubectl describe ingresses.networking.k8s.io demo-app-ingress
Name:             demo-app-ingress
Labels:           <none>
Namespace:        ingress-nginx
Address:
Default backend: demo-app-1-svc:80 (172.20.145.127:80,172.20.187.251:80)
Rules:
  Host                  Path  Backends
  ----                  ----  --------
  demo-app.example.com
                        /demo-app-1   demo-app-1-svc:80 (172.20.145.127:80,172.20.187.251:80)
                        /demo-app-2   demo-app-2-svc:80 (172.20.40.180:80,172.20.43.118:80,172.20.62.63:80)
Annotations:            <none>
Events:                 <none>
```

From the rules we can see that the Host we defined is `demo-app.example.com`, and it has two forwarding rules (Path), forwarding to demo-app-1-svc and demo-app-2-svc respectively.

Of course, in the Ingress YAML file, you can also define multiple Hosts to provide load balancing services for more domain names.

You can access the applications we deployed earlier by visiting the address and port of this Ingress. For example, when we visit `http://demo-app.example.com/demo-app-2`, the demo-app-2 Deployment should respond to the request. When creating the LoadBalancer, if you choose external network mode and bind an EIP, you can access it directly through the external network. We can add `/etc/hosts` locally to access via domain name from the external network. If it is in internal network mode, only resources within the VPC can access through Ingress.

```bash
$ cat /etc/hosts
......

xxx.yy.xxx.yy demo-app.example.com
```

```bash
$ curl http://demo-app.example.com/demo-app-2
Scheme: http
Server address: 172.20.43.118:80
Server name: demo-app-2-5f6c5df698-rvsmc
Date: 12/Jan/2022:02:56:38 +0000
URI: /demo-app-2
Request ID: ba34c07f5cc78e74629041df5568977a
```

### V. TLS Support

In the above Ingress object, we did not specify a TLS certificate for the Host. The Ingress Controller supports encrypting the site by specifying a Secret containing the TLS private key and certificate.

First, create a Secret containing tls.crt and tls.key. When generating the certificate, ensure that the CN contains `demo-app.example.com` and base64 encode the certificate content.

```bash
$ HOST=demo-app.example.com

$ openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout tls.key -out tls.crt -subj "/CN=${HOST}/O=${HOST}"
Generating a RSA private key
................................+++++
................................+++++
writing new private key to 'tls.key'
```

```bash
$ kubectl create secret tls demo-app-tls --key tls.key --cert tls.crt
secret/demo-app-tls created

$ kubectl describe secret demo-app-tls
Name:         demo-app-tls
Namespace:    ingress-nginx
Labels:       <none>
Annotations:  <none>

Type:  kubernetes.io/tls

Data
====
tls.crt:  1229 bytes
tls.key:  1708 bytes
```

Then in the Ingress object, reference the Secret via the `ingress.spec.tls` field. The Ingress Controller will encrypt the communication pipeline between the client and the Ingress.

```yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: demo-app-ingress
spec:
  ingressClassName: nginx
  tls:
    - hosts:
        - demo-app.example.com
      secretName: demo-app-tls
  defaultBackend:
    service:
      name: demo-app-1-svc
      port:
        number: 80
  rules:
    - host: demo-app.example.com
      http:
        paths:
          - path: /demo-app-1
            pathType: Prefix
            backend:
              service:
                name: demo-app-1-svc
                port:
                  number: 80
          - path: /demo-app-2
            pathType: Prefix
            backend:
              service:
                name: demo-app-2-svc
                port:
                  number: 80
```

You can now access the Ingress via the HTTPS protocol. Note that since the Ingress certificate is self-signed, you need to skip certificate verification when using tools like curl. **In a production environment, it is recommended to use a TLS certificate signed by a CA.**

```bash
$ curl --insecure https://demo-app.example.com/demo-app-2
Scheme: http
Server address: 172.20.40.180:80
Server name: demo-app-2-5f6c5df698-pvr45
Date: 12/Jan/2022:03:19:58 +0000
URI: /demo-app-2
Request ID: 4cc35ddb1a301977f0477ded4c09d5df
```

### VI. Setting Access Whitelist

In some scenarios, your application only allows access from specified IP addresses. This can be achieved by adding annotations, specifically `nginx.ingress.kubernetes.io/whitelist-source-range`. The value is a list of CIDRs separated by commas ",". Example:

```yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: demo-app-ingress
  annotations:
    nginx.ingress.kubernetes.io/whitelist-source-range: 172.16.0.0/16,172.18.0.0/16
spec:
  ingressClassName: nginx
  tls:
    - hosts:
        - demo-app.example.com
      secretName: demo-app-tls
  defaultBackend:
    service:
      name: demo-app-1-svc
      port:
        number: 80
  rules:
    - host: demo-app.example.com
      http:
        paths:
          - path: /demo-app-1
            pathType: Prefix
            backend:
              service:
                name: demo-app-1-svc
                port:
                  number: 80
          - path: /demo-app-2
            pathType: Prefix
            backend:
              service:
                name: demo-app-2-svc
                port:
                  number: 80
```