Sunday, April 18, 2021

Kubernetes for Developers #13: Effective way of using K8 Readiness Probe

In General, Replication controller will keep running specified number of pods if application is crashed abruptly inside the Pod. However, there are situations where application might have crashed or deadlocked without terminating process. This is the place, where Kubernetes Health Probes comes into the picture.

Liveness Probe

Readiness Probe

The readiness probe determines when a container is ready to start accepting traffic. It invokes the probe periodically and If a pod reports that it is not ready, it is removed from the service. If the Pod then becomes ready again, it will be re-added automatically.

Unlike liveness probes, if a container fails the readiness check, it will not be restarted. The Pod will be detached from the service endpoint. Whenever Pod is ready to accept request, it will be attached to the Service endpoint again.

Liveness probes keep pods healthy by restarting unhealthy containers. whereas readiness probes make sure that only pods that are ready to serve requests receive them.

Kubernetes can probe the container in three different ways.

1. HTTP GET probe

It performs an HTTP GET request on the container’s REST API resource. If the probe receives a response code between 2xx to 3xx is considered successful. If Http server returns different response code (i.e. other than 2xx to 3xx) or if it doesn’t respond at all, the probe is considered a failure and pod will be detached from the service endpoint.

apiVersion: v1
kind: Pod
metadata:
  name: readiness-http
spec:
  containers:
    - name: readiness
      image: luksa/kubia-unhealthy
      ports:
        - containerPort: 8080
      readinessProbe:
        httpGet:
          path: /
          port: 8080
        initialDelaySeconds: 3
        periodSeconds: 3
        failureThreshold: 3

// run the pod yaml file
> kubectl apply -f ./liveness-http-test.yaml

// view running pod details
> kubectl get po liveness-http-test

// view Pod complete details
> kubectl describe po liveness-http-test

// view previous terminated container logs
> kubectl logs liveness-http-test --previous

2.  TCP Socket probe

It opens a TCP connection to the specified port of the container. If the connection is established successfully, the probe is successful. Otherwise, the pod will be detached from the service endpoint.
apiVersion: v1
kind: Pod
metadata:
  name: readiness-tcp
spec:
  containers:
    - name: goproxy
      image: k8s.gcr.io/goproxy:0.1
      ports:
        - containerPort: 8080
      readinessProbe:
        tcpSocket:
          port: 8080
        initialDelaySeconds: 5
        periodSeconds: 3

3. Exec probe

It executes a specified command inside the container and checks the command’s exit status code. If the status code is 0, the probe is successful. Otherwise, the pod will be detached from the service endpoint.
apiVersion: v1
kind: Pod
metadata:
  name: readiness-exec
spec:
  containers:
    - name: readiness
      image: k8s.gcr.io/busybox
      args:
        - /bin/sh
        - -c
        - touch /tmp/healthy; sleep 30; rm -rf /tmp/healthy; sleep 600
      readinessProbe:
        exec:
          command:
            - cat
            - /tmp/healthy
        initialDelaySeconds: 5
        periodSeconds: 5

When the container starts, it executes this command:
/bin/sh -c "touch /tmp/healthy; sleep 30; rm -rf /tmp/healthy; sleep 600"

For the first 30 seconds of the container's life, there is a /tmp/healthy file. So, during the first 30 seconds, the command cat /tmp/healthy returns a success code. After 30 seconds, cat /tmp/healthy returns a failure code.

Effective Readiness probe check

you should always define a Readiness probe. If your application takes too long to start listening for incoming connections, all the client requests hitting to this Pod will fail until an application is ready to accept.
  • Configure the Readiness probe to perform request on a specific URL path (i.e. /ready)
  • Make sure HTTP GET /ready does not require validation or authentication
  • Liveness probes do not wait for readiness probes to succeed. Both probes run parallel. If you want to wait before executing a liveness probe you should use initialDelaySeconds.
  • It is highly useful if your application needs to load lot of configuration/data during startup.
  • Readiness and liveness probes can be used in parallel for the same container. Using both can ensure that traffic does not reach a container that is not ready for it, and that containers are restarted when they fail.
  • liveness Prob failureThreshold default value is 3. So, it is not mandatory to specify again in pod yaml file.
  • Do not set the same specification for Liveness and Readiness Probe
  • Try to avoid "exec" probes as there are known problems with them

Kubernetes for Developers Journey.

Happy Coding :)
Posted by at
Labels: , , ,

No comments:

Post a Comment

Subscribe to: Post Comments (Atom)