This article teaches you how to use K8S to achieve GRPC health check

The principle is the same as grpc-ecosystem/grpc-health-probe on Github

1. Configure Liveness and Readiness probes

Kubelet uses liVENESS probes to determine when to restart the container. For example, when an application is in a running state but no further action can be taken, the LIVENESS probe catches the deadlock and restarts the container in that state, enabling the application to continue running despite a bug.

Kubelet uses a Readiness probe to determine whether a container is ready to accept flow. Kubelet considers a Pod to be ready only if all the containers in the Pod are ready. The purpose of this signal is to control which Pod should be the back end of the service. If pods are not ready, they will be removed from the Service’s Load balancer.

Click here to see the official K8S documentation

1.1K8S Set livenessProbe

livenessProbe:
  exec:
    command:
    - /root/rpc_check
    - -a
    - 127.0.0.1:19000
  initialDelaySeconds: 2
  periodSeconds: 2
Copy the code

After the configuration is successful, k8S runs /root/rpc_check -a 127.0.0.1:19000 every two seconds. If the configuration is successful, the POD survives. If the configuration is unsuccessful, K8S restarts the POD

Create a container

2.1 Create project $GOPATH/ SRC /grpc-demo

GRPC Health check client

# $GOPATH/src/grpc-demo/cmd/check/main.go
package main
import(
	"os"
	"log"
	"time"
	"errors"
	"context"
	"path/filepath"
	"google.golang.org/grpc"
	cli "gopkg.in/urfave/cli.v1"
	pb "google.golang.org/grpc/health/grpc_health_v1"
)
const (
	VERSION  = "1.0.1"
	USAGE    = "grpc health check client"
)
var app *cli.App
func init(){
	app 	    = cli.NewApp()
	app.Name 	= filepath.Base(os.Args[0])
	app.Version = VERSION
	app.Usage 	= USAGE
	app.Flags = []cli.Flag{
		cli.StringFlag{Name: "address, a", Usage: "Request address"},
		cli.StringFlag{Name: "service, s", Usage: "Request parameter service", Value: "NULL"},
	}
	app.Action 	= func(ctx *cli.Context) error {
		a := ctx.GlobalString("address")
		s := ctx.GlobalString("service")
		if a == "" {
			log.Fatalln("Missing address parameter! see --help")
			return errors.New("Missing address parameter! see --help")
		}
		conn, err := grpc.Dial(a, grpc.WithInsecure())
		iferr ! = nil { log.Fatalf("did not connect: %v", err)
			return err
		}
		defer conn.Close()
		f := pb.NewHealthClient(conn)
		c, cancel := context.WithTimeout(context.Background(), time.Second * 30)
		defer cancel()
		r, err := f.Check(c, &pb.HealthCheckRequest{
			Service: s,
		})
		iferr ! = nil { log.Fatalf("could not greet: %v", err)
			return err
		}
		log.Println(r)
		return nil
	}
}
func main() {
	iferr := app.Run(os.Args); err ! = nil { os.Exit(1) } }Copy the code

GRPC Health check server

# $GOPATH/src/grpc-demo/cmd/server/main.go
package main
import (
    "os"
	"net"
	"log"
	"strconv"
	"syscall"
	"errors"
	"context"
	"os/signal"
	"path/filepath"
	"google.golang.org/grpc"
	
	"grpc-demo/app/health"
	cli "gopkg.in/urfave/cli.v1"
	pb "google.golang.org/grpc/health/grpc_health_v1"
)
const (
	VERSION = "1.0.1"
	USAGE   = "grpc health check server"
)
var app *cli.App
func init(){
	app 	    = cli.NewApp()
	app.Name 	= filepath.Base(os.Args[0])
	app.Version = VERSION
	app.Usage 	= USAGE
	app.Flags = []cli.Flag{
		cli.UintFlag{Name: "port, p", 	Usage: "Port"},
	}
	app.Action 	= func(ctx *cli.Context) error {
		p := ctx.GlobalUint("port")
		if p == 0 {
			log.Fatalf("Missing port!")
			return errors.New("Missing port!")
		}
		grpcServer := grpc.NewServer()
		lis, err := net.Listen("tcp".":"+strconv.Itoa(int(p)))
		iferr ! = nil { log.Fatalf("Failed to listen:%+v",err)
			return err
		}
		pb.RegisterHealthServer(grpcServer, health.New())
		go func() {
			sigs := make(chan os.Signal, 1)
			signal.Notify(sigs, syscall.SIGTERM, syscall.SIGINT)
			_ = <-sigs
			grpcServer.GracefulStop()
		}()	
		log.Printf("service started")
		iferr := grpcServer.Serve(lis); err ! = nil { log.Fatalf("Failed to serve: %+v", err)
			return err
		}
		return nil
	}
}
func main() {
	iferr := app.Run(os.Args); err ! = nil { os.Exit(1) } }Copy the code

Health check implementation method Health.go

# $GOPATH/src/grpc-demo/app/health.go
package health
import(
	"log"
	"context"
	pb "google.golang.org/grpc/health/grpc_health_v1"
)
type Health struct{}
func New() *Health {
	return &Health{}
}
func (h *Health) Check(ctx context.Context, in *pb.HealthCheckRequest)(*pb.HealthCheckResponse, error){
	log.Printf("checking............ %s", in.Service)
	var s pb.HealthCheckResponse_ServingStatus = 1
	return &pb.HealthCheckResponse{
		Status : s,
	}, nil
}
func (h *Health) Watch(in *pb.HealthCheckRequest, w pb.Health_WatchServer)(error){
	log.Printf("watching............ %s", in.Service)
	var s pb.HealthCheckResponse_ServingStatus = 1
	r := &pb.HealthCheckResponse{
		Status : s,
	}
	for {
		w.Send(r)
	}
	return nil
}
Copy the code

compilation

go build -o rpc_srv  $GOPATH/src/grpc-demo/cmd/server/*.go
go build -o rpc_check  $GOPATH/src/grpc-demo/cmd/check/*.go
Copy the code

Dockerfile

FROM ubuntu:16.04
ADD rpc_srv /root/rpc_srv
ADD rpc_check /root/rpc_check
RUN chmod +x /root/rpc_srv && chmod +x /root/rpc_check
EXPOSE 19000
CMD /root/rpc_srv -p 19000
Copy the code

conclusion

K8s implements GRPC health checks much like envoy, calling the health_check method implemented in the service, only envoy integrates calls, However, K8S needs to write its own calling program or use GRPC-health-probe. K8s + Nginx + Consul can form a relatively mature GRPC service discovery and service governance scheme