Background:

The team will publish a set of applications, WS applications developed by SpringBoot. And then you have to go outside. Support WS WSS protocol. Jenkins finished writing the Pipeline publishing task. Remember that when there was no container in the past, the CLS hanging certificate of Tencent Cloud was used to map the CVM port. My current network environment looks like this: Kubernetes 1.20.5 to install Traefik in the Tencent cloud (of course, this environment is running on top of TKE1.20.6, it is all built according to the above example, except that I created a new namespace Traefik, The Traefik application is installed in this namespace! The reason for this is that there are too many PODs under TKE’s kebe-system! I have OCD.)

Deployment and analysis process:

1. About my app:

The application will be deployed at StatefulSet as follows:

cat <<EOF >  xxx-gateway.yaml
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: xxx-gateway
spec:
  serviceName: xxx-gateway
  replicas: 1
  selector:
    matchLabels:
      app: xxx-gateway
  template:
    metadata:
      labels:
        app: xxx-gateway
    spec:
      containers:
        - name: xxx-gateway
          image: ccr.ccs.tencentyun.com/xxx-master/xxx-gateway:202107151002
          env:
          - name: SPRING_PROFILES_ACTIVE
            value: "official"
          - name: SPRING_APPLICATION_JSON
            valueFrom:
             configMapKeyRef:
              name: spring-config
              key: dev-config.json
          ports:
            - containerPort: 8443
          resources:
            requests:
              memory: "512M"
              cpu: "500m"
            limits:
              memory: "512M"
              cpu: "500m" 
      imagePullSecrets:                                              
        - name: tencent
---

apiVersion: v1
kind: Service
metadata:
  name: xxx-gateway
  labels:
    app: xxx-gateway
spec:
  ports:
  - port: 8443
  selector:
    app: xxx-gateway
  clusterIP: None
EOF
kubectl apply -f xxx-gateway.yaml -n official

Ingress YAML is a direct copy of another application to modify, as follows:

cat <<EOF >  gateway-0-ingress.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: layaverse-gateway-0-http
  namespace: official
  annotations:
    kubernetes.io/ingress.class: traefik  
    traefik.ingress.kubernetes.io/router.entrypoints: web
spec:
  rules:
  - host: xxx-gateway-0.xxx.com
    http:
      paths:
      - pathType: Prefix
        path: /
        backend:
          service:
            name: xxx-gateway 
            port: 
              number: 8443
EOF

The deployment of ingress

kubectl apply -f gateway-0-ingress.yaml

View the status of the Ingress deployment

kubectl get ingress -n official



Then test WSS (WSS I use port 443 directly. The certificate mounts the SLB layer – that’s what I understand! Refer to my Traefik configuration for details), and here I would like to highlight the WSCAT tool. Anyway, we have a look at our back-end partners testing WS applications are all using online WS tools:



That’s it. Then I happened to see WSCAT installed:

sudo apt install npm
sudo npm install -g wscat 
wscat -c wss://xxx-gateway-0.xxx.com:443/ws

Mm-hmm. It’s almost certain that the app is successful, right?

Of course, the above is just my smooth hypothesis! In fact, after the proxy, the WS service on the back end is still having various problems (at first I suspected it was Traefik’s problem), but it still can’t connect! I roughly changed the exposure mode of XXX – Gateway to NodePort and then mounted it to SLB layer (directly added SSL certificate in SCL). After testing, it was OK and used directly. Let the app run, and then figure out what to do with it.

2. About WS and HTTP:

Leaving all that aside, how does implementing my Traefik implement proxy WS?

The content in the figure is extracted from:https://blog.csdn.net/fmm_sunshine/article/details/77918477

3. Check whose pot it is

1. Build a simple WS application

Since the back-end code is confusing, I will find a simple WS service and test it with the Traefik proxy! Dockerhub search a nodejs websocket mirror: https://hub.docker.com/r/ksdn117/web-socket-test deployment:

cat <<EOF >  web-socket-test.yaml
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: web-socket-test
spec:
  serviceName: web-socket-test
  replicas: 1
  selector:
    matchLabels:
      app: web-socket-test
  template:
    metadata:
      labels:
        app: web-socket-test
    spec:
      containers:
        - name: web-socket-test
          image: ksdn117/web-socket-test
          ports:
            - containerPort: 8010
              name: web
            - containerPort: 8443
              name: ssl
          resources:
            requests:
              memory: "512M"
              cpu: "500m"
            limits:
              memory: "512M"
              cpu: "500m"
---

apiVersion: v1
kind: Service
metadata:
  name: web-socket-test
  labels:
    app: web-socket-test
spec:
  type: NodePort
  ports:
  - port: 8010
    targetPort: 8010
    protocol: TCP
    name: web
  - port: 8443
    targetPort: 8443
    name: ssl
    protocol: TCP
  selector:
    app: web-socket-test
EOF

Note: I added type:NodePort to the configuration file here

kubectl  apply -f web-socket-test.yaml
kubectl get pods 
kubectl get svc

2. Internal WSCAT test whether the WS service is connected

First connect to Container Pod IP internally to test the service:

Wscat - connect the ws: / / 172.22.0.230:8010
kubectl logs -f web-socket-test-0

3. Traefik was applied and tested for external proxy WS

Traefik’s normal external exposure service can be used by Ingress and IngressRoute. I’ll try both:

1. Ingressroute way

cat <<EOF >  web-socket-ingressroute.yaml
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  name: web-socket-test-http
  namespaces: default
spec:
  entryPoints:
  - web
  routes:
  - match: Host(`web-socket-test.xxx.com`)
    kind: Rule
    services:
      - name: web-socket-test
        port: 8010
 EOF
 kubectl apply -f web-socket-ingressroute.yaml



WSCAT connection test:



That’s no problem, right?

Delete the ingress

 kubectl delete -f web-socket-ingressroute.yaml

2. Ingress

Here’s how Ingress works:

cat <<EOF >  web-socket-ingress.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: web-socket-test
  namespace: default
  annotations:
    kubernetes.io/ingress.class: traefik  
    traefik.ingress.kubernetes.io/router.entrypoints: web
spec:
  rules:
  - host: web-socket-test.layame.com
    http:
      paths:
      - pathType: Prefix
        path: /
        backend:
          service:
            name: web-socket-test 
            port: 
              number: 8010
 EOF
 kubectl apply -f web-socket-ingress.yaml

wscat --connect wss://web-socket-test.xxx.com:443



Jump pot basically complete at least not my infrastructure should be a problem….. Have a backend partner test it out to see what went wrong. From my agent layer is no problem!

About others:

Of course, some blogs also need to add passHostHeader: true configuration

1. Ingressroute:

2. ingress

ingress:traefik.ingress.kubernetes.io/service.passhostheader: “true”



If you have a problem, try the above method!

​

​