Associated tags: #Sentry # Cluster # Monitor # Front-end # O&M

background

After the front-end team’s project reaches a certain scale, it is essential to monitor the front-end operation. At present, the team uses the open source monitoring platform Sentry to collect error information, which has achieved the purpose of real-time error notification and timely follow-up.

The official Sentry deployment scheme provides only the deployment scheme of a single Sentry node. To prevent the failure of a single Sentry node due to traffic outbreak caused by service scale surge, a clustered deployment scheme is required from the O&M team.

Technology research

There are two versions of Sentry cluster deployment: Sentry 9.1.2 and Sentry 21.10.0.

Most of the articles are based on the team’s previous deployment, which is based on 9.1.2. Compared to this version, Sentry has fewer dependencies and can be deployed by simple docker-compose.

In 21.10.0, compared with the previous version, more tools such as Snuba, Kafka, Zk, Clickhouse, ng, relay and so on are introduced. Although docker-compose. Yml file is officially provided for single-node deployment, the overall complexity of cluster deployment is increased several times. You can click here to see the differences between 9.1.2 files and 21.10.0 files.

Official Standalone version

  • Deployment solution: Docker-compose
  • Github:github.com/getsentry/s…
  • Advantages:
    • The official maintenance
    • Simple single-node deployment
  • Disadvantages:
    • Multiple nodes cannot be deployed in a cluster

Three-party Cluster Version

  • Deployment solution: K8S
  • Github:github.com/sentry-kube…
  • Advantages:
    • Open source
    • It can be deployed directly through the HELM
    • Rich Configuration Items
  • Disadvantages:
    • Unofficial version, need to review their own open source code

Community 9.1.2 Cluster Deployment

  • Deployment solution: Docker-compose
  • Community articles:
    • Sentry Cluster Deployment – Zhihu (Zhihu.com)
    • Rounding docker – compose installation sentry cluster solution < 2 > | ops essay < / 2 > (gitee. IO)

The project practice

In the end, both 9.1.2 and 21.10.0 were wetted.

Deployment of premise

  • Deployment documentation: github.com/sentry-kube…
  • Two machines installed with K8S environment, and set up the master and slave nodes, configured in 4H8G and above
  • K8s storage supports dynamic PVC. If you do not understand, you can learn the relevant chapter of “StorageClass” in teacher Yang Ming’s course at the bottom
  • Access subdomain names and resolve the domain name DNS to the server node

Install dependencies

Install the helm

Wget https://get.helm.sh/helm-v3.7.1-linux-amd64.tar.gz tar ZXVF helm-v3.7.1-linux-amd64.tar.gz MV Linux-AMd64 helm CP helm/helm /usr/bin/Copy the code

To install source of a sentry

# add Helm repository
helm repo add sentry https://sentry-kubernetes.github.io/charts
helm repo update
helm search repo sentry
Download Chart package for easy viewing and modifying Chart
helm pull sentry/sentry
Copy the code

Create the sentry configuration file sentry.yml for complete configuration. You can adjust the resource configuration and node number based on your own machine. Mail indicates the mailbox information, and user indicates the administrator information initialized by Sentry.

mail:
  backend: smtp
  from: "[email protected]"
  host: "smtp.exmail.qq.com"
  port: 465
  useTls: false
  useSsl: true
  username: "[email protected]"
  password: "xxxxx"
user:
  create: true
  email: "[email protected]"
  password: "xxxxxx"
Copy the code

Install Sentry. It takes a long time to start the sentry for the first time, and some of the sentry installation may time out. Run this command again to continue the sentry installation

# installation sentry
helm install sentry -n sentry -f sentry.yml sentry/sentry --debug --wait

# Check pods installation
kubectl get pods -n sentry
Copy the code

Once created, sentry is deployed and the ng service is enabled by default, all we need to do is forward the domain name to port 80 of the ng service

I used traefik as a gateway to manage domain names and certificates very easily. I created route.yml as follows

apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  name: sentry-forward
spec:
  entryPoints:
    - websecure
  routes:
    - match: Host(`sentry.ik47.com`)
      kind: Rule
      services:
        - name: sentry-nginx
          namespace: sentry
          port: 80
Copy the code

Execute kubectl apply -f route.yml to validate the route, then request [https://sentry.ik47.com](https://sentry.ik47.com) to view it.

Matters needing attention

The deployment itself is not difficult, the trouble is that there are all kinds of strange scenarios that need to be solved one by one, as long as you have the patience to look at the log, and Google search log keywords, can be solved.

  • Common command

    # Check pods status
kubectl get pods -n sentry

# check log
kubectl logs pods/xxx -n sentry

# delete container
kubectl delete pods xxx -n sentry

# to check the PVC
kubectl describe pvc xxx -n sentry

# Enter the container
kubectl exec -it -n sentry xxxx /bin/sh

# Edit sentry configuration
kubectl edit cm  sentry-sentry -n sentry

Start BusyBox and test the network
kubectl run --rm busy --image=busybox --namespace=sentry  -it /bin/sh

Copy the code

  • Uninstall the sentry

    # uninstall sentry
helm uninstall sentry -n sentry
kubectl delete deployments -n sentry --all
kubectl delete pods -n sentry --all
kubectl delete job -n sentry --all
# remove PVC
kubectl delete pvc --all -n sentry
Copy the code

  • If the database initialization fails, initialize and create the user yourself by using the following command

    kubectl exec -it -n sentry $(kubectl get pods  -n sentry  |grep sentry-web |awk '{print $1}') bash
sentry upgrade
If there is no administrator, create one
sentry createuser
Copy the code

Results demonstrate

The number of nodes started is as follows

Start Pod as follows

conclusion

I have tried the 9.1.2 version and the three-party cluster version in my own server. The deployment operation cost of both is very low, but the resource occupation is really much different. If it is used by companies, it is recommended to deploy the latest version. The most important thing is to bring in the operation and maintenance leaders. Professionals have more energy to do things more professionally.

In the deployment of the latest version of Sentry, there were a lot of problems with the software version related to operation and maintenance. It was too reckless to just do it without reading the manual. Later, I spent a week in the introduction of K8S, which was deployed in a VIRTUAL machine using Vagrant. Feel oneself direction to run continuously partial, fortunately finally take two online server directly to do, spent a day to do.

Finally, I strongly recommend Teacher Yang Ming’s Docker course “Advanced from Docker to K8s”, which is very practical. After a week of serious study, I gave all my servers to K8s, which is not too cool. The key is that this class is really cheap, compared with other courses, Just like no money 😂.

Reference documentation

  1. Architecture | Sentry Developer Documentation
  2. Github.com/getsentry/s…
  3. Github.com/getsentry/s…
  4. Github.com/sentry-kube…
  5. Sentry 9.1.2 Cluster Deployment – Zhihu (Zhihu.com)
  6. Sentry high availability deployment morrowind | dress dance (jiankunking.com)
  7. Rounding docker – compose installation sentry cluster solution < 2 > | ops essay < / 2 > (gitee. IO)
  8. Advantages knowledge – Course Introduction page (youdianzhishi.com)
  9. Advanced from Docker to Kubernetes
  10. Calabash’s Blog
  11. Sentry Real-time application error tracking system in Kubernetes private deployment – SegmentFault thinking no
  12. Helm install Sentry – Nuggets (juejin. Cn)
  13. Who exactly is Sentry? – the nuggets (juejin. Cn)
  14. Helm Install Sentry (ICode9.com)