1. Write it first

“This article has participated in the call for good writing activities, click to view: the back end, the big front end double track submission, 20,000 yuan prize pool waiting for you to challenge!”

Last week, WE did an internal sharing of the company, and found that everyone joked a lot about the existing deployment method, “Wherever there is no peace, we should strive for peace”. Therefore, as a hard-working worker, I intend to try to use the open source platform to release, but THE learning curve of K8S is really a bit high, and the worker is really difficult.

During my time at Google, Nomad stands out and supports binary and multi-platform deployments, which is you.

2. Introduction

A simple and flexible workload orchestration tool for large-scale deployment and management of containerized and non-containerized applications across local and cloud. Advantage:

  • Simple and lightweight: a single binary integrated into the existing infrastructure. Easy to operate locally or in the cloud with minimal overhead.
  • Flexible workload support: Orchestrate any type of application, not just containers. Docker, Windows, Java, VM, etc.
  • Modernizing legacy applications without rewriting: Bringing choreography advantages to existing services. Zero downtime deployment, greater flexibility, higher resource utilization, and more can be achieved without containerization.
  • Large scale easy federation: a single command for multi-region, multi-cloud federation. Use Nomad as a single unified control plane to globally deploy applications to any region.
  • Easy deployment and scaling: Deploy to bare metal just as easily as in the cloud. Scale globally without complexity. See the 2 Million Container Challenge.
  • Native integration with Terraform, Consul, and Vault: Nomad seamlessly integrates with Terraform, Consul, and Vault to provide service networking and key management.

Note: The above is from Nomad’s website. The flexibility to deploy multiple types of applications and platforms is Nomad’s absolute strength.

3. Internal introduction

3.1 Basic Concepts

Workload choreography essentially addresses the question of what platform operations users need and what types of services are deployed to what regions.

3.1.1 platform

The Nomad platform consists of two parts.

  • Client: deployed as an Agent on a machine that is waiting for service provisioning to run. It is used to collect statistics about used and to be used resources on the machine and must be registered with the Nomad Server

  • Server: deployed as a cluster, manage all running service information, synchronize information between cluster nodes using raft consistency algorithm to ensure availability, with a millisecond network delay

Note: Parameter limits for server deployment machines

3.1.2 service

Nomad’s abstraction of services is very interesting. I think docker has preconceived and occupied most of the market, so many concepts in the abstraction will refer to the existing concepts in Docker.

Note: A service here can refer to a single service or a group of interrelated services.

  • Job: A logical unit of a set of services to be run, specifying Regions, DataCenters attributes, etc., to be deployed.
  • Task Group: A Group of related services that are deployed on a Client machine.
  • Task: a real service to run. You can specify Docker, QEMU, Java, and Static binaries for deployment.
    • Note: The deployment type is specified by the Driver configuration.

3.1.3 region

Nomad supports cross-local and cloud platforms, so the concept of Regions and Datacenters is used in the regional abstraction.

  • Regions: A Region contains multiple data centers. A group of Nomad servers is combined to represent a Region.
  • Datacenters: Nomad clients do not need to be deployed in the same data center as the service, but they need to be deployed in the same Region. Datacenters can be thought of as a way to divide clusters.

3.1.4 Other abstract concepts

  • Allocation: Allocation can be interpreted as a mapping that allocates Task groups in a Job to nodes of clients. A single Job may have hundreds or thousands of Task groups, which means that there must be an equal number of assignments to map jobs to clients. Allocation is created by the Nomad server as part of the decision scheduling during evaluation.
  • Evaluation: Evaluation is the mechanism for Nomad to make scheduling decisions. When the expected state (the job) or the actual state (the client) changes, Nomad creates a new Evaluation to determine if an action needs to be taken. Evaluation may cause an Allocation to occur if needed.
  • Bin Packing: Bin Packing is the process of filling a Bin with items to maximize the utilization of the Bin. This extends to Nomad, where Client is “Bin” and Task Group can be understood as an item. Nomad sends a Job to a Client using an effective boxing algorithm.

3.2 Scheduling Policies

Scheduling is the core function of Nomad. It refers to the process of assigning a Job to a Client. Its design and implementation was encouraged by Google’s Omega and Nomad.

  • Omega: flexible, scalable schedulers for large compute clusters
  • Large-scale cluster management at Google with Borg.

See the following for the core scheduling diagram (ps if you understand the concepts above, you should be better at understanding.

3.2.1 Preemptive scheduling

When creating a Job, you can specify Priority to set the Priority of the Job. Preemption Allows Nomad to kill existing jobs to provide resources for higher priority jobs. Preemption allows high-priority tasks to run even when the entire cluster is competing for resources.

See: Preeption

3.2.2 availability

Nomad uses two consistency protocols.

  • Raft Protocol: Used to synchronize information between nodes in single-region cluster mode.
  • Gossip Protocol: Allows servers to execute cross-region Job deployment requests and handle the loss of the entire Region in multi-region cluster mode

Detailed see:

  • Gossip Protocol
  • Consensus Protocol

Note: it is impossible to draw a conclusion purely from the introduction of the official website. After all, generally people who provide services will not say that their services are not good

Practice of 4.

To borrow a phrase from the founder of Linux, “Talk is cheap show me the code”. Deployed using Consul based automated service discovery, it has to be said that Hashicorp’s home ecosystem integration is really delicious.

4.1 Points of advantage

  • Clear version change process, support fast rollback

  • Visual deployment logs for convenience and simplicity

  • Client – and Job-level resource water level monitoring

  • Quick container login

  • Topology display of total resource usage

4.2 disadvantages

Because it is not used much and Chinese is mostly a scaffolding, real business deployment is difficult

  • How to connect private warehouse?
  • How do I integrate with Consul?
  • How to deploy other Agent nodes?
  • How to mount volume?
  • How to use host network like docker?
  • …

Stop it! These are all my tears!

4)

I’ve decided that my next post will be about all the jokes about deploying it. This week can still end with flowers and fun.

  • I’m willing to stay positive until life completely knocks me down. Politics textbook says: ‘The development of things is a spiral, wave forward. ‘I’m going to turn around and jump around and make myself happy.
  • I have no talent, but I want to see what it’s like to be a normal person with a dream.
  • To be uncomfortable not huan is cloud smoke to like willing to try again. (PS: Still greedy, haha

5. Reference materials

  • Connect Nodes into a Cluster

  • Nomad Reference Architecture

  • What is the relation between docker0 and eth0

  • The Two Million Container Challenge

  • Nomad is used to implement cluster management and micro-service deployment scheduling

  • Nomad tutorial

  • Container optimization

  • P2P network core technology: Gossip protocol

  • Nomad Ports Used

  • Connect Nodes into a Cluster

  • Nomad Configuration

  • Docker and Podman: One rising, the other rising

  • Brief introduction to QEMU architecture

  • task

  • Docker Driver

  • Google ‘s Borg scheduler

  • Docker Authentication

  • Job Specification

  • Nomad: Increases the amount of local development

  • Plugin

  • Client Requirements

  • Host networking no longer working with non-deprecated syntax for network stanza