Introduction to Microservices

Microservice Architecture is an architectural concept that aims to decouple solutions by decomposing functionality into discrete services.

Going back to the concept of microservices, it is not a specific technology, but rather a collection of architectural styles, so microservices themselves are not clearly defined, but we know that it is an overall architecture consisting of more than one independent service.

As a consultant at ThoughtWorks, I would love to refer you to Martin Fowler’s article on Microservices:

Martinfowler.com/articles/mi…

The initial monomer application

We take the taxi service similar to Uber as a business case. Due to the small team size and small business volume in the initial stage, the whole system is a single application as follows:

As shown in the above, the passenger and the driver through the REST API to interact, all service requests is a database, and all the services, such as individual center, orders, payment and so on all exists in a framework of service, in the early days of the time, so the development of architecture, is very common for an entrepreneurial product, centralized management, development of high efficiency, But with the continuous expansion of business and the increase of magnitude, slowly this single application becomes a boulder application, then we carry out code maintenance on it, it is easy to encounter the following problems:

  • When we make code updates to one of the services, the entire system is re-executed for testing, integration and deployment, and the process is very slow.
  • When a service fails, the entire server becomes unavailable, and fixing Bug locations is difficult in a system repository.
  • Extending services and introducing new features can be difficult, and may involve refactoring the entire system.
  • .

A revamped microservices architecture

In order to solve the current business pain points, the taxi-hailing company referred to the application architecture of Amazon, Netflix and other giants, and finally redesigned its Boulder app according to the architecture of microservices:

In this service map, we see that each core business module is split off as a separate service, and the concept of an API gateway is introduced for users to navigate to internal services. Now, this microservice architecture solves some of the defects that used to exist in single application:

  • All function modules are independent, removing the code interdependence between services, enhancing the extensibility of the application
  • Each module can be deployed independently, which shortens the deployment time of applications and helps locate errors more quickly

Of course, microservices also bring more problems and challenges, which we won’t discuss here. As you can see from this example, moving from a monolithic application to a microservice architecture is a process of service evolution. No architecture can occur in a vacuum, and the best architecture depends on how well it fits the current business model.

Microservice design guidelines

Microservices themselves bring many advantages over traditional architectures, but at the same time add additional complexity and administrative costs, so I’ve always been a believer in the following: Don’t microservices for microservices’ sake. Therefore, at the beginning of the architecture, I prefer to organize the code as a single application, with clear unpacking logic to isolate the business and reduce the complexity between modules, and then later in the project, if the business and specific architecture fit the microservices design concept, then we will split the modules.

There is an article for micro service design summary is in place, here is not repeated, directly recommended to you:

medium.com/@WSO2/guide…

Use Nest to develop microservices

NestJS is a Node.js Server framework written by TypeScript. The underlying HTTP Server is supported by Express. Different from Koa and Express, it pays more attention to architectural design, allowing loose JS server projects to come out of the box with various design patterns and specifications, and drawing on various design patterns from Angular and Spring Boot frameworks. Such as DI, AOP, Filter, Intercept, Observable, etc.

Nest is a progressive framework with built-in support for microservices that you can use to try to build complex Web applications. In the following steps, I’ll explore how to build a Nest microservice application from scratch.

Service to service communication protocol

Nest has several different built-in microservices Transport layer implementations defined in the Transport module of the @NestJS/MicroServices package, which we simply categorize:

  • Direct transmission: TCP
  • Message transfer: REDIS, NATS, MQTT, RMQ, KAFKA
  • Remote process scheduling: GRPC

We must choose a communication protocol as the communication mechanism between micro-services. It is very fast and convenient for Nest framework to switch the transmission protocol, so we need to decide according to the characteristics of our own project. In the following articles, I will first choose TCP directly as the transport mode, then change it to Redis, and finally talk about how to use gRPC for scheduling, as well as practice with services that connect to other languages.

Inter-service communication mode

In Nest MicroService, there are two communication modes:

  • Request-response mode is used when we need to exchange messages between internal services. Asynchronous response functions are also supported. Our return result can even be an Observable.
  • Event-based mode. When services are based on events — we only want to publish events rather than subscribe to events, there is no need to wait for the response of the response function. In this case, the event-based mode is the best choice.

In order to transmit data and events accurately between microservices, we need a value called pattern, which is a common object value, or a string, customized by us. Pattern is the language of communication between microservices. When communicating, It is automatically serialized and a matching service module is found through a network request.

Build a simple WordCount service

Now, I’ll take you through a simple example of implementing a microserver. Suppose you want to add a data processing module called Math to the entire system, and one of the services is WordCount. Let’s look at how to build in Nest.

We first implemented the function on the single architecture:

The following is an example of a curl request from MS-app:

curl --location --request POST 'http://localhost:3000/math/wordcount' \
--header 'Content-Type: application/json' \
--data-raw '{
    "text": "a b c c"
}'
Copy the code

With an idea of what needs to be done, let’s build the project from scratch and initialize it by executing the following script:

npm i -g @nestjs/cli
nest new ms-app
cd ms-app && nest g service math
Copy the code

Nest automatically generates the Math module for you and references MathService as a Provider in app.module.ts. The core function of our service is WordCount. Before writing the function, let’s add a test case for service that defines the expected request parameters and return format:

// math.service.spec.ts 
it('should be return correct number', () => {
  expect(service.calculateWordCount('a b c c')).toEqual({ a: 1.b: 1.c: 2 });
  expect(service.calculateWordCount('c c c d')).toEqual({ c: 3.d: 1 });
});
Copy the code

The benefits of TEST-driven development are not to be enumerated here, but we are now going to write a simple WordCount method in math.service.ts. All we need to do is split each word in the text with a space and then do the word:

import { Injectable } from '@nestjs/common';

@Injectable()
export class MathService {
  calculateWordCount(str: string) {
    const words = str.trim().split(/\s+/);
    return words.reduce((a, c) = > ((a[c] = (a[c] || 0) + 1), a), {}); }}Copy the code

We declare mathService in the constructor, Nest will initialize the instance through dependency injection, and then write the route. The code is as follows:

import { Controller, Post, Body } from '@nestjs/common';
import { MathService } from './math/math.service';

@Controller()
export class AppController {
  constructor(private readonly mathService: MathService) {}

  @Post('/math/wordcount')
  wordCount(@Body() { text }: { text: string }): { [key: string]: number } {
    return this.mathService.calculateWordCount(text); }}Copy the code

Run the curl command to perform a terminal test. The following result is displayed:

{"a":1."b":1."c":2}
Copy the code

Refactoring code using microservices

For a variety of reasons, we may be facing microservice unassembly, where WordCount is used to interact as a microservice:

  • Modules are maintained by other teams
  • Modules are developed in other languages
  • For better architecture, etc

The default communication protocol of Nest microservice is TCP, and the architecture diagram is as follows:

We create a new service with Nest New MS-Math by first installing the built-in microservice module:

yarn add @nestjs/microservices
Copy the code

SRC /main.ts is now a microservice instead of a normal instance:

import { NestFactory } from '@nestjs/core';
import { Transport, MicroserviceOptions } from '@nestjs/microservices';
import { AppModule } from './app.module';

async function bootstrap() {
  const app = await NestFactory.createMicroservice<MicroserviceOptions>(
    AppModule,
    {
      transport: Transport.TCP,
    },
  );
  app.listen((a)= > console.log('Microservice is listening'));
}
bootstrap();
Copy the code

At this point, we can copy calculateWordCount from math.service to app.service.ts and then modify app.controller.ts. In the controller, we no longer use @get or @POST to expose the interface. Instead, we use @Messagepattern to set the pattern for identification between microservices. Take a look at our elegant code:

import { Controller } from '@nestjs/common';
import { AppService } from './app.service';
import { MessagePattern } from '@nestjs/microservices';

@Controller()
export class AppController {
  constructor(private readonly appService: AppService) {}

  @MessagePattern('math:wordcount')
  wordCount(text: string): { [key: string]: number } {
    return this.appService.calculateWordCount(text); }}Copy the code

Now that the Microservice has been created, let’s start it. If you are careful enough, you should receive the command line: Microservice is listening.

At this point, we also need to modify the original MS-App service. We will delete the math directory directly, because we no longer need to call mathService here, and delete all relevant code in app.module.ts and app.controller.ts. Then install microservice dependencies:

yarn add @nestjs/microservices
Copy the code

The first step is to register a client for transferring data to the microservice. Here we register the mathService using the register() method provided by ClientsModule:

import { ClientsModule, Transport } from '@nestjs/microservices';

@Module({
  imports: [
    ClientsModule.register([
      { name: 'MATH_SERVICE'.transport: Transport.TCP },
    ]),
  ]
  ...
})
Copy the code

Once the module is registered, we can reference it in app.controller using dependency injection:

constructor(@Inject('MATH_SERVICE') private client: ClientProxy) {}
Copy the code

ClientProxy objects have two core methods:

  • send()This method calls the microservice and returns the response body of an Observable object. Therefore, it is easy to subscribe to the data returned by the microservice. Note that the corresponding message body will be sent only after you subscribe to the object.
  • emit()Event-based message delivery method, the message will be sent immediately whether you subscribe to data or not.

With some basic concepts in mind, let’s make some changes to the routes in app.Controller:

	@Post('/math/wordcount')
  wordCount(
    @Body() { text }: { text: string },
  ): Observable<{ [key: string]: number }> {
    return this.client.send('math:wordcount', text);
  }
Copy the code

Then start the MS-app service and run the same curl command on the terminal. It is expected to get the same result.

Try using event-based transport

Although the above project has met our microservice transformation requirements, in order to learn how to use it, we can add another data triggered by the event response model here, and the event name is: Math :wordcount_log, first add a line of event trigger code to the original /math/wordcount routing method in MS-app:

this.client.emit('math:wordcount_log', text)
Copy the code

Then open the MS-Math service and register the corresponding subscriber in app.Controller:

	@EventPattern('math:wordcount_log')
  wordCountLog(text: string): void {
    console.log(text);
  }
Copy the code

That’s all you need to do. Now run the curl command and you can see the following print on the mS-Math service terminal:

receive: a b c c
Copy the code

Use Redis as the message broker

In the previous section, we built a simple microservice architecture, which uses TCP for direct communication between microservices. In this section, we are going to change the message transmission mechanism to use Redis as message broker for forwarding, so as to make our microservice more robust.

What is a message broker

Message Broker is an intermediate program module used to exchange messages in computer networks. It is the building block of message-oriented middleware and therefore does not include responsibility for remote process scheduling (RPC).

Message broker is also an architectural pattern for message validation, transformation, and routing. Tune application communication to minimize mutual awareness (dependency) and effectively decouple. For example, a message broker can manage a workload queue or message queue for multiple receivers, providing reliable storage, guaranteed message distribution, and transaction management.

Why Redis

The above part explained Nest framework for message protocol implementation support, currently support the following: REDIS, NATS, MQTT, RMQ, KAFKA, switching in these message services itself is very convenient, and I choose REDIS mainly for the following reasons:

  • Redis itself is lightweight and efficient enough that usage is high and popular
  • In my own engineering code, I already have Redis service in my service list. I don’t want to choose other services as message brokers alone, which increases service dependency and management costs.
  • I have a certain understanding of Redis itself, while the others have not much experience in using it.

Changes to the service architecture

If you have any questions about why you use a message broker, let me draw a diagram for you:

Code implementation

We will first create a docker-comemess. yml file in the project folder to manage the Redis service:

Version: '3.7' services: redis: image: redis:latest container_name: service-redis command: redis-server --requirepass rootroot ports: - "16379:6379" volumes: - ./data:/dataCopy the code

Docker-compose up -d docker ps

CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS                                NAMES
6faba303e0ae        redis:latest        "Docker - entrypoint. S..."   9 minutes ago       Up 9 minutes        0.0. 0. 0:16379->6379/tcp              service-redis
Copy the code

First we install Redis dependencies in MS-app and MS-math:

yarn add redis
Copy the code

First of all, in the bootstrap function in MS-Math, we replace Transport with REDIS and attach the service address:

// before
const app = await NestFactory.createMicroservice<MicroserviceOptions>(
    AppModule,
    {
      transport: Transport.TCP,
    },
  );

// after
const app = await NestFactory.createMicroservice<MicroserviceOptions>(
    AppModule,
    {
      transport: Transport.REDIS,
      options: {
        url: "redis://:rootroot@localhost:16379",}});Copy the code

That’s all mS-Math needs to do, and then open mS-app, where we register the client, we’ll also replace it accordingly:

// before
ClientsModule.register([
      { name: 'MATH_SERVICE'.transport: Transport.TCP },
    ]),

// after
ClientsModule.register([
      {
        name: 'MATH_SERVICE'.transport: Transport.REDIS,
        options: {
          url: 'redis://:rootroot@localhost:16379',},},]),],Copy the code

Using curl, we can still get the correct output.