In this article, we show you how to build microservices using gRPC, an open source framework for building scalable, high-performance microservices and creating communication between services.

background

As enterprises increasingly turn to microservices, the need for low-latency and extensible frameworks to build these microservices is increasing. To meet this need, tool and framework providers are stepping up to meet microservices requirements. While learning from their experience building large microservice applications, technology professionals share their knowledge of reusable components so that others can build architectures with the same scale and performance.

What is the gRPC

GRPC is an open source framework (created by Google) that is a generic RPC framework for building high-performance web applications on a large scale. Implementations are available in multiple languages and support cross-platform communication.

Usage scenarios

GRPC is ideal for RPC communication between services. Here, we’ll use Java to implement microservices and related frameworks to make it more functional. To make it accessible to other services, we will create a wrapper REST service that will use the gRPC client to communicate with the gRPC service.

The preparatory work

We need to set up a base environment to build and run the sample. The basic requirements are Java and Maven installed. Other dependencies, such as gRPC tools and server runtime libraries, will be downloaded automatically during the build process. See the code below for the core dependencies needed to build the application.

<dependency> <groupId>io.github.lognet</groupId> <artifactId>grpc-spring-boot-starter</artifactId> < version > 4.5.0 < / version > < / dependency > < the dependency > < groupId > org. Springframework. Boot < / groupId > <artifactId>spring-boot-starter-log4j2</artifactId> </dependency> <dependency> <groupId>org.mapstruct</groupId> <artifactId>mapstruct</artifactId> <version>${mapstruct.version}</version> </dependency>Copy the code

The core function

GRPC is basically a platform – and code-independent protocol. This means that you can use any type of encoding, such as binary, JSON, XML, etc., but the recommended approach is to use “Protobuf,” which supports binary encoding with a specialized serialization/deserialization mechanism. The pluggable design allows users to extend it to support the desired platforms and stacks.

The core construction of protobuf is proto IDL (Interface Definition Language), which defines message types and service definitions. It also provides tools to generate model classes and service interfaces for the required platforms.

Message type

We can start with the proto definition that defines the message type in the.proto file. Look at the following example.

message AccountProto { int32 accId = 1; string accName = 2; string accBalance = 3; bool status = 4; . }Copy the code

For a complete reference to data types and keywords, see the Proto3 documentation.

Protobuf provides a tool to generate code for model classes based on message definitions applicable to your platform/programming language. The following command will generate the Account class in Java based on the given message definition.

$ > protoc -I=proto-demo --java_out=proto-demo account.proto
Copy the code

The service definition

A gRPC service definition is a set of operations that need to be performed on defined message types. These operations can take one of the following four forms of communication:

  1. Single channel RPC – this is the simplest form of communication. It is synchronous in nature, allowing the user to send a request in blocking mode and wait for a response until the server completes processing.

  2. Streaming RPC – In this form, the client sends data once, but the server returns a response as a stream.

  3. Client-side streaming RPC – Unlike server streaming, in this form, the client sends the requested data as a stream and the server returns the data as a whole.

  4. Two-way streaming RPC – In this form, both the server and client support streaming data based on request and response.

    A sample service definition of a message type with standard CRUD operations would take the following input:

service AccountService {
    rpc GetAccount (google.protobuf.Int32Value) returns (AccountProto);
    rpc GetAllAccounts (google.protobuf.Empty) returns (stream AccountProto);
    rpc CreateAccount (AccountProto) returns (AccountProto);
    rpc UpdateAccount (AccountProto) returns (AccountProto);
    rpc DeleteAccount (google.protobuf.Int32Value) returns (google.protobuf.Empty);
}
Copy the code

The GRPC-Java implementation provides extended tools to help generate service interfaces that users need to implement based on domain logic and server stubs that clients will use to invoke deployed services.

$ > protoc -I=grpc-demo\src\main\proto --java_out=grpc-demo\src\main\proto account.proto
Copy the code

Standard server and client

The GRPC-Java library provides a responsive server implementation (based on Netty) to deploy your service and a blocking/non-blocking client implementation to connect your service to other services.

You need to register your service implementation and start the server programmatically.

server = ServerBuilder.forPort(port), .addService(new GreeterImpl()).build().start();
Copy the code

Find the GitHub reference here.

To connect to a service deployed on a Netty-based gRPC server, you need to create a message channel and connect it to the generated server stub for invocation.

ManagedChannel channel = ManagedChannelBuilder.forTarget(target).usePlaintext().build(); 
blockingStub = GreeterGrpc.newBlockingStub(channel); 
HelloRequest request = HelloRequest.newBuilder().setName(name).build();
HelloReply response = blockingStub.sayHello(request); 
Copy the code

Find the GitHub reference here.

Network client

There is also a gRPC Web module that allows Web clients to access your gRPC services seamlessly. Their earlier versions supported connecting to Web clients through a reverse proxy, but can now do so without an intermediate proxy.

Another approach is to expose the wrapper service layer to the externally facing world using the REST/GraphQL protocol and connect via a gRPC client.

Data mapping and persistence

We may need to add another layer on top of this to create fully functional domain services using data access libraries such as Hibernate. As with Hibernate or any database, the required entities need to be modified in a way that may not be feasible for the protobuf model generated. Therefore, we may need some mapping logic to transform model classes into entity classes. One such good library is MapStruct, which automatically maps based on bean conventions. We just need to provide the mapping interface:

@Mapper
public interface AccountProtoMapper {
        Account map(AccountProto accountProto);
        AccountProto map(Account account);
}
Copy the code

Third Party support

To further simplify the overall build and run environment, there are some popular third-party Maven plug-ins and libraries that also help.

1. Run the Proto Tool integrated with Build

Protocol buffer Maven plug-in (provided by Xolstice) runs the Proto tool and its extensions, as well as builds and generates source code from *. Proto files. It also appends.proto* files as resources for the project. See sample configuration to generate Java code.

<configuration>
    <protocArtifact>com.google.protobuf:protoc:${protoc.version}:exe:${os.detected.classifier}</protocArtifact>
    <pluginId>grpc-java</pluginId>
    <pluginArtifact>io.grpc:protoc-gen-grpc-java:${grpc.version}:exe:${os.detected.classifier}</pluginArtifact>
</configuration>
Copy the code

The reference can be found here.

2. Generate the mapper class

The MapStruct library supports the generation of message-type mapper classes that move messages from/to the entity of the class. It provides an annotation handler that generates a mapper class by analyzing the specified annotation at build time. Refer to the configuration below to compile the Maven plug-in.

<configuration> <annotationProcessorPaths> <path> <groupId>org.mapstruct</groupId> <artifactId>mapstruct-processor</artifactId> <version>${mapstruct.version}</version> </path> </annotationProcessorPaths>  </configuration>Copy the code

3. Automatically start the gRPC server and register the service implementation

By default, the gRPC server is not started with the Web server (in this case, Netty). In addition, it needs to register all gRPC services before the server starts. LogNet’s gRPC Spring Boot automatically scans all classes annotated with @grpcService and registers the service definition with the server builder. Once the server is built, it automatically starts the gRPC server on the port configured in the Spring application properties.

In addition to registering services and starting servers, it supports automatically configured security, health checks, and service discovery. See here for more information.

A full sample implementation of the above approach is available on GitHub.

differentiation

  • Google built gRPC out of a learning need to build large-scale microservices from scratch. Most popular microservice frameworks still lack performance and cross-platform support requirements because no large system can be built on a single stack and a single encoding.

  • Most popular is support for HTTP/2, which remains a roadmap for many vendors. It provides advantages for binary line transport using multiplexed non-blocking data streams over a single TCP connection. Header compression is further supported, which provides additional performance benefits.

  • In addition to supporting Protobuf as the primary encoding mechanism, it also adds support for JSON-based services that can be easily used by low-end clients. While Protobuf is the recommended way to implement gRPC, Google has added support for FlatBuffers and increased adoption both internally and across the industry.

  • The initial release supports exposing gRPC services through reverse proxies such as Envoy and Ngnix based systems. Recent developments in gRPC Web bridge this gap and add support for exposing gRCP services to Web clients through the adoption of HTTP/2 across JavaScript libraries. Further development is underway to add support for popular Web frameworks such as Angular and React.

  • With a full-featured development stack, it also provides unit testing assistants such as InProcessServer and InProcessChannelBuilder.

using

  • Google uses it in internal tools and platforms such as Bigtable, Pub/Sub, Speech, and TensorFlow.
  • CoreOS was an early adopter of gRPC and added support to its service discovery tool and container engine.
  • Microsoft is an advanced adopter of gRPC and has added support for its Web APIS and build tools to make it easier for developers to build services seamlessly.
  • Now, more and more projects are using it across the industry, such as open source developers like Docker, Square, Netflix, and Cisco.

Alternative methods

In addition to building gRPC, Google’s application provides the tools/libraries mentioned in this article and the Spring Boot startup framework. Several specialized microservice frameworks provide out-of-the-box support for gRPC service implementations.

  1. Quarkus gRPC
  2. Asp.NET gRPC
  3. Akka gRPC
  4. Salesforce gRPC