Introduction to the

  • Protocol Buffers (a.k.a., protobuf)Google is a language-independent, platform-independent format for data interchange. Implementations are provided for three languages: Java, c++, and python, each of which contains a compiler and library files for the corresponding language. Because it is a binary format, it is much faster than using XML for data exchange. It can be used for data communication between distributed applications or data exchange in heterogeneous environments. As a binary data transmission format with excellent efficiency and compatibility, it can be used in many fields such as network transmission, configuration files, data storage and so on.

Compared with the traditional Json and XML data storage formats, PB is a more lightweight and efficient structured data storage format.

Basic usage

Data structure: message_name{message_body; } Message_body format: for example, required int32 query =1 [defaut=10]; Rule type name = value[other_rule]; Rule: Required indicates that the value field must exist; Optional indicates an optional range of values; Repeated means repeatable range of values (i.e. >=0); Repeated samples=4[Packed = True]; Repeated samples=4[Packed = True]; Form; Value value: The minimum value is 1. For low-level encoding, the value from 1 to 15 occupies one digit, and the value greater than 15 occupies several digits. Values in different messages do not interfere with each other, usually starting with 1. Basic types of data types:  .proto Type C++ Type Java Type double double double float float float int32 int32 int int64 int64 long uint32 uint32 int uint64 uint64 long sint32 int32 int sint64 int64 long fixed32 uint32 int fixed64 uint64 long sfixed32 int32 int Sfixed64 INT64 Long bool bool Boolean String String String Bytes String ByteString Complex types of data types. Complex types include: Enumeration, other messages, groups, etc. Enumeration definitions For example: enum Corpus{WEB=0; LOCAL=1} Enumeration is defined in message. Members can be defined using other messages as types. My understanding of groups is somewhat like the union structure in C++. Nested definitions: The message structure can be nested, and the nested message definition needs to be of the form outmessage.inMessage when used as a member type by other messages. Package structure: defined form: package foo.bar; In C++, two namespaces foo and bar are generated, and bar is defined in foo. This can be done by importing "myproject/other_protos.proto"; To import.proto files; References to messages from other packages require the full package path; Services: used in RPC systems. Proto defines interfaces. The definition format is as follows: service SearchService {RPC Search(SearchRequest) return (SearchResponse); }.proto file compilation: Format: protoc - - proto_path=(.proto file path) - - cpp_out=(.cc.java generation file path) (.proto file path)/? . Proto -- proto_path is simplified to: --I Where cpp_OUT can be changed as required: the option is javA_OUT/PYTHon_OUT. Example: protoc -i =./ --cpp_out=./ model.protoCopy the code

Simple example

Test environment:

Ubuntu 18.04

GCC 7.2.0

protoc 3.15.8

  1. write.protofile
syntax = "proto3";
package test;

message person

{
    int32   id = 1; // ID
    string  name = 2;   // str
    repeated int32  email = 3;  //optional field
}
Copy the code

There are two large versions of Protobuf, Proto2 and Proto3, compared to Python versions 2.x and 3.x, and it is also recommended to get the Proto3 version directly if you are new to it. So the following descriptions are based on Proto3. Proto3 simply supports more languages than proto2 (Ruby, C#, etc.), removes some complex syntax and features, introduces more conventions, etc.

syntax = "proto3"; / / PB protocol version import ". Google/protobuf/any proto "; // Refer to an external message, either local or, more specifically, Any package jet.protobuf; Protobuf. PersonTest (); // The package name can be used by other proTo when referencing this proto. Java_package = "com.jet.protobuf"; java_package = "com.jet.protobuf"; // Generate the package name of the class. Note: the folder option JAVA_outer_className ="PersonTestProtos" is generated according to the package name definition in the specified path. Message PersonTest {int32 id = 1; message PersonTest {int32 id = 1; // int string name = 2; // String Type string email = 3; Sex sex = 4; // Enumeration type repeated PhoneNumber phone = 5; // Refer to message Map <string, string> tags = 6 of type PhoneNumber defined below; // Map type repeated Google.protobuf. Any details = 7; Enum Sex {DEFAULT = 0; MALE = 1; Female = 2; PhoneNumber {string number = 1; PhoneType type = 2; enum PhoneType { MOBILE = 0; HOME = 1; WORK = 2; }}}Copy the code
  1. Generate corresponding classes
protoc test.proto --cpp_out ./
Copy the code

Two files test.pb.cc test.pb.h are generated in the current directory

  1. Write c++ code for testing
#include <stdio.h> #include <string.h> #include "test.pb.h" using namespace std; int main() { test::person a; a.set_id(8); a.set_name("Kios"); string tmp; bool ret = a.SerializeToString(&tmp); if(ret) { cout << "encode success!" << endl; } else { cout << "encode failed!" << endl; } test::person b; ret = b.ParseFromString(tmp); if(ret) { cout << "decode success! \n id = " << b.id() << "\n name = " << b.name().c_str() << endl; } else { cout << "decode faild! \n" << endl; } return 0; }Copy the code
  1. Specify the link library and compile
g++ test.pb.cc main.cpp -o main.o -L/usr/local/protobuf/lib/ -I/usr/local/protobuf/include/ -I./ -lprotobuf --std=c++11 // it is worth remembering to specify the c++ versionCopy the code

5. View the running result

(base) seemmo@ubuntu:~/projects/proto_try$ ./main.o
> encode success!
> decode success!
> id = 8
> name = Kios
Copy the code

Some environmental issues that need attention

  • command not found

Environment variables are not set properly and will <… /protobuf/bin> Added to PATH

  • Run The Times after the compile passeserror while loading shared libraries: libprotoc.so.9: cannot open shared object file:No such...
Problems setting up the dynamic link library Step 1: create the/etc/ld. So. Conf. D/libprotobuf. Conf input/usr/local/protobuf/lib/execution sudo ldconfig step 2: Vim /etc/profile --> Export LD_LIBRARY_PATH=/usr/local/protobuf/lib:$LD_LIBRARY_PATH --> # source /etc/profileCopy the code

$LD_LIBRARY_PATH = /usr/local/protobuf/lib:$LD_LIBRARY_PATH specifies the LD_LIBRARY_PATH variable assigned by export. However, in the child process, neither export nor = can change the value of the parent process.

Reference link: blog.csdn.net/happyday_d/…