Defining message types

First, create a *.proto file

// Declare using proto3 syntax; If this is not specified, the compiler will use Proto2. The specified syntax must be the first line of the non-empty, non-commented file
syntax = "proto3";

// We define a SearchRequest message format with three fields
message SearchRequest{
  // The query field is a string
  string query = 1;
  // The page_number field of the 32-bit integer (int32) type
  int32  page_number = 2;
  // The result t_per_page field is a 32-bit integer (int32)
  int32 result_per_page = 3;
}
Copy the code

Specifying the field type

In the example above, all the fields are scalar types.

  • Two integer variables:page_numberandresult_per_page
  • A string variable:query

Of course, you can also specify other types for fields, including enumerations or other message types

Assignment identification number

Each field is assigned a unique number when the message is defined. These field numbers are used to identify fields in the message binary format and should not be changed if the tone message type is used.

Please note:1-15The field number in the range takes up one byte for encoding, including the field number and field type

16-2047.Field numbers in the range take up two bytes

Therefore, more use is recommended1-15

You can specify a minimum field number of 1 and a maximum field number of 2^ 29-1 or 536,870,911

You can’t use19000-19999.(FieldDescriptor::kFirstReservedNumber 到FieldDescriptor::kLastReservedNumber),protobufThe protocol implementation makes reservations for these, which will alert you at compile time if necessary, and also cannot use any previous reservations (reserved).

Specifying field rules

The specified message field modifiers must be one of the following:

  • sigular: A well-formed message should have zero or one of these fields (but no more than one). This is aproto3Default field rules for syntax.
  • repeated: This field can be repeated any number of times (including zero) in a well-formed message. The order of duplicate values is preserved.

In Proto3, the repeated scalar field is packed by default.

Add more message types

Multiple message types can be defined in a.proto file

syntax = "proto3";

message SearchRequest{
  string query = 1;
  int32  page_number = 2;
  int32 result_per_page = 3;
}

message SearchResponse{
    string result = 1;
}
Copy the code

Add comments

To add comments to.proto files, use the C/C++/ Java-style (//) and (/**… */) Syntax format

// Declare using proto3 syntax; If this is not specified, the compiler will use Proto2. The specified syntax must be the first line of the non-empty, non-commented file
syntax = "proto3";

// We define a SearchRequest message format with three fields
message SearchRequest{
  // The query field is a string
  string query = 1;
  // The page_number field of the 32-bit integer (int32) type
  int32  page_number = 2;
  // The result t_per_page field is a 32-bit integer (int32)
  int32 result_per_page = 3;
}

message SearchResponse{
  /** The result field of the string type */
  string result = 1;
}
Copy the code

Reserved fields (Reserved)

If the message type is updated by deleting the field completely or commenting it out, the user can potentially reuse the field number when making their own updates to the type. If they later load the same old version of. Proto, this can cause serious problems including data corruption, privacy errors, etc. To make sure this doesn’t happen, you can specify the field number (or name) to keep the deleted field, which can also cause JSON serialization problems. If the user tries to use these field identifiers, the compiler will report an error.

message Foo {
  reserved 2.15.9 to 11; // Reserve the field number
  reserved "foo"."bar"; // Reserve the field name
  string a = 2; // The compiler reported an error because 2 was marked as a reserved field
  string foo = 1; // The compiler reported an error because "foo" was already marked as a reserved field
}
Copy the code

Notice, not in the samereservedBoth the field name and field number are used in the statement.

.protoWhat does the file generate?

When a.proto file is run with the Protocol buffer compiler, the compiler generates code for the selected language that can manipulate the message types defined in the.proto file, including getting and setting field values, serializing the message to an output stream, and parsing the message from an input stream.

  • C++: The compiler will make a list for each.protoFile to generate a.hFile and a.ccFile,.protoEach message in the file has a corresponding class.
  • Java: The compiler generates one for each message type.javaFiles, as wellBuilderA special class for creating an instance of the message class.
  • Kotlin: in addition to theJavaBeyond the generated code, the compiler does not yet generate one per message.ktFile that contains information that can be used to simplify the creation of message instancesDSL.
  • Python: It’s a little different.PythonThe compiler to.protoEach message type in the file generates a module with a static descriptor that works with a metaclass (metaclassAt run time (runtime) is used to create the requiredPythonData access class.
  • go: The compiler generates one for each message type.pd.goFile.
  • Ruby: The compiler generates one for each message type.rbFile.
  • Objective-C: The compiler generates one for each message typepbobjc.hFiles andpbobjcmFile,.protoEach message in the file has a corresponding class.
  • C#: The compiler generates one for each message type.csFile,.protoEach message in the file has a corresponding class.
  • Dart: The compiler will be.pb.dartA file with classes is generated for each message type in the file.

Scalar type (scalar types)

.proto Type Notes C++ Type Java/Kotlin Type[1] Python Type[3] Go Type Ruby Type C# Type PHP Type Dart Type
double double double float float64 Float double float double
float float float float float32 Float float float double
int32 Use variable length encoding. Encoding negative numbers is inefficient – if your field may have negative values, use SINt32 instead. Uses variable-length encoding. Inefficient for encoding negative numbers — If your field is likely to have negative values, use sint32 instead. int32 int int int32 Fixnum or Bignum (as required) int integer int
int64 Use variable length encoding. Encoding negative numbers is inefficient – if your field may have negative values, use SINt64 instead. Uses variable-length encoding. Inefficient for encoding negative numbers — If your field is likely to have negative values, use sint64 instead. int64 long int/long[4] int64 Bignum long integer/string[6] Int64
uint32 Use variable length encoding. Uses variable-length encoding. uint32 int[2] int/long[4] uint32 Fixnum or Bignum (as required) uint integer int
uint64 Use variable length encoding. Uses variable-length encoding. uint64 long[2] int/long[4] uint64 Bignum ulong integer/string[6] Int64
sint32 Use variable length encoding. Signed integer value. These encode negative numbers more efficiently than the regular INT32. Uses variable-length encoding. Signed int value. These more efficiently encode negative numbers than regular int32s. int32 int int int32 Fixnum or Bignum (as required) int integer int
sint64 Use variable length encoding. Signed integer value. These encode negative numbers more efficiently than the regular INT64. Uses variable-length encoding. Signed int value. These more efficiently encode negative numbers than regular int64s. int64 long int/long[4] int64 Bignum long integer/string[6] Int64
fixed32 It’s always four bytes. If the value is usually greater than 228, it is more effective than uint32. Always four bytes. More efficient than uint32 if values are often greater than 228. uint32 int[2] int/long[4] uint32 Fixnum or Bignum (as required) uint integer int
fixed64 It’s always eight bytes. If the value is usually greater than 256, it is more efficient than uint64. Always eight bytes. More efficient than uint64 if values are often greater than 256. uint64 long[2] int/long[4] uint64 Bignum ulong integer/string[6] Int64
sfixed32 It’s always four bytes. Always four bytes. int32 int int int32 Fixnum or Bignum (as required) int integer int
sfixed64 It’s always eight bytes. Always eight bytes. int64 long int/long[4] int64 Bignum long integer/string[6] Int64
bool bool boolean bool bool TrueClass/FalseClass bool boolean bool
string Strings must always contain UTF-8 encoded or 7-bit ASCII text and must not exceed 232 in length. A string must always contain UTF-8 encoded or 7-bit ASCII text, and cannot be longer than 232. string String str/unicode[5] string String (UTF-8) string string String
bytes Can contain any sequence of bytes up to 232. May contain any arbitrary sequence of bytes no longer than 232. string ByteString str []byte String (ASCII-8BIT) ByteString string List
  • [1] KotlinUse fromJavaOf the corresponding types, even unsigned types, to ensure in mixingJava/KotlinCompatibility in the code base.
  • In [2]JavaIn, unsigned32And the64Bit integers are represented by their signed counterparts, with the highest bit simply stored in the sign bit.
  • [3] In all cases, setting a value for a field performs type checking to ensure that it is valid.
  • [4] 64Bit or unsigned32Bitwise integers are always represented aslong, but if given when setting the fieldint, can beint. In all cases, the value must be appropriate for the type represented when set. See [2].
  • [5] PythonThe string is represented asunicodeBut if I give youASCIIString, can bestr(That could change).
  • [6] 64Using integers on a bit machine,32Use strings on bit machines.

The default value

When a message is parsed, if the encoded message does not contain a particular Singular element, the corresponding field in the parsing object is set to the default value for that field. These defaults are type dependent:

  • forstringThe default value is an empty string.
  • forbytes, the default value is nullbytes.
  • forbool, the default value isfalse.
  • For numeric types, the default is0.
  • For enumerations, the default is the first enumeration variable defined, which must have a value of 0.
  • For message fields, not set. Its value depends on the language.

Note: For scalar message fields, once the message is parsed, it is not possible to tell whether the field is really set to the default value (for example, bool variables are set to false) or not set at all: keep this in mind when defining message types. For example, if you don’t want certain behaviors to be performed by default, don’t switch them with Boole set to false. Also, if a scalar message field is set to its default value, the change value will not be serialized during transmission.

The enumeration

When you are defining a message type, you may want its field to use only one value from the predefined list.

message SearchRequest {
  string query = 1;
  int32 page_number = 2;
  int32 result_per_page = 3;
  
  enum Role {
    RESIDENT = 0;
    CLEANER = 1;
    ADMIN = 2;
    OWNER = 3;
    SU = 4;
  }
  Role role = 4;
}
Copy the code
  • There has to be a zero so that we can take0As the default value for the number
  • Zero value hours the first element, so as to facilitateproto2Semantically compatible, the first enumeration value is always the default.

You can also set aliases for enumerations. Option ALLOW_alias needs to be set to true, otherwise the protocol compiler will generate an error message.

enum Role {
  option allow_alias = true;
  RESIDENT = 0;
  HOUSEHOLDER = 0;
  ENFORCER = 1;
  GATHERER = 2;
  PC_ADMIN = 3;
}

enum UserRole {
  RESIDENT = 0;
  HOUSEHOLDER = 0;
  ENFORCER = 1;
  GATHERER = 2;
  PC_ADMIN = 3;
}
Copy the code
  • Enumeration constants must be in32Range of integer values. becauseenumValues use variable encoding, which is not efficient for negative numbers and is therefore not recommendedenumUse negative numbers in.
  • You can define an enumeration inside a message definition, or you can define an enumeration type outside the message, so that the enumeration values can be the same.protoIn any message defined in the file. It is also possible to use an enumeration type defined in another message in one message — usingMessageType.EnumTypeSyntax format. But at the same time.protoEnumeration types are defined in the file. The values of enumeration types cannot be the same. The compiler will assume it already exists.
  • When you compile a useenumthe.protoFile, the generated code will containJavaorC++The correspondingenum, in view of thePythonThe specificEnumDescriptorClass to create a series of symbolic constants containing numeric values in the class generated by execution.

During deserialization, unrecognized enum values remain in the message, although how this value is represented when deserializing the message depends on the language. In languages such as c++ and Go that support open enumeration types that use values outside the specified symbol range, the unknown enumeration value is stored only as its underlying integer representation. In languages with closed enumeration types, such as Java, case in the enumeration is used to represent unrecognized values, and special accessors are available to access the underlying integers. In either case, if the message is serialized, unrecognized values are still serialized with the message.

Keep variable

If you update an enumeration type by completely deleting or annotating a field, later users will be able to reuse the field’s ordinal number when updating their own types. If they later use an older version of.proto, it can cause serious problems, including data corruption, privacy bugs, etc. One way to avoid this problem is to indicate that the fields you want to remove need to be reserved (or the names that will cause problems during JSON serialization), so that the protocol Buffer compiler will alert users when they use them in the future. You can specify the range of numbers you want to keep up to the maximum possible (via the Max keyword).

enum Foo {
  reserved 2.15.9 to 11.40 to max;
  reserved "FOO"."BAR";
}
Copy the code

Notice, not in the samereservedStatements that mix field names and field ordinals.

Use other message types

You can also use other message types as field types.

For example, if you want to include a Result message in the SearchResponse message, you can define a Result message type in the same.proto file, and then declare a Result field in the SearchResponse.

message SearchResponse {
  repeated Result results = 1;
}

message Result {
  string url = 1;
  string title = 2;
  repeated string snippets = 3;
}
Copy the code

Import the definition

In the example above, the Result message type and SearchResponse are defined in the same.proto file. What if the field type you want to use is already defined in another.proto file?

You can use these definitions by importing them from other.proto files. To use other.proto definitions, you need to import declarations in the header of your file:

import "myproject/other_protos.proto";
Copy the code

By default you can only use the definitions in the directly imported.proto file. However, sometimes you need to move a.proto file to a new location. Instead of moving the.proto file directly and updating all call points in a single change, you can now place a dummy.proto file in the old location to use the command import public to forward all imports to the new location. Anyone who imports the Proto that contains the import public statement can import the public dependencies. Such as:

// new.proto
// All definitions are moved here
Copy the code
// old.proto
// This is the proto that all clients are importing.
import public "new.proto";
import "other.proto";
Copy the code
// client.proto
import "old.proto";
// You use definitions from old.proto and new.proto, but not other.proto
Copy the code

The compiler looks for imported files in a series of specified directories (specified on the command line by the -i / –proto_path flag). If not specified, the compiler will look in the current directory. Normally you should set the –proto_path flag to the root directory of your project and use the full path import.

Use the Proto2 message type

You can import and use proto2 message types in your Proto3 message and vice versa. However, proto2 enumerations cannot be used directly in Proto3 (they can be used in imported Proto2 messages).

Nested types

You can define and use other message types in one message type, as in the following example of a Result message defined in a SearchResponse:

message SearchResponse {
  message Result {
    string url = 1;
    string title = 2;
    repeated string snippets = 3;
  }
  repeated Result results = 1;
}
Copy the code

If you want to reuse the message outside of the Parent message Type, you can use parent-type:

message SomeOtherMessage {
  SearchResponse.Result result = 1;
}
Copy the code

You can nest as deep as you want:

message Outer {                  // Level 0
  message MiddleAA {  // Level 1
    message Inner {   // Level 2
      int64 ival = 1;
      bool  booly = 2; }}message MiddleBB {  // Level 1
    message Inner {   // Level 2
      int32 ival = 1;
      bool  booly = 2; }}}Copy the code

Update message type

If an existing message type no longer meets your needs, for example, you want to add a new field to the message format, but still want to use the code generated for the format. Don’t worry! Updating message types without breaking your existing code is simple. Just remember the following rules:

  • Do not change the ordinal number of an existing field.
  • If you add new fields, any messages serialized in the old format can still be parsed by the newly generated code. You should remember the default values for these elements so that the new code can correctly interact with the messages generated by the old code. Similarly, messages created by the new code can be parsed by the old code: the old binary simply ignores the new field when it is parsed.
  • The field can be removed as long as its sequence number is no longer used by the message type you are updating. You can either rename the fields, add the prefix “OBSOLETE_”, or leave the field number alone, and so on.protoFuture users of the number will not accidentally reuse the number.
  • int32,uint32,int64,uint64andboolAre all compatible — this means you can change a field from one of these types to the other without breaking forward or backward compatibility. If you parse out a number from the wire that doesn’t fit the corresponding type, then you will get and inc++Cast the number to the same effect as that type (for example, if the64The digit is read asint32, then it will be truncated to32A).
  • sint32andsint64Is compatible with each other, but not with other integer types.
  • stringandbytesCompatible,byteswithUTF-8Compatible.
  • If the byte contains an encoded version of the message, the embedded message is associated withbytesCompatible.
  • fixed32withsfixed32,fixed64andsfixed64Compatible.
  • In the transmission formatenumwithint32,uint32,int64,uint64Compatibility (Note that incompatible parts of the variable will be truncated). Note, however, that when a message is deserialized, the client code is treated differently: for example, although unrecognizedproto3In theenumThe type is stored in the message, but how it is represented when the message is deserialized depends on the language.intA field always retains its value. Modify thenew oneofThe individual variables in the member are secure and binary compatible. If you are sure that no code sets multiple fields at once, it is probably safe to move multiple fields into a new field. It is not safe to move any field into an existing field.

The unknown fields

Unknown fields are data that the Protocol Buffer cannot parse when serializing the data. For example, when the old binary is parsing data sent by the new binary with new fields, these new fields will become unknown fields in the old binary.

Initially, Proto3 always discarded unknown fields during parsing, but after version 3.5, the retention of unknown fields was reintroduced to match Proto2’s behavior. In versions 3.5 and later, unknown fields are retained during parsing and included in the serialized output.

Any

The Any message type allows you to use your message type as an embedded type without a.proto definition. Any contains Any serialized message as bytes and a URL that acts as a globally unique identifier and resolves to that message type. To use Any types, you need to import the Google/protobuf/Any proto.

import "google/protobuf/any.proto";

message ErrorStatus {
  string message = 1;
  repeated google.protobuf.Any details = 2;
}
Copy the code

The default type URL for a given message type is type.googleapis.com/_packagename_._messagename_.

Different language implementations will support runtime library helpers to package and unpack Any values in a type-safe way — for example, in Java, the Any type will have special pack() and unpack() methods, while in C++ there are PackFrom() and UnpackTo() methods:

// Storing an arbitrary message type in Any.NetworkErrorDetails details = ... ; ErrorStatus status; status.add_details()->PackFrom(details);// Reading an arbitrary message from Any.ErrorStatus status = ... ; for (const Any& detail : status.details()) { if (detail.Is<NetworkErrorDetails>()) { NetworkErrorDetails network_error; detail.UnpackTo(&network_error); . processing network_error ... }}Copy the code

Currently, for processingAnyA runtime library for the type is under development.

If you are already familiar with proto2 syntax, Any can hold Any Proto3 message, similar to a Proto2 message that allows for extensions.

Oneof

If you have a message that contains more than one field and can only set one field at a time, you can enforce this behavior and save memory by using the oneof feature.

Oneof fields are similar to regular fields, except that all fields share the same Oneof memory and at most one field can be set at a time. Setting any member in the oneof field automatically clears the others. Depending on the language you’re using, you can use (if necessary) specific case() or WhichOneof() methods to check which variables in Oneof are set.

useOneOf

To define a Oneof field in your.proto file, you can follow the Oneof keyword with your Oneof name, as in test_oneof:

message SampleMessage {
  oneof test_oneof {
    string name = 4;
    SubMessage sub_message = 9; }}Copy the code

You can then add your oneof field to the oneof definition. You can use any field except that you can’t use repeated fields.

In the code you generate, the oneof field has the same getters and setters as the regular field. If necessary, you can also use specific methods to determine which values in oneof are set.

Oneoffeatures

  • Set up theoneofAny member in the field automatically clears the others. If you set more than one field, only the last field set will remain the variable. 
    SampleMessage message;
message.set_name("name"); CHECK(message.has_name()); message.mutable_sub_message(); // Will clear name field. CHECK(! message.has_name());Copy the code
  • If the parser encounters the same in the networkOneofWith only the last seen member used when parsing the message.
  • You can’t userepeated.
  • oneofField using reflectionAPIs.
  • If you setoneofFields are default values (such as Settingsint32Field is0) of the fieldcaseWill be set and serialized during transmission.
  • If you useC++Make sure your code does not cause a memory crash. The following code will cause a crash because theset_name()methodssub_messageAlready deleted. 
    SampleMessage message;
SubMessage* sub_message = message.mutable_sub_message();
message.set_name("name");      // Will delete sub_message
sub_message->set_...            // Crashes here
Copy the code
  • Also inC++If you useSwap()Let’s swap two bandsoneofsEach message will be followed by anotheroneof caseEnd: In the following example,msg1It will havesub_message.msg2It will havename. 
    SampleMessage msg1;
msg1.set_name("name");
SampleMessage msg2;
msg2.mutable_sub_message();
msg1.swap(&msg2);
CHECK(msg1.has_sub_message());
CHECK(msg2.has_name());
Copy the code

Backward compatibility problem

Be careful when adding or removing the oneof field. If oneof is detected with a return value of None/Not_SET, it may mean that this oneof has not been set or has been set in a different version of Oneof. The difference between the two cannot be distinguished because it is impossible to determine whether an unknown field in the transport is given to a member of Oneof.

TagReuse problem

  • Move in/out fields tooneof: After message serialization and parsing, you may lose part of the message (some fields are cleaned up). However, you can safely move individual fields to a new oneoneofField, you can move multiple fields if you know that only one field is set.
  • Add after removing a oneof field: May reset your current Settings after message serialization and parsing.
  • Cut/merge oneof: Similar to the problem of moving regular fields.

Maps

If you want to create an association map as part of your data definition, Protocol Buffers provides a handy syntax:

map<key_type, value_type> map_field = N;
Copy the code

Key_type can be any integral or string type (that is, any scalar type except floating point and bytes). Note enum is not a valid key_type. Value_type can be any type except for other maps.

So, if you want to create a project map, each project is associated with a string key, defined as follows:

map<string, Project> projects = 3;
Copy the code
  • MapThe field cannot berepeated.
  • The network format sorting of the mapping values and the mapping iteration sorting are undefined, so you cannot rely on the composition of your mapping elements in a particular sorting.
  • for.protoWhen the text format is generated, the mapping is sorted by key. Number keys sort by number size.
  • When parsing/merging from the network, if there are multiple copies of the key, the last key encountered is used. When parsing a map from a text format, the resolution may fail if a copy of the key exists.
  • If you only provideMapField’s key without providing a value. The behavior of field serialization varies from language to language. inC++,JavaandPythonIn, the value is serialized to the default value of the type, which is not serialized in other languages.

Backward compatibility

The map syntax is equivalent to the following, so protocol buffer implementations that do not support Map can still process your data:

message MapFieldEntry {
  optional key_type key = 1;
  optional value_type value = 2;
}

repeated MapFieldEntry map_field = N;
Copy the code

Any protocol buffer implementation that supports mapping must generate and accept data that is acceptable by the above definition.

packages

You can add package specifiers to.proto files to avoid name conflicts for protocol message type keys.

package foo.bar;
message Open {... }Copy the code

Later when defining your message type fields, you can use the package specifier:

message Foo {... foo.bar.Open open =1; . }Copy the code

The way package specifiers affect generated code depends on the language you choose:

  • inC++, the generated classes are packaged intoC++In the namespace. Such as:OpenLocated in thefoo::barNamespace.
  • inJava,packageAs aJavaPackage used except in.protoAdditional provided in the fileoption java_package.
  • inPython,packageInstructions are ignored,PythonModules are organized according to their location in the file system.
  • inGo,packageWill be used asGoPackage name, unless in.protoAdditional provided in the fileoption go_package.
  • inRuby, the generated classes are packaged and embedded inRubyAnd convert to the desiredRubyCase style (capitalize the first letter; If the first character is not a letter,PB_Is the prefix). Such as:OpenLocated in thefoo::barNamespace.
  • inC#,packageIn being converted toPascalCaseLater used as a namespace, except in.protoAdditional provided in the fileoption csharp_namespace.

Package and name resolution

Type name resolution in the Protocol buffer language is similar to C++ : first in the innermost layer, then in the next layer, and so on, with each package “inside” its parent. “.” The beginning (for example,.foo.bar.baz) means that the search starts at the outermost scope.

The Protocol buffer compiler parses all type names from the imported.proto file. Even with different scoping rules, the code generated by each language knows how to use each type.

Define the service

If you are using your message type in an RPC (remote call) system, you can define the RPC service interface in the.proto file, after which the Protocol Buffer compiler generates the service interface code and stubs for the selected language. For example, if you want to define an RPC service that uses your SearchRequest and returns a SearchResponse, you can define this in the.proto file:

service SearchService {
  rpc Search (SearchRequest) returns (SearchResponse);
}
Copy the code

The most direct RPC system to use protocol Buffers is gRPC: a language – and platform-independent open source RPC system developed by Google. GRPC works well with Protocol Buffers. It allows you to generate RPC code directly from.proto files using the special Protocol Buffer plugin.

If you do not want to use gRPC, you can also use the Protocol buffer in your own RPC implementation.

There are also ongoing third-party projects to develop RPC implementations for protocol Buffers.

JSON Mapping

Proto3 supports the Json encoding specification, which makes it easier to share data between different systems. In the following table, the encodings are described type by type.

If a value is lost or null in the JSON encoding, it will be interpreted as the appropriate default value when parsed to the Protocol buffer. If a field in the Protocol buffer has a default value, it will be omitted from the JSON-encoded data by default to save space. In the Json encoded output, the implementation can provide options with default fields.

proto3 Json Json sample note
message object {“fooBar”:v,”g”:null,_} Generate a Json object. The message field name is mapped to lowerCamelCase and becomes the key of the Json object. If you specifyjson_nameField option, the specified value will be used as a key. The parser accepts both the lowerCamelCase name (or one specified using json_name) and the native proto field name. Null is acceptable for all field types and is considered the default value for that type.
enum string “FOO_BAR” Use the enum value name specified in PROto. The parser accepts both enumeration names and integer values.
map<K,V> object {“K”:v,_} All keys are converted to strings.
repeated V array [v, …] nullIs treated as an empty list [].
bool true,false true,false
string string “Hello World!”
bytes base64 string “YWJjMTIzIT8kKiYoKSctPUB+” The Json value becomes a standard Base64 encoded string with padding. Standard or URL-safe Base64 encoding with/without padding is acceptable.
int32,fixed32,uint32 number 1,-10,0 The Json value is changed to a decimal number. Either a number or a string is acceptable.
int64,fixed64,uint64 string “1”, “to 10” The Json value becomes a decimal string. Either a number or a string is acceptable.
float,double number 1.1, 10.0, 0, “NaN”, “Infinity” The Json value becomes a number or one of “NaN”, “Infinity”, or “-infinity”. Either a number or a string is acceptable. Exponential notation is also accepted.
Any object {“@type”:”url”,”f”:v,… } If Any contains a value with a specific Json mapping, it will be converted to the following format: {“@type”: XXX, “value”: yyy}. Otherwise, the value is converted to a Json object, and the “@type” field is inserted to indicate the actual data type.
Timestamp string “The 1972-01-01 T10:00:20. 021 z” With RFC 3339, the generated output is always z-normalized and uses 0, 3, 6, or 9 decimal places. Offsets other than “Z” are also acceptable.
Duration string “1.000340012 s”, “1 s” Depending on the desired precision, the generated output always contains 0, 3, 6, or 9 decimal places, followed by the suffix “s”. Any decimal (or no decimal) is acceptable as long as the nanosecond accuracy and the suffix “s” are met.
Struct object {… } Any Json object.
Wrapper types various types 2,”2″,”foo”,true,”true”,null,0,… The wrapper uses the same JSON representation as the wrapped primitive type, but allows and retains null during data conversion and transfer.
FieldMask string “f.fooBar,h”
ListValue array [foo,bar, …]
Value value Any Json value
NullValue null Json null
Empty object {} Any empty Json object.

JSONoptions

Proto3’s Json implementation supports the following options:

  • Null field with default value: By default, inproto3 JSONFields with default values are omitted from the output. The implementation can provide an option to override this behavior and output the field with its default value.
  • Ignore unknown types: By default,Proto3 JsonThe parser rejects unknown fields, but you can provide an option to ignore unknown fields when parsing.
  • useprotoField instead oflowerCamelCaseThe name of the: By default,proto3 JsonThe output of should convert the field name tolowerCamelCaseAnd as aJsonName used. This implementation can be used by providing optionsprotoField asJsonThe name.Proto3 JsonThe parser is designed to accept both the transformedlowerCamelCaseThe name andprotoField name.
  • Indicate theenumValue as an integer instead ofstring: By default, inJsonThe name of the enumeration value used in the output. The option to use numbers instead of enumerations is specified.

options

The individual declarations in the.proto file can be commented with a number of options. Options do not change the overall meaning of the declaration, but may affect how it is handled in a particular context. A complete list of available options in the Google/protobuf/description. The proto is defined.

Some options are file-level, meaning they should be written at the beginning of a message, enumeration, or service definition. Some options are message-level, meaning they should be written in the message definition. Some options are field options, meaning they should be written in the field definition. Options can also be written in enumeration types, enumeration values, service types, and service methods; however, no options currently exist that are useful for Any. Here are some of the most common options:

  • Java_package (file level) : This is the package you want to use to generate Java classes. If no additional javA_package option is given in the.proto file, the default is to use the proto package (as indicated by the package keyword in the.proto file). In general, however, the Proto package is not suitable for Java packages because you do not want the Proto package to be expanded in reverse domain names. This is invalid if no Java code is generated.

    option java_package = "com.example.foo";
Copy the code

  • Java_outer_classname (file level) : The classname (and file name) of the outermost Java class you want to generate. If an explicit java_outer_className is not specified in the.proto file, the classname is constructed by converting the.proto file name to a camel’s form (for example, foo_bar.proto to foobar.java). This option is invalid if no Java code is generated.

    option java_outer_classname = "Ponycopter";
Copy the code

  • Java_multiple_files (file level) : If false, only. Java generates a file for the. Proto file, all Java classes/enumerations/etc. Messages generated for top-level messages, services, and enumerations are nested in external classes. If true,.java will generate separate files for each Java class/enumeration/and so on. Generated for top-level messages, services, and enumerations, and the wrapper Java classes generated for this.proto file will not contain any nested classes/enumerations/etc. This is a Boolean option, which defaults to false. This option is invalid if no Java code is generated.

    option java_multiple_files = true;
Copy the code

  • Optimize_for (file option) : Can be set to SPEED, CODE_SIZE, or LITE_RUNTIME. This affects C++ and Java code generators (and possibly third-party generators) in the following ways:

    • SPEED (default) :Protocol bufferThe compiler generates code for serialization, parsing, and other common operations for your message type. This code is highly optimized.
    • CODE_SIZE:Protocol bufferThe compiler generates the smallest class that relies on shared, reflected code for serialization, parsing, and other operations. Hence the generated code ratioSPEEDIt’s a lot smaller, but it’s also a lot slower.ClassesWill still be implemented withSPEEDThe same common patternAPI. This model contains a large number of.protoFiles and not all of them need to be generated quickly.
    • LITE_RUNTIME:Protocol bufferThe compiler relies on “lightweight” runtime libraries (usinglibprotobuf-liteRather thanlibprotobuf).liteThe runtime is much smaller (about an order of magnitude smaller) than the full library, but some features, such as descriptors and reflection, are ignored. This is especially useful for applications that run on limited platforms, such as mobile phones. The compiler will still look like inSPEEDPattern that generates a fast implementation of all methods. The generated classes will be implemented in each language onlyMessageLiteInterface that provides only the completeMessageA subset of interface methods.
    option optimize_for = CODE_SIZE;
Copy the code

  • Cc_enable_arenas (file level) : enable arena allocation for C++ code generation.

  • Objc_class_prefix (file level) : Sets the prefix for all objective-C classes generated by the. Proto file. There is no default value. You should use the apple recommended prefix, which is 3-5 capital letters. Note that apple retains all 2-letter prefixes.

  • Deprecated (file-level) : If set to true, the field has been deprecated and should not be used by new code. In most languages, this option has no real effect. In Java, this becomes an @deprecated annotation. In the future, code generators in other languages may generate deprecated comments on field accessors, which will cause the compiler to warn when it tries to use the field. If the field is no longer in use and you don’t want new users to use it, consider replacing the field declaration with a reserved statement.

    int32 old_field = 6 [deprecated=true]
Copy the code

The customoption

The Protocol Buffer also allows you to define and use custom options. This is an advanced feature that most people don’t use. If you really want to create custom options, check out Proto2. Note that extensions are used to create custom options, which are the only custom options allowed in Proto3

Compile the generated

To include Java, Python, C++, Go, Ruby, Objective-C, or C# code for the message type you define from the.proto file, you need to allow protocol Buffer compiler Protoc. If you haven’t already installed the compiler, you need to. For Go, you also need to install specific build plug-ins.

The Protocol compiler uses the following:

protoc --proto_path=IMPORT_PATH --cpp_out=DST_DIR --java_out=DST_DIR --python_out=DST_DIR --go_out=DST_DIR --ruby_out=DST_DIR --objc_out=DST_DIR --csharp_out=DST_DIR path/to/file.proto
Copy the code

  • IMPORT_PATH specifies the path to look for the. Proto file when the import command is resolved. By default, the current directory is used. Multiple import commands can be specified by using the –proto_path option multiple times, and they will be retrieved in order. –proto_path can be shortened to -i =IMPORT_PATH.

  • You can provide one or more output commands:

    • --cpp_outinDST_DIRGenerate in directoryC++The code.See C++ generated code references.
    • --java_outinDST_DIRGenerate in directoryJavaThe code.See Java Generated Code references for more details.
    • --python_outinDST_DIRGenerate in directoryPythonThe code.See The Python generated Code reference.
    • —go_outinDST_DIRGenerate in directoryGoThe code.See Go Generate code references.
    • —ruby_outinDST_DIRGenerate in directoryRubyThe code.See Ruby generated Code references for details.
    • —objc_outinDST_DIRGenerate in directoryObject-CThe code.See object-c generated code reference for details.
    • —csharp_outinDST_DIRGenerate in directoryC#The code.See C# generated code references.
    • —php_outinDST_DIRGenerate in directoryPHPThe code.See PHP generated Code references for details.

    As an added convenience, if DST_DIR is.zip or.jar, the compiler will generate a zip package with the specified name. The.jar output also provides a manifest file as required by the Java JAR specification. Note that if the output file already exists, it will be overridden and the compiler will not make a new copy.

  • You must provide one or more.proto files as input. Multiple.proto files can be specified at one time. Although these files are named relative to the current directory, each file must reside in one of the IMPORT_PATH so that the compiler can determine its canonical name.