The original address: www.charlesproxy.com/documentati…

Original author:

Release Date:

Background information

To quote from the Protocol buffer website: “The protocol buffer is Google’s language neutral, platform neutral, extensible structured data mechanism.

Protocol buffers are Google’s language-neutral, platform-neutral, extensible mechanism for serializing structured data — think XML, but smaller, faster, and simpler. You only have to define once how you want your data structured, and then you can easily write or read your structured data from various streams using specially generated source code in a variety of languages –Java, C++, or Python.

The Protocol Buffers serialization format is a binary encoding format that is not easily read by humans. Since protocol buffer messages are typically exchanged over HTTP, we have added full support for viewing and editing protocol buffer messages in a human-readable manner.

Charles currently supports version 2.4.1 of Protocol Buffers, which is largely backward compatible with earlier versions.

An overview of the

When the CONTent-Type header has a MIME Type of Application/x-Protobuf or Application/X-Google-Protobuf, Charles recognizes that the HTTP request or response contains a protocol buffer message. When viewing content, two new HTTP body content viewers become available, the Protobuf text viewer and the Protobuf structured viewer.

In order for these viewers to be able to display message content, they need access to the protocol buffer descriptor of the message contained in the HTTP body content. Charles looks for the desc and messageType arguments in content-type to find the location of FileDescriptorSet (*.desc file) and the fully qualified messageType name, which it then uses to retrieve and load the appropriate descriptor for the message. FileDescriptorSet can be used by the protocol buffer compiler (Protoc) by using the -o or –descriptor_set_out option, e.g. Protoc -omodel.desc model.proto, Generated from a *.proto file.

Finally, the content of the HTTP body may contain a single message or a list of messages serialized using a standard protocol buffer length delimited format. To determine whether the HTTP body contains a single message or a delimit list of messages, look for an optional delimit parameter in the content type. This parameter must exist and have a value of true to indicate that a delimited list of messages has been sent. When a delimited list of messages is sent, all messages must be of the same message type.

This means that a full content-Type header will look like: Content-Type: application/x-protobuf; desc=”http://localhost/Model.desc”; messageType=”com.xk72.sample.PurchaseOrder”; Delimited = true.

Load the file descriptor set

Charles expects the desc parameter to the content-type header to contain a valid URL to the location of the FileDescriptorSet of the protocol buffer message. This can be any valid URL, but is usually a file or HTTP URL. Charles tries to retrieve the FileDescriptorSet each time he selects a Protobuf viewer to display HTTP body content, meaning that desc URLS are not resolved until needed, which is critical when Charles logs a large number of protocol buffer transactions.

FileDescriptorSet cache

In general, you’ll still get many transactions that use the same protocol to buffer message types, so Charles does implement FileDescriptorSets caching based on the standard HTTP 1.1 cache for HTTP urls and the last-modified timestamp for file urls.

This means that you can ensure FileDescriptorSet loading performance by ensuring that the Web server serving the file has proper HTTP 1.1 caching and/or validation (last-Modified and ETag) headers. Caching can also be effectively disabled by setting the appropriate HTTP 1.1 no-cache header.

When no HTTP cache or validation header exists in the FileDescriptorSet response, Charles calculates its own heuristic expiration time for the cached FileDescriptorSets. The length of time Charles can cache these resources can be configured in Charles’ Preferences->Viewers popup, which defaults to 5 minutes. It can be set to 0 to disable heuristic caching.

Note that this cache is only used to resolve desc URLs, and it does not change Charles’ behavior in logging HTTP transactions in any way.

Protobuf Text browser

The Protobuf text ontology content viewer displays the default text representation of protocol buffer messages. This with the com in the Java API. Google. Protobuf. TextFormat, Google. Protobuf. Text_format. H c + + API or thegoogle. Protobuf. Text_format The Python modules are implemented in the same format, which comes with the protocol buffer distribution.

When display the list of messages and bound, each message is by delimiter separated text: > — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — next message — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — <.

Protobuf browser

Basic information display

Protobuf Body content viewer Displays a tree structure view of protocol buffer packets, displaying the complete hierarchy of packets, including all fields and subpackets.

In the structured viewer, the first column displays the definition field name for each field in the tree structure of the protocol buffer message definition.

The second column shows the type of the field.

  • Scalar fields are given their. Proto type names, such as String, uint32, bool, sfixed64, etc.
  • The label of an enumeration is the fully qualified type name of an enumeration prefixed with Enum.
  • The field of the message type is given the fully qualified type name of the message.
  • Any repeated field is prefixed with repetition, for example: repeated string

The third column displays information about field values.

  • A scalar field displays a literal representation of its value.
  • Enumeration values are displayed with defined names.
  • A repeated field displays the number of times the field has been repeated.

Message types and duplicate fields can be extended to display individual fields or duplicates that they encapsulate. Note that for string fields, instead of the literal value of the string, its encoding is displayed to support representing non-printable characters. The encoding used is the literal format of the string in C code, so newlines become \n, tabs become \t, and so on.

Any field can be double-clicked to open a pop-up dialog box containing a string representation of the field’s value. This is useful when the field content is too large to be easily displayed in a table. Similarly, when you double-click an information input field, the entire information content contained in that field, including all its subfields, is displayed.

Processing of missing optional fields

For protocol buffered messages, fields defined as optional in the message definition need not be included in the message. In this case, the field still has an implied value when it does not explicitly exist in the message. If a default value is defined for a field in the message definition, that value will be the default, otherwise it will get normal defaults for its type, such as numeric type 0, Boolean type false, and string type empty string.

Usually these unspecified fields are hidden because they are considered meaningless. This includes hiding duplicate fields with 0 repetitions. You can configure the viewer to display these unspecified fields in Charles’s Preferences -> Viewer dialog box.

When unspecified fields are displayed, their field names, type information, and implied or default values are displayed in italics in the viewer.

The unknown fields

For protocol buffer messages, it is possible to obtain unknown fields in the message content, which usually occurs when the message definition has changed and an old message containing fields that no longer exist in the message definition is being parsed.

When this happens, the structured viewer displays an unknown segment child node in the message hierarchy that can be expanded to see the available information about the unknown field. Because the metadata typically found for these fields in the message definition is not available, all you see is the field’s numeric label, serialization line type, and serialization value. The possible lines are

  • Varint
  • 32 –
  • A 64 – bit
  • Length limit

See the protocol buffer encoding documentation for the mapping from.proto scalar types to these line types and how to decode serialized values.

Unknown information

In some cases, the protocol buffer descriptor for the message is not available. For example, this can happen if the desc or messageType arguments are not specified in the Content-Type header, or if an error occurs when parsing the URL provided in the desc argument.

When this happens, the protocol buffer message is resolved as an unknown message, which in practice means that all fields in the message become unknown and only the minimal information we can detect from the line format is displayed. The error that caused the message to be resolved as unknown is also displayed in the viewer, hopefully allowing you to fix whatever caused the problem.

Configuration options

Some aspects of the protocol buffer functionality are user configurable. These options are provided in the Viewer section of the Preferences dialog box. The Preferences dialog can be activated from the Edit menu (or the Charles menu on Mac OS X).

  • Hide Unspecified Fields — When this option is selected, Unspecified optional Fields in the protocol buffer message are hidden when the message is displayed in the structured viewer. When this option is not selected, Unspecified Fields are displayed in italics, along with their default or implied values.
  • Caching Protobuf descriptors – Enables caching of protocol buffer descriptors described in the caching section. Descriptors are not cached when no descriptor is selected.
  • Heuristic cache TTL – When caching is enabled, this is the cache TTL or expiration time used to retrieve descriptors from HTTP urls that do not specify an HTTP cache header. When there is no explicit HTTP cache header, set to 0 to never cache.
  • Clean up cache Resources – Click this button to clean up all cached protocol buffer descriptors immediately.

More information

When you happen to select two transactions in the structure view or sequence view, you can compare their contents using Comparecommand in the right-click context menu.

When both transactions contain protocol buffer messages and standard comparison viewers that are usually available, you can choose to use the Protobuf text viewer or the Protobuf structured viewer to see the comparison between transactions.

Edit the information

The Protobuf Text Viewer and Protobuf Structured Viewer can also be used when editing the content of an HTTP request. When you choose to edit an HTTP request, if it is recognized as a protocol buffer request (via a content type with application/ X-Protobuf or Application/X-Google-Protobuf), Then two Protobuf-specific viewers will be available as options for editing the requested content.

Protobuf Text viewer – Edit mode

In edit mode, the Protobuf Text Viewer simply allows you to edit a readable Text representation of protocol buffer messages. This text is generated, and using the standard protocol buffer library API (namely com in the Java API. Google. Protobuf. TextFormat class, Google.protobuf.text_format. H C++ API or thegoogle.protobuf.text_format Python module) reparsed (edited) into a serialized protocol buffer message.

One quirk you may want to be aware of is that if the original Message contains unknown fields, these fields are represented in the text generated by the protocol buffer text formatter, but they raise an error when the text tries to be re-ars (Message Type XXX has no field named YYY). This means that you need to manually remove them from the text when editing.

When editing a delimited list of news, the editor hope text delimiters are used between each message > — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — next message — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — <. You can add or remove instances of this delimiter to add or remove information from the list.

Protobuf Viewer – Edit mode

The Protobuf viewer in edit mode displays the current message content in the same structured view described in the Protobuf structured view. One key difference is that all defined fields are always displayed, regardless of whether they are currently populated. Fields defined for the message that are not currently populated are displayed in italics. This makes it easy to see which fields are present in the message and set values for those fields.

The value of any scalar field can be set by simply double-clicking on the value column (double-clicking elsewhere on the row opens a dialog box containing the value of the field, which is useful for viewing very large values). Scalar fields can be cleared (to become a field that is not populated in the message) by right-clicking the field and selecting clear Value.

Similarly, you can clear the message value field by right-clicking on the field and selecting Clear the entire message. Unpopulated message value fields are treated much like scalar fields, in that they are displayed in italics and populated with their default or implied values. Once any subfields of an unpopulated message are explicitly set, the message becomes a concrete, well-defined value.

Duplicate fields can be added by right-clicking on the field and selecting Add duplicate, or by using the Insert before or Insert After commands when right-clicking on an existing duplicate. There is also an option to delete duplicates.

Translation via www.DeepL.com/Translator (free version)