How to decode protobuf without a schema
A protobuf message is a list of fields, and each field starts with a tag. The tag is a varint that holds the field number and a wire type. The wire type says how the value is stored: as a varint, as 4 or 8 fixed bytes, or as a length followed by that many bytes. Because the wire type is in the data, a protobuf decoder can read any message without the schema, the way protoc --decode_raw does. The names and the exact types are lost, but the structure is not.
08 96 01 12 07 74 65 73 74 69 6e 67 1a 03 08 96 01 22 06 03 8e 02 9e a7 05 2d 00 00 80 3f 31 18 2d 44 54 fb 21 09 40 38 01 42 04 de ad be ef
{"1":150,"2":"testing","3":{"1":150},"4":[3,270,86942],"5":1065353216,"6":"4614256656552045848","7":1,"8":"3q2+7w=="}The JSON uses field numbers as keys. A number that repeats becomes an array, and 64-bit values that do not fit in a JavaScript number are written as strings. The bytes are decoded in your browser, as the Privacy Policy describes, so captured traffic stays on your computer.
Wire types and what the numbers mean
Only four wire types occur in modern protobuf, and each field shows the one it uses.
- Varint (type 0): int32, int64, uint32, uint64, sint32, sint64, bool and enum. The decoder shows the unsigned value, the signed value and the zigzag value, because the bytes do not tell which type the author used.
- Fixed 64-bit (type 1): fixed64, sfixed64 and double. Shown as unsigned, signed and as a double.
- Length-delimited (type 2): string, bytes, nested messages and packed repeated fields.
- Fixed 32-bit (type 5): fixed32, sfixed32 and float. Shown as unsigned, signed and as a float.
Zigzag is how sint32 and sint64 store negative numbers compactly. The value 3 as zigzag is -2. Wire types 3 and 4 belonged to groups, an old feature that this decoder does not read.
Strings, nested messages and packed fields
A length-delimited field is ambiguous. The bytes 08 96 01 can be the nested message {1: 150}, or three raw bytes. The bytes of the word John can be a string, and they could also, rarely, parse as a message by chance. The decoder uses a simple rule. Valid UTF-8 text with no control characters is shown as a string, bytes that parse completely as a message are shown as a message, a run of two or more varints or of fixed numbers is shown as packed numbers, and anything else is shown as bytes.
Because that is a guess, every such field has buttons for the other readings. Click message, string, packed varints or bytes and the field is shown that way. The choice is only for the view, so nothing is lost. With a .proto file the types are known and the guess is not needed.
Decode with a .proto file
Paste your message definitions into the .proto box and choose the root message. Fields get their names, enums show their names, nested messages and maps are decoded with their types, repeated fields become arrays and packed fields are unpacked. Fields that are not in the schema stay in the output under their number, so a schema that is out of date does not hide data.
{"id":150,"name":"testing","inner":{"value":150},"scores":[3,270,86942],"ratio":1,"pi":3.141592653589793,"ok":true,"blob":"3q2+7w=="}The parser reads proto2 and proto3 definitions: messages, nested messages and enums, repeated and optional fields, maps, oneof, comments and options. It does not follow imports, so paste the messages that you need together. If a field's wire type does not fit its declared type, the field is shown without the schema and a warning explains why.
gRPC and message streams
gRPC wraps each message in a 5-byte header: one byte that says whether the message is compressed, and four bytes for its length. With Framing on Auto, the decoder checks whether the whole input is made of such frames and decodes each one. You can also force gRPC, or choose Length-delimited stream, where every message starts with a varint length, as in many log and capture files.
00 00 00 00 05 08 96 01 12 00
[{"1":150,"2":""}]A frame with the compressed flag set cannot be decoded here, and the decoder says so. Decompress it first. For gzip data, Gzip Decompress can help.
Errors and damaged data
When the data is not valid protobuf, the decoder tells you where. Messages such as the field tag is cut off, field 2 says it holds 7 bytes but only 1 are left, and wire type 6 does not exist come with the byte offset. Fields that were read before the problem are still shown, so you can see how far the data was valid.
A truncated message often means that the capture is cut off, that a header or a length prefix was included in the bytes, or that the data is compressed or encrypted. Try turning gRPC framing on or off, and check whether you copied the whole body. To turn hex into text first, Hex to Text and Base64 Decode help.
Limits
- It decodes protobuf. It does not encode JSON to protobuf or call gRPC servers.
- Without a schema, field names, signedness and the choice between string and bytes are guesses. Use the buttons or a .proto file to be sure.
- The hex view shows the first 4,096 bytes, and a message lists up to 1,500 fields. The JSON tab has everything.
- Group fields, proto extensions and imports are not supported.
- A message is read up to 200,000 fields and 12 levels of nesting. Deeper levels are shown as bytes, and you can switch them to a message with the buttons.
- A file can be up to 2 MB. Inputs near that size, with many thousands of fields, can take a couple of seconds to show.