Protobuf Binary Format Basics

Juneau supports converting arbitrary POJOs to and from the Protocol Buffers binary wire format using native, bean-driven serializers and parsers. No .proto schema file or code generation is required — the field-number/type schema is derived directly from Juneau bean metadata.

The ProtobufSerializer converts POJOs directly to protobuf binary bytes, and the ProtobufParser creates POJOs directly from protobuf binary bytes.

This is distinct from the text-format Prototext (Protobuf Text Format) marshaller (text/protobuf). This codec emits the compact, non-self-describing protobuf binary wire format used by protoc-generated code.

It is interop-capable but bean-driven: it works losslessly out of the box for Juneau-to-Juneau messaging, and upgrades field-by-field to true external protoc interop when you supply explicit field numbers and scalar types via the @Protobuf annotation.

Protobuf binary support is included in juneau-marshall.

Maven	<dependency> <groupId>org.apache.juneau</groupId> <artifactId>juneau-marshall</artifactId> <version>${juneau.version}</version> </dependency>

Quick Start

// A simple bean - properties are sorted alphabetically: id=1, name=2
public class Person {
    public String name;
    public int age;
    public Person() {}
    public Person(String name, int age) { this.name = name; this.age = age; }
}

// Serialize a bean to protobuf binary bytes
Person alice = new Person("Alice", 30);
byte[] protobuf = Protobuf.of(alice);

// Parse protobuf binary bytes back into a bean (target type is required)
Person parsed = Protobuf.to(protobuf, Person.class);

Serialization

// Use the default serializer
byte[] protobuf = ProtobufSerializer.DEFAULT.write(someObject);

// Use the marshaller for convenience
byte[] protobuf = Protobuf.of(someObject);

// Serialize to OutputStream
try (OutputStream os = new FileOutputStream("data.pb")) {
    Protobuf.of(someObject, os);
}

Parsing

Protobuf binary is not self-describing, so the target type is always required on read.

// Parse protobuf binary into a bean
Person p = Protobuf.to(protobuf, Person.class);

// Parse from InputStream
try (InputStream is = new FileInputStream("data.pb")) {
    Person p = Protobuf.to(is, Person.class);
}

Java-to-Protobuf Scalar Mapping

Defaults are derived from the Java property type. All are overridable per-property via @Protobuf(type=...).

Java Type	Default Proto Type	Wire Type	Notes
boolean/Boolean	bool	VARINT
byte/short/int/Integer	int32	VARINT	Plain (not zigzag)
long/Long	int64	VARINT	Plain
float/Float	float	I32	IEEE 754, little-endian
double/Double	double	I64	IEEE 754, little-endian
String	string	LEN	UTF-8
byte[]	bytes	LEN	Native binary
enum	int32	VARINT	Ordinal by default
BigInteger/BigDecimal/char	string	LEN	Lossless string form (no native proto scalar)
Date/Calendar/Temporal/Duration/Period	string	LEN	ISO-8601 (lossless)

The following proto scalar types are available via @Protobuf(type=...) for protoc interop: SINT32/SINT64 (zigzag), FIXED32/FIXED64, SFIXED32/SFIXED64, UINT32/UINT64, and ENUM_STRING (encode enums by name instead of ordinal).

Container Encodings

These follow the proto3 wire conventions so they interoperate with real protoc-generated messages:

Java Container	Protobuf Encoding
Repeated numeric / bool / enum (int[], List<Integer>, ...)	Packed (single length-delimited block)
Repeated string / bytes / message	Repeated tagged entries (cannot be packed)
Nested bean	Length-delimited embedded message
Map<K,V>	Repeated entry { key=1; value=2 } sub-messages (proto map<> shape)

On parse, repeated scalar fields accept both the packed and the unpacked (repeated-tagged) forms for interop tolerance.

Field-Number Assignment

Protobuf wire bytes carry field numbers, not names, so every property is assigned a stable number:

1. Explicit numbers win. @Protobuf(fieldNumber=N) claims number N.
2. Auto-assignment fills the remaining properties sequentially starting at 1, ordered alphabetically by property name, skipping any explicitly-claimed numbers.
3. The reserved 19000–19999 band is always skipped.

public class Order {
    @Protobuf(fieldNumber=10)
    public String id;          // pinned to field 10

    @Protobuf(type=ProtobufScalarType.SINT32)
    public int delta;          // zigzag-encoded; auto field number

    public String customer;    // auto field number
}

Reordering properties in source does not change field numbers; renaming a property may shift its auto-assigned number (an explicit source change). For long-term external interop, pin numbers with @Protobuf(fieldNumber=...).

Presence / Null / Round-Trip Guarantees

• A field is emitted only if its value is non-null; an absent field leaves the property at its default/null on parse.
• Round-tripping is lossless for nullable properties and for all scalar/container shapes above.
• proto3 zero-value caveat: an unboxed primitive left at its zero value (e.g. int 0, boolean false) is emitted, but on the wire a zero value is indistinguishable from "unset". Use boxed types (Integer, Boolean) if you need to distinguish "zero" from "absent".
• uint64: a long-typed field carries the raw 64-bit pattern (surfaced via Long.toUnsignedString); a BigInteger-typed field carries the full unsigned magnitude losslessly.

Media Types

• Produces: application/protobuf
• Accepts: application/protobuf, application/x-protobuf

(No collision with the text-format marshaller, which uses text/protobuf.)

Known Limitations

• Parse requires the target type. There is no schema in the bytes, so parsing arbitrary unknown protobuf into a generic map is a non-goal.
• Unknown fields are skipped, not preserved. Unknown field numbers are skipped correctly by wire type (parsing never breaks) but are not retained on re-serialization.
• Deferred (assess-only): oneof, full proto2 presence/required/defaults, unknown-field retention, well-known types (Timestamp/Duration/Any/wrappers), and groups (deprecated) are not implemented in this iteration.

Annotations

@Protobuf controls how a bean property maps onto the wire format (classes, methods and fields):

Attribute	Description
fieldNumber	Explicit protobuf field number for the property (see Field-Number Assignment above).
type	Explicit scalar type (ProtobufScalarType); overrides the mapping derived from the Java type. AUTO is the default.
description	Optional description.

public class Person {
    @Protobuf(fieldNumber=1) public String name;
    @Protobuf(fieldNumber=2, type=ProtobufScalarType.SINT32) public int age;
}

@ProtobufConfig sets serializer/parser properties on a REST class or method (see Context Annotations):

Attribute	Description
addBeanTypes	Add _type properties to beans whose type can't be inferred through reflection (default false).
nativeTypes	Reserved opt-in for native-type encodings; currently a placeholder.
maxLength	Maximum wire-declared length (bytes) for length-delimited blocks when parsing.
rank	Ordering relative to other config annotations.

@ProtobufApply applies an @Protobuf to other classes/members via on()/onClass() (see Dynamically Applied Annotations).

See Also

• Protocol Buffers Encoding
• Prototext (Protobuf Text Format) Basics
• Protobuf marshaller
• org.apache.juneau.marshall.protobuf package