Is optional field officially supported in proto3?

[mukuntharajaa]

Hello,

After going through documentation and little bit of source code, I cannot come to a conclusion whether "optional" keyword is officially supported or not. Release notes of 3.15 shows,

2021-02-05 version 3.15.0 (C++/Java/Python/PHP/Objective-C/C#/Ruby/JavaScript)

  Protocol Compiler
  * Optional fields for proto3 are enabled by default, and no longer require
    the --experimental_allow_proto3_optional flag.

Does this mean that its promoted to officially supported feature from experimental from 3.15 onwards ?

But latest code shows this, which is little confusing ( especially the comments part)

 uint64_t GetSupportedFeatures() const override {                                                                      
    // **We don't fully support this yet**, but this is needed to unblock the tests,                                        
    // and we will have full support before the experimental flag is removed.                                           
    return FEATURE_PROTO3_OPTIONAL;                                                                                     
  }

[fowles]

What language bindings did you find that comment in?

[mukuntharajaa]

cpp code,
src/google/protobuf/compiler/cpp/generator.h:91

[perezd]

It is officially supported, this is probably just a typo, asking @haberman to confirm.

[mcy]

We officially support proto3 optional in C++; this is a documentation bug.

[Sand2Silicon]

In the Proto3 language guide, all descriptions and documentation of keyword 'optional' have been removed and. all examples of 'optional' are in the context of Proto2 if any indication is given (that I can find).

https://developers.google.com/protocol-buffers/docs/proto3

• There is no documentation that optional is a valid keyword in Proto3
• There is no documentation for the behavior of optional in Proto3 that I can find. Additionally, is it the same as Proto2? How is it different?
• There is no clear documentation on if the keyword was removed from Proto3, or removed from certain versions and added back in others.
• There is no guidance on when or if I should use optional, or if it is discouraged, or if there is a recommended alternative method to solve the same problem.
• There are a few ancillary mentions; suggesting to me that optional might have existed and these instances should have been removed from the documentation but were missed, but aren't enough to clarify this is a language feature I can use.

Oddly, I feel like there was more documentation on 'optional' a few months ago, discussing how it changed from Proto2 to Proto3, and the alternatives to use instead,. I remember reading about it, and now I can't find what I read. I thought it was in the context of "how to tell if a field has been explicitly set or if was default initialized".

Could the documentation please be updated?

[Sand2Silicon]

Is this how Proto3 'optional' works?

Going a step beyond "is it supported", I've surmised in Proto3 all values are implicitly "optional". If no value is set, then it is default-initialized in all cases. This leaves the question "is this int score really exactly 0? Or was no score given?", "is this bool really false, or was no value set at all?", which could be a very different thing.

Based on a few references, my guess is that the entire purpose of optional in Proto3 is to tell ProtoC to generate the has_() method for a particular value, thereby allowing one to know if it was explicitly set by the program, or ignored and default-initialized.

According to C++ Reference Singular Embedded Message Fields this happens (the has_() methods are generated) automatically in the case of a Singular Embedded Message (aka. a value of a user defined type), but not in any other case (except for oneof). Presumably optional gets you the has_foo() method for all scalars and any other cases. Whether any of this is true, and the details of it are exactly what I was hoping to find on the Language Guide (proto3) page (in addition to the bullet points in my prior comment)!

Somewhat ironically, reading the Proto2 Optional Fields And Default Values the implication is that using optional creates this ambiguity problem with no solution given, and no mention of the has_() methods. Yet in Proto3 my understanding is that optional solves the ambiguity problem rather than creating it. Hopefully this can be discussed on the Proto3 page in the optional section when it is updated.

---

Additional references: StackOverflow How to define an optional field in protobuf 3

[haberman]

There is now proper documentation for optional in proto3: https://developers.google.com/protocol-buffers/docs/proto3#specifying_field_rules