Protobuf and Go

Generating Go code from Protobuf files is, to my surprise, not as straightforward as it is for other languages: properly configuring the go_package option is crucial to have working generated code and there’s surprisingly little information on the internet. Let’s see an example.

Problem #

We have the following two protobuf files (a complete, runnable example is available in this repository):

api/types/v1/messages.proto

syntax = "proto3";
option go_package = "github.com/fedragon/protobuf-go-example/types/v1;typesv1";
package types.v1;

// Change describes a change.
message Change {
    string value = 1;
}

api/semver/v1/messages.proto

syntax = "proto3";
option go_package = "github.com/fedragon/protobuf-go-example/semver/v1;semverv1";
package semver.v1;

import "types/v1/messages.proto";

// Version describes a semantic version.
message Version {
    types.v1.Change major = 1;
    types.v1.Change minor = 2;
    types.v1.Change patch = 3;
}

The second file uses a message type defined by the first one.

Note: For such a small example, there would be no need to split the definitions in multiple files. It is done so simply to explain the issue.

The go_package option plays a crucial role in getting things right here:

• it describes a full Go import path (github.com/.../v1)
• it contains an additional, explicit package name (semverv1) that will become the Go package of the generated files (this is not strictly required, but handy to avoid having multiple packages named v1)

Code generation #

In order to generate code from the above files, we have to run

$ protoc --go_out=paths=source_relative:. -I./api types/v1/messages.proto semver/v1/messages.proto

The -I option (short for --proto-path) sets the directory in which to search for imports: in my case, the api folder because that’s where I have decided to place my Protobuf files, following the Go standard project layout guidelines. You can of course also place them to another folder (including the project’s root folder). However please note that, if you are vendoring your dependencies, you have to make sure to exclude your vendor folder from the paths where protoc will look for Protobuf files, otherwise you might get obscure errors such as gogo.proto: file not found.

The --go_out=paths=source_relative:. option prevents the compiler from generating code using the full import path, so that we end up with:

/
|_ semver/
   |_ v1/
      |_ messages.pb.go

whose Go import path is

import "github.com/fedragon/protobuf-go-example/semver/v1

instead of

/
|_ github.com/
   |_ fedragon/
      |_ protobuf-go-example/
         |_ semver/
            |_ v1/
               |_ messages.pb.go

whose Go import path is

import "github.com/fedragon/protobuf-go-example/github.com/fedragon/protobuf-go-example/semver/v1"

which is less than ideal.

Running the command will generate Go files in the semver/v1 and types/v1 folders, respectively. If we open semver/v1/messages.pb.go, we can see that:

• the Go package name is semverv1, as specified in the go_package option
• the import to types/v1/messages.pb.go is correctly resolved

Many thanks to this Stackoverflow answer for getting me on the right track!

Package naming #

Making the version number part of the import path in the go_package option value (and of the short package name, as in semverv1) is not a strict requirement; it is a recommendation of the Uber 2 style guide, which one may choose to adopt or not.