Getting Started with gRPC: A Developer’s Guide

Introduction to gRPC

Within the realms of microservices and distributed systems, gRPC has emerged as a cornerstone technology. Its adoption by tech giants like Google, Netflix, and Square underscores its capability to facilitate high-performance, scalable inter-service communication. Built as a modern take on the traditional Remote Procedure Call (RPC) paradigm, gRPC enables services, potentially written in different languages, to communicate efficiently and reliably across networks. gRPC is particularly effective in distributed environments, especially those utilizing microservices architectures.

This article is appropriate for software engineers getting started with gRPC that want the high level points on how to inspect it reliably. This is not a comprehensive deep dive on load balancing, code ergonomics or other advanced topics (but let us know if you’d like to see more on that).

What is Remote Procedure Call

Remote Procedure Call (RPC) is a powerful communication protocol that allows one program to request a service from another program located on a different computer within a network. Essentially, RPC makes remote procedure calls appear as if they were local, simplifying the process of building distributed systems. This capability is crucial for enabling multiple requests to be performed concurrently across different systems.

In an RPC setup, the requesting program is known as the client, while the service-providing program is referred to as the server. The RPC mechanism acts as a low-level transport protocol, carrying data packets between these communicating programs. This client-server model is fundamental to the operation of RPC, allowing for seamless communication over a network. By abstracting the network’s details, RPC enables developers to focus on the logic of their applications rather than the intricacies of network communication.

RPC Protocol and Framework

The RPC protocol extends the concept of local procedure calls to remote systems, allowing procedures to be executed on different computers or shared networks. This extension is facilitated by the RPC framework, which provides a suite of services to manage the underlying network communications. These services include binding, establishing communication over the appropriate protocol, passing call data between the client and server, and handling any communication errors that may arise.

A key component of the RPC framework is the Interface Definition Language (IDL). IDL is used to describe the application programming interface (API) of a software component, serving as a bridge between machines that may be using different operating systems and programming languages. By defining the API in a language-agnostic manner, IDL ensures that the client and server can communicate effectively, regardless of their underlying technologies. This capability is essential for building robust and interoperable distributed systems.

gRPC 101: A Quick Primer on Remote Procedure Call

In theory, gRPC makes calling remote systems as simple as calling a local function. In practice, you should be aware of two key enabling technologies:

Protocol Buffers (Protobuf) – The Interface Definition Language Layer

• IDL and Message Format – Protobuf is gRPC’s default Interface Definition Language (IDL) and data format. Defines service contracts in .proto files. For REST users, this will feel similar to an OpenAPI or Swagger spec.
• Compact Binary Encoding – Fields use numeric tags instead of names for smaller message sizes, more efficient than JSON or XML.
• Code Generation – .proto files are compiled into strongly-typed client and server code (“stubs” and “skeletons”) in many languages, simplifying dev and ensuring consistent message structure.

HTTP/2 – The Transport Layer

• Multiplexed Connections – Supports multiple simultaneous requests/responses over one connection, unlike HTTP/1.1.
• Binary Framing & Compression – Uses binary format with compressed headers for low-overhead communication.
• Streaming Support – Supports four communication types:
  • Unary: one request, one response
  • Server Streaming: one request, many responses
  • Client Streaming: many requests, one response
  • Bidirectional Streaming: both sides stream messages independently

For a deeper dive on gRPC, check out https://grpc.io/. For the rest of this post we’ll talk about a specific implementation including protobuf encoded in gRPC and HTTP/2 with TLS enabled. We’ll just say “gRPC” to denote this stack as a shorthand since it is pretty common. RPC aficionados will note that gRPC can be run in a wide variety of permutations but my fingers get tired and I start falling asleep if I have to type that over and over.

Common gRPC Pitfalls

In no particular order, here are some of the problems that arise with gRPC:

• Binary Payloads – gRPC uses a binary wire format rather than JSON.
• Error Handling & Status Mapping – gRPC uses its own status codes (e.g. UNAVAILABLE, DEADLINE_EXCEEDED) rather than HTTP status codes. One common issue arises when a client makes an RPC request, which can lead to various challenges such as error handling and status mapping.
• Limited Browser Support
• HTTP/2 is not always supported in the enterprise
• Complexity Around Streaming – Check for resource leaks due to back-pressure, flow-control and connection lifecycle issues.
• Head‑of‑Line Blocking – By default, HTTP/2 multiplexes all RPCs over a single TCP connection.
• Versioning & Backwards Compatibility – Protobuf schemas require careful handling of field additions, removals, and renames. Accidentally reusing or changing tag numbers can silently corrupt messages or break clients without obvious errors.
• Load Balancing & Service Discovery – Out‑of‑the‑box gRPC clients typically do DNS-based round‑robin. Advanced patterns (circuit‑breaking, weighted routing, zone‑awareness) require integration with external systems (xDS, service meshes, or custom resolvers).
• TLS‑Only by Default – gRPC implementations generally assume end-to-end TLS. While this improves security, it prevents you from offloading TLS to an edge proxy or load‑balancer unless you introduce special configuration or sidecars.

For this guide, we will focus on the top two issues because they are more applicable to engineers building features. If you are a network engineer… or are a software engineer that enjoys out suffering… you may want to check out the Wireshark gRPC Dissector.

Capturing Requests and Responses

First things first, let’s get some visibility into what rpcs are being called. There are three common solutions for gathering this type of information:

1. Logging from within the app – The developer’s interface to gRPC is via auto-generated code. This puts you several layers above the network stack so perfect visibility can be a challenge. Remember that like your social media posts, higher level abstractions sometimes obscure details and stretch the truth.
2. Passive capture using Wireshark – Good choice but the data will be network-engineer oriented and will require that you provide TLS certificates manually (remember this is the default with gRPC).
3. Proxy-based capture and decoding with proxymock – Requires no manual configuration to decode TLS but not all languages honor proxy settings.
4. mitmproxy with protoscope – mitmproxy captures traffic while protoscope decodes payloads. This is similar to the proxymock approach but requires more effort.

The client transport layer is responsible for receiving messages from the server transport layer and passing them to the client stub after the server procedure has completed execution.

We’ll focus on using proxymock since it saves us from providing TLS certificates, modifying code or copy/pasting binary payloads.

Installing proxymock

For MacOS users, you can install proxymock using brew:

brew install speedscale/tap/proxymock
proxymock init

For users on Linux or Windows please visit the installation page.

Directing Traffic to proxymock

Run the following command to start the proxy listener:

proxymock record

Now we need to direct traffic to the proxy. Modern gRPC clients honor an environment variable called grpc_proxy for this purpose. Set these variables on the client so that outbound traffic is captured. If you’re wanting to run an app on your desktop and capture inbound traffic you can follow the proxymock docs.

export http_proxy=http://localhost:4140
export https_proxy=http://localhost:4140
export grpc_proxy=http://$(hostname):4140

http_proxy and https_proxy both redirect HTTP traffic to localhost. grpc_proxy works in the same way except for gRPC clients.

Some gRPC clients, like the golang variant, are optimized to skip the proxy when talking to localhost. For that reason you need to enter an actual hostname or IP instead of localhost.

At this point, the traffic sent to your client will be routed through proxymock and on to its original destination.

Analyzing a Request

Stop recording by pressing CTRL-C. Now you can view your data:

proxymock inspect

Navigate between requests using the up and down arrows. Press return to select a request. Use tab to move between data items on a request. After capturing the data, the process of unmarshalling data converts it back to its original format for analysis.

URL and Method

The URL is the fully-qualified RPC method name, e.g., /package.Service/Method.

In this example, the GetLatestLaunch rpc on the LaunchService is being called.

The HTTP Method for gRPC calls is always a POST. Don’t ask me why, it just likes being assertive.

Status Codes

The first thing to notice is that gRPC typically returns its status code as an HTTP Trailer. Most engineers have probably never seen a trailer actually used so now’s your chance. gRPC supports streaming and the final status code is not always known until the stream is complete. For that reason it is possible to receive an HTTP Status Code of zero even if a request fails. This choice means that you need to check the trailers instead of solely relying on the regular HTTP status code field. The final status code is in Grpc-Status in the response trailers.

These are the most common codes you are likely to see during development:

OK (0)

“We’re all fine here now, thank you. How are you?”  — Han Solo, Star Wars: A New Hope (1977)

INVALID_ARGUMENT (3)

The client sent a bad argument. Don’t confuse this with FAILED_PRECONDITION or other internal errors. gRPC status codes allow for the server to be stateful. That means that if you send a correct request at the wrong time you might get a FAILED_PRECONDITION instead of an INVALID_ARGUMENT.

DEADLINE_EXCEEDED (4)

Server-side timeout.

PERMISSION_DENIED (7) 

The gRPC status code 7, PERMISSION_DENIED, typically indicates that the client does not have sufficient authorization to perform the requested operation.  gRPC authentication can be surprisingly complex for those coming from REST. This can happen for several reasons, including missing or invalid authentication tokens (like OAuth2 access tokens or API keys), insufficient IAM permissions on the server-side, or misconfigured access control policies. In environments like Google Cloud or AWS, this often reflects role-based access issues where the client’s identity lacks the necessary scopes or roles. Another common cause is when intermediate proxies or API gateways strip or block required headers, resulting in the backend service treating the request as unauthorized. Importantly, PERMISSION_DENIED is distinct from UNAUTHENTICATED (code 16), as it implies that the identity is known but not authorized for the requested action. You can learn more about the main auth mechanisms outlined in the spec.

IMPORTANT: During development you are likely to see just a handful of status codes because you are in a controlled environemnt. However, in production you will see codes like RESOURCE_EXHAUSTED or DATA_LOSS that are unlikely to pop up during testing. Consider how you want your code to react in different failure modes, like when the server is too busy. Check out the Envoy docs to help you understand the concerns. Also, you can run stress tests if you want to try wearing a seat belt.

Headers and Trailers

HTTP headers are transcribed gRPC metadata. Here are some common examples:

Header	Purpose
Content-Type	Usually application/grpc (or application/grpc+proto, application/grpc+json, etc. for transcoding).
grpc-timeout	Optional. Indicates the timeout for the RPC, e.g., 1S for 1 second.
grpc-encoding	Optional. Specifies the compression algorithm (e.g., gzip).
grpc-accept-encoding	Lists supported compression formats.
te	Must be trailers. Required for streaming responses.

These are highlighted because they may not be intuitive for those used to a straight HTTP encoding.

HTTP Trailers typically include:

Header	Purpose
Grpc-Message	Status code explanation
Grpc-Status	Status code (supercedes the HTTP status code)

Request and Response Payloads

As stated before, gRPC clients typically encode traffic into the binary protobuf format. Unless you spend your evenings creating ANSI pictures of your cat, you’ll want a tool to make payloads human readable. Proxymock converts the binary wire encoding into a human readable JSON automatically:

gRPC payloads are nested sets of key/value pairs. You start at the top level field and then embedded fields become JSON arrays within.

Each key is the <field ID>:<field type>.

Here are the possible field types:

1. I32 – int32 or uint32
2. I64 – int64 or uint64
3. VARINT – Variable length integer
4. LEN – Everything else

This decoding takes place without access to the original protobuf definition. That is good in that it always works but it’s bad because you have to do a little bit of work to get the field names. If someone wrote a script to map the .proto onto this output automatically I would happily buy that person a sandwich. However, until then we have to do a tiny bit of mapping ourselves.

Let’s look at the corresponding sections of the .proto file:

In the proxymock screenshot we can see that:

• The top level field 1:LEN corresponds to the rockets field.
• Each element in the JSON array corresponds to a RocketSummary
• Within each array element we see field 1 and 2 that correspond to id and name within RocketSummary

If you capture binary payloads directly you can decode them in a similar fashion using protoscope.

Engineers love being helpful and that sometimes means posting protobufs publicly. If you're looking for a common service you might find it's protobuf definitions with a simple web search. For example, here is one for Google's Big Query service: https://github.com/googleapis/googleapis/blob/master/google/cloud/bigquery/storage/v1beta2/protobuf.proto

High-Performance RPC

High-Performance RPC is all about optimizing Remote Procedure Call protocols and frameworks to achieve low latency, high throughput, and efficient communication between distributed systems. In modern operating systems, where distributed applications and microservices are prevalent, high-performance RPC is critical for ensuring fast and reliable communication.

gRPC, a high-performance RPC framework, leverages Protocol Buffers (Protobuf) for data serialization and encoding. This approach reduces message size and enables faster transmission, making gRPC an ideal choice for high-performance applications. Additionally, gRPC supports bidirectional streaming, allowing real-time communication between the client and server. This feature is particularly useful for applications that require continuous data exchange, such as streaming services and Internet of Things (IoT) devices.

Security is another crucial aspect of high-performance RPC. gRPC provides built-in support for TLS-based security and pluggable authentication, ensuring that communication between distributed systems is secure. This combination of low latency, high throughput, and robust security makes high-performance RPC essential for various applications, including cloud-native APIs, microservices, and IoT devices.

Running a Curl

gRPC uses HTTP so we should be able to run a simple curl, right? Yes, but only in the sense that it’s possible to climb a mountain on a unicycle. Curl is capable of producing a valid gRPC request but it requires handling binary payloads which is not intuitive or straightforward. proxymock will take care of this for you in the cURL tab:

If you have access to the raw .proto files you should check out grpcurl. This will give you similar ergonomics to curl. Just keep in mind that you either need the .proto files or have server reflection turned on, which is uncommon in production.

Conclusion

Debugging gRPC presents unique challenges due to its binary payloads, custom status codes, and the complexities of HTTP/2. However, with the right tools and techniques, these challenges can be effectively managed. By utilizing tools like proxymock, you can gain valuable insights into gRPC traffic without extensive manual configuration or code modifications. Understanding how gRPC uses HTTP trailers for status codes and translating binary payloads into human-readable formats are crucial skills for any engineer working with gRPC. Whether it’s tracking requests, analyzing status codes, or inspecting payloads, the methods outlined in this guide provide a solid foundation for troubleshooting gRPC-based systems. Remember to explore the additional tools mentioned for deeper analysis and consider how your code handles various failure modes, especially in production environments. With these practices, you’ll be better equipped to navigate the complexities of gRPC and ensure your services communicate smoothly and reliably.

Client Stub Tools

These are the tools referenced in this article (or are just generally useful):

grpcurl – https://github.com/fullstorydev/grpcurl

gRPC-Web Developer Tools – https://chromewebstore.google.com/detail/grpc-web-developer-tools/cacoibopgjlodngfokahhkphgcohakai 

protoscope – https://github.com/protocolbuffers/protoscope

proxymock – https://proxymock.io/

WireShark gRPC Dissector – https://grpc.io/blog/wireshark/