gRPC Observability and Testing using the Speedscale CLI


gRPC is a remote procedure call framework that has been widely adopted among cloud native applications. It is an attractive choice for engineers needing more capabilities than what REST APIs provide. However, observability and testing tools for gRPC are few and far between. Because of this, triaging or understanding application behavior is potentially time-consuming and overly burdensome. In this post, I’ll demonstrate how you can use the Speedscale CLI to capture and even replay gRPC traffic to and from your application.

Getting Started

For today’s exercise we are going to be focusing on a trivial application I’ve written called “haiku-maker”. This service exposes a gRPC endpoint that will query a remote database and return a randomly generated haiku. You can use any database of your choosing, but in this demonstration I will use Firestore. The Go client library for Firestore uses gRPC to make API calls, which will allow us to view both inbound and outbound gRPC requests.

Let’s dive in! You’ll need the following available on your system in order to follow along:

  • go
  • git
  • docker
  • make
  • Speedscale CLI
  • A Firestore collection named “poems”

Begin by first cloning the example haiku service located at

					git clone
cd haiku-maker

The gRPC server implementation, its protobuf definition, and the generated Go protobuf code are all located in this repository. There is also a utility client you can use to interact with the server when it is running.

The server exposes two simple gRPC endpoints: AddLine and MakeHaiku. AddLine accepts a string of text and number indicating the syllable count (5 or 7) and adds this to a Firestore collection named “poems”. MakeHaiku queries this collection and generates a random haiku.

Build the haiku server by running `make`, using the environment variable `PROJECT_ID` to indicate the project to which your Firestore collection belongs:

					PROJECT_ID=my-project make image


This will create a docker image with the tag “haiku-maker”. When this is complete, we are ready to begin capturing gRPC traffic.


The Speedscale CLI uses a proxy to capture your application’s inbound and outbound traffic. You can run this proxy in a couple of ways – either a simple inline proxy or a fully transparent proxy (more information can be found in our documentation). Transparent proxy capture provides by far the simplest and fastest way to start observing and testing application traffic. However, your application must already containerized. Since the haiku server runs in a container, we will use this mode.

Start a new traffic snapshot using the CLI. The haiku service needs GCP key credentials for a service account in order to function properly. If you followed the Firestore quickstart linked above, you should have a JSON file containing these credentials. Ensure that this is mounted onto the service container with the `–mount` flag. Change the source location to the absolute path of your credentials file. Do not alter the target location.

					speedscale start capture -i haiku-maker -p 9999:9999 \
    --mount type=bind,source=/path/to/svcaccount-keys.json,target=/svcaccount-keys.json


You should see output similar to the following:

Speedscale stop capture - - Speedscale API Testing

Now let’s use the client utility included in the example repository to generate some gRPC traffic and add a few lines to our haiku collection:

					go run client/client.go -add-fives "It's not DNS"
go run client/client.go -add-sevens "There's no way it's DNS"
go run client/client.go -add-fives "It was DNS"


Now we can generate a haiku using data stored in firestore:

					go run client/client.go


At this point, we should have created multiple inbound and outbound gRPC requests. For gRPC observability and testing, we can quickly view this traffic by running `speedscale inspect`:

API Screenshot - Speedscale API Testing

From this traffic list, we can quickly see the incoming and outgoing gRPC requests that we made, as well as some high level summary information such as request time, location url, status code, and request duration.

If you prefer a much more broad view of this set of gRPC traffic, press the TAB key while on the traffic list. This will change the view to display a high level service map, shown below.

gRPC Observability and testing - Speedscale API Testing

The service map allows you to quickly see a rollup summary of all of the inbound and outbound transactions, counts, target endpoints, and what primary technology was observed. Switch back to the traffic list by pressing TAB again.

Let’s say you want to view more granular details for one specific transaction. By using the up and down arrow keys (or the j and k keys), select the transaction you are interested in and press ENTER. A tabbed panel will appear below the traffic list and will contain the full details of what was recorded. In the case of gRPC, the binary wire format of the protobuf is shown in an easy to understand JSON format – all without needing any generated client code or the original protobuf message definitions. To change sections, press the TAB key. To close the detail view, press ESC.

Request Body - Speedscale API Testing

“Copy as cURL” is a particularly useful and powerful feature for gRPC testing, found by selecting the `cURL` tab in the detail screen. Here, you can view a cURL command that, when executed, will re-issue the gRPC request as it was recorded. Press the ENTER key while on this screen to copy the command to your clipboard. Paste this into your terminal to replay the request.

eURL - Speedscale API Testing


As we’ve seen, using the Speedscale CLI can provide quick, easy, and invaluable gRPC observability and testing for your applications. This level of introspection can go a long way to empower engineers during development lifecycles, without needing to write any additional code or special tooling.

About The Author