Testing Golang with httptest

httptest—How It Works

Before looking at the test package, it’s important to first understand how the HTTP handler package itself works and how your HTTP requests are processed.

HTTP Handler Package

The standard library http package net/http has a client and a server interface. The server is basically a group of handlers. When you send a request to an endpoint or a server path, the handler intercepts this request, and based on the request, it returns a specific response.

Handler Interface

Below is a simple interface of a handler (http.Handler):

type Handler interface {

   ServeHTTP(ResponseWriter, *Request)

}

The ServeHTTP takes in ResponseWriter and Request. The Request object holds the incoming HTTP request from the client, and the ResponseWriter can be used to create a response.

Example Handler

Here is an example of a simple handler:

// With no ServeHTTP

func handler(w http.ResponseWriter, r *http.Request) {

   w.Write([]byte("Hello, World!"))

}

With ServeHTTP

type home struct {}

func (h *home) ServeHTTP(w http.ResponseWriter, r *http.Request) {

   w.Write([]byte("Hello, World!"))

}

The first method is more common than the second. It’s less confusing and clearer to declare your handler as a function.

When the handler takes a HTTP request, you can compose a HTTP response after carrying out whatever needs to be done using the ResponseWriter interface. This process can either be reading from the database, third-party services, static server data, or processing a file. In the above code, the line w.Write([]byte(“Hello, World!”)) sends the response.

Test Golang HTTP Handlers with httptest

Even though the handlers are just functions, you can’t write unit tests for them the usual way. The hindrance comes from the parameters of the handler function with types ResponseWriter and Request. These are constructed automatically by the http library when a request is received by the server.

So how do you construct a ResponseWriter and Request object to test the handler? This is where httptest comes in.

httptest has two methods: NewRequest and NewRecorder, which help simplify the process of running tests against your handlers. NewRequest mocks a request that would be used to serve your handler. NewRecorder is a drop-in replacement for ResponseWriter and is used to process and compare the HTTP response with the expected output.

Here’s a simple example of the httptest methods, an instruction to print the HTTP response status code:

import (

   "fmt"

   "net/http"

   "net/http/httptest"

)

func handler(w http.ResponseWriter, r *http.Request) {

   w.Write([]byte("Hello, Worldn"))

}

func main() {

   req := httptest.NewRequest("GET", "http://google.com", nil)

   w := httptest.NewRecorder()

   handler(w, req)

   resp := w.Result()

   fmt.Println(resp.StatusCode)

}

Line by line breakdown

The line below allows you to create a new mock request for your server. You are passing it to your handler as your request object. Details like the request method or query parameters are specified here:

req := httptest.NewRequest("GET", "http://google.com", nil)

The next line is your ResponseWriter interface and records all the responses from the handler:

w := httptest.NewRecorder()

You can now use these variables to call the handler function:

handler(w, req)

Once the request has been fulfilled, these lines let you see the results and the HTTP response details of the request:

resp := w.Result()

fmt.Println(resp.StatusCode)

Full Example

Let’s now build a complete example. First, create a new Go project and create a file named server.go:

package main

import (
   "fmt"
   "log"
   "net/http"
   "net/url"
)

func RequestHandler(w http.ResponseWriter, r *http.Request) {
   query, err := url.ParseQuery(r.URL.RawQuery)
   if err != nil {     w.WriteHeader(http.StatusBadRequest)
       fmt.Fprintf(w, "Bad request")
       return
   }
   name := query.Get("name")
   if len(name) == 0 {     w.WriteHeader(http.StatusBadRequest)
       fmt.Fprintf(w, "You must supply a name")
       return
   }
   w.WriteHeader(http.StatusOK)
   fmt.Fprintf(w, "Hello %s", name)
}
func main() {
   http.HandleFunc("/greet", RequestHandler)  
log.Fatal(http.ListenAndServe(":3030", nil))
}

The above code creates a handler that returns a greetings message. The name query parameter is used to create a greeting, which is returned as a response. Run the code and send a request to http://localhost:3030/greet with a name parameter, for example, http://localhost:3030/greet?name=john to see the response.

Testing our example

To test this server, create a file server_test.go and add the following code:

package main

import (
   "io/ioutil"
   "net/http"
   "net/http/httptest"
   "testing"
)

func TestRequestHandler(t *testing.T) {

       expected := "Hello john"

   req := httptest.NewRequest(http.MethodGet, "/greet?name=john", nil)

   w := httptest.NewRecorder()

   RequestHandler(w, req)

   res := w.Result()

   defer res.Body.Close()

   data, err := ioutil.ReadAll(res.Body)

   if err != nil {

       t.Errorf("Error: %v", err)

   }

   if string(data) != expected {

       t.Errorf("Expected Hello john but got %v", string(data))

   }

}

The expected variable holds the expected response of the server. The NewRequest method creates a mock request to /greet with a name parameter. The handler then responds with the appropriate data, which is then validated against the expected value. Run the test with go test, and you should see it pass.

Test Golang HTTP Clients with httptest

Another important use case of httptest is to test HTTP clients. Whereas HTTP servers intake requests and churn out a response, HTTP clients sit on the other end, making requests to a HTTP server and accepting responses from it. Testing client code is trickier since they depend on an external HTTP server. Imagine a scenario where your client makes a request to a third-party service, and you wish to test your client against all types of responses returned by the third-party service; however, it’s not in your control to decide how the third-party service will respond. This is where the NewServer function of httptest comes into play.

Testing clients with NewServer

The NewServer method creates a new server that mocks the response you want. You can use it to mimic the response of a third-party system for end-to-end HTTP tests. Let’s see an example in action.

Installing dependencies

Create a new Go project and install the required dependencies:

mkdir httptest-client && cd httptest-client

go init example/user/httptest

go get github.com/pkg/errors

Create a file called client.go. Here, you’ll define the client:

package main

import (
   "io/ioutil"
   "net/http"
   "fmt"
   "github.com/pkg/errors"
)

type Client struct {
   url string
}

func NewClient(url string) Client {
   return Client{url}
}

func (c Client) MakeRequest() (string, error) {
   res, err := http.Get(c.url + "/users")
   if err != nil {
       return "", errors.Wrap(err, "An error occured while making the request")

   }

   defer res.Body.Close()

   out, err := ioutil.ReadAll(res.Body)

   if err != nil {

       return "", errors.Wrap(err, "An error occured when reading the response")

   }

   return string(out), nil

}

func main() {

       client := NewClient("https://gorest.co.in/public/v2/")

       resp, err := client.MakeRequest()
       if err != nil {
               panic(err)
       }
       fmt.Println(resp)
}

The client simply makes an API call to https://gorest.co.in/public/v2/users, which returns some data. The response is then printed to the console.

To test the client, create a file called client_test.go:

package main

import (
   "fmt"
   "net/http"
   "net/http/httptest"
   "strings"
   "testing"
)

func TestClientUpperCase(t *testing.T) {

   // Dummy JSON response
   expected := "{'data': 'dummy'}"
   svr := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
       fmt.Fprintf(w, expected)
   }))
   defer svr.Close()
   c := NewClient(svr.URL)
   res, err := c.MakeRequest()
   if err != nil {
       t.Errorf("expected err to be nil got %v", err)
   }
   res = strings.TrimSpace(res)
   if res != expected {
       t.Errorf("expected res to be %s got %s", expected, res)
   }
}

Explanation

Here, the expected variable holds the expected result that the client must return. In this case, the client returns whatever it gets from the new server intact, but you might have some processing done before returning the data.

The next line creates the mock server with a handler that returns the expected result:

svr := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {

       fmt.Fprintf(w, expected)

}))

You can change the HTTP response value here to test the client against different responses. Remember to change the expected value as well.

Finally, a client is constructed against this mock server, and the request is made:

c := NewClient(svr.URL)

res, err := c.MakeRequest()

Run the test with go test to see the results.

Conclusion

Unit tests are crucial for validating the proper working of any application. Testing HTTP servers or clients is tricky because of the dependence on external services. To test a web server, you need a client and vice versa. The httptest package solves this problem by mocking requests and responses. The lightweight and fast nature of httptest makes it an easy way to test Golang code with end-to-end HTTP tests. This article showed you how to use httptest to test your API handlers and HTTP clients.