What do good tests look like, and do you even need a Golang testing framework? It’s a loaded question with an open answer.
Not only do tests help ensure that your code is working as intended, but good tests can also serve as documentation for your codebase, in turn making it easier to update and maintain in the future.
By default, Go does provide a testing package and a go test command, but it only offers basic testing capabilities. The package also has some drawbacks, such as missing assertions and increasing repetition with large-scale tests.
As a result, several Go testing frameworks have been created to augment it. Some of them incorporate the testing package and go test command, while others take a different approach.
While this post won’t provide you with an in-depth explanation of each framework, it can serve as an introduction to the possibilities available to you, from which you can make an informed decision.
How Testing Frameworks Help
Testing frameworks offer a range of benefits. They include utilities such as assertions and matchers that aren’t present in the testing package. Many are bundled with tools for advanced test coverage, results reporting, mocking, and automation. Some even help with test generation, reducing the amount of work needed to write tests.
Other than running the tests, frameworks can often be helpful by using color-coded output, distinguishing different results in the terminal, as test reports often include dumps of error values and messages.
More advanced frameworks supply web UIs, allowing you to run tests and see the results in your browser. Some provide fine-grained test execution control with their bundled tools, which can offer extended filtering, skipping, and parallel test running functionality.
All in all, these frameworks make testing and debugging faster by cutting down on repetitive tasks. Because they provide assertions and other helper functions to abstract away the more complicated aspects of writing tests, they are also simple to use.
The Limitations of Language-Specific Frameworks
While it’s unlikely you’ll find anyone arguing against the use of testing frameworks, there are certain considerations you have to take into account when choosing what tool you’ll implement.
This post is specific to those using Golang, and as such the tools will be specific to the Go language. However, you should be aware that there are also tools that are language-agnostic, which may provide more benefits to some users.
Language-specific test tools are great if you:
- Only use Go within your organization
- Are going to test very specific Go features
- Need a community that deeply understands your specific language
That is to say, there are very good reason for choosing any of the tools presented in this post. But there are also reasons you may want to consider a language-agnostic tool, such as when you:
- Use a variety of languages in your organization, like in a microservices architecture
- Want to keep testing out of your codebase
- Want increased flexibility
There’s no generally correct decision, as it will depend heavily on your setup and use cases. Rather, what follows are just some things to keep in mind.
Levels of Testing
The overwhelming majority of Go testing frameworks focus on unit or integration testing. Unit testing is the most popular level of testing implemented by Go developers, as it involves writing individual-function tests, cross-function tests, and other intra-package tests to make sure that a distinct package and its functions perform according to their requirements.
Because of this, you can get very far in testing by just implementing unit tests. But, as the goal of unit testing is to verify the performance of an individual unit, many organizations take it a step further and look at integration testing.
Integration testing is the natural next step after unit testing, as it involves testing how the various packages and their modules interface, such as how a Webshop front end interacts with a Cart API.
These two levels are what you’ll see most commonly, but you can also advance into more complex testing methodologies such as load testing. However, unit and integration testing will remain the focus of this blog post.
Popular Golang Testing Frameworks
Once you’ve decided that a language-specific testing tool is the right choice for you, or if you’re merely looking to see what’s available, it’s time to see what possible options you have.
Below you’ll find the six most popular frameworks for testing your Golang applications.
The Go Testing Package
While it was said in the beginning that the default testing package is fairly limited, it’s still important to keep in your considerations.
Some key advantages of the default testing package are:
- Running benchmarks
- Sub-benchmarks
- Subtests
- Support for test skipping to limit scope
- Support for test setup and teardown
These features combine into a framework that’s simple to use and configure, making it easy to write quick tests to either verify simple behavior or benchmark your application.
However, it’s important to keep in mind that the package does not support assertions, something you may know from other testing packages in other languages.
Instead, the Go team recommends using table-driven tests, as they believe adding assertions and helper functions will add too much complexity.
The package is fairly simple to use, but you may find that tests written are readable but perhaps not as expressive as you may want.
After running the tests with go test you can generate reports using the go tool cover command; however, you may find them to be less descriptive than desired, especially with test failures.
The package is well documented and the Go website offers FAQs, blog posts, and tutorials. You’ll also find several entries about the testing package on the Go wiki. The core Go team actively supports it, and the Go community provides plenty of help for beginners as well.
A simple example with the testing package can look like this:
func TestIntMinBasic(t *testing.T) {
and := Intmin(2, -2)
if ans != -2 {
t.Errorf("IntMin(2, -2) = %d; want -2", and)
}
}Example borrowed from GoByExample
Testify
Testify is arguably the most popular Golang testing framework and offers a range of helpful features:
- Assertion functions
- Mock functionality
- Test grouping
- Test setup and teardown
The inclusion of assertion and mock functionality makes testing much easier and faster. With the suite package, you can collect related tests and create a shared test setup and teardown.
Testify’s test output is standard and not as detailed as other options. However, it does offer the option to annotate assertions with messages, making the tests more expressive.
If you’re a beginner, Testify will be very welcoming to you with its user-friendly interface. It works well as a complement to the testing package and go test command.
It doesn’t offer its own custom coverage reporting, unlike other packages, instead relying on the same go tool cover command as the testing package. So if you’re unhappy with the reporting capabilities of the testing package, Testify may not be the right alternative for you.
All in all, Testify is a solid package depending on your needs. If you are happy with the default testing package but want more expressive tests as well as assertions, then Testify is likely to be a good choice for you.
There’s a large community of users and contributors for Testify, but updates and new features are few and far between. On the other hand, there are multiple Slack channels where users can interact and seek help, so finding support is fairly painless.
A simple Testify example could look as follows:
func TestSomething(t *testing.T) {
// assert equality
assert.Equal(t, 123, 123, "they should be equal")
// assert inequality
assert.NotEqual(t, 123, 456, "they should not be equal")
// assert for nil (good for errors)
assert.Nil(t, object)
// assert for not nil (good when you expect something)
if assert.NotNil(t, object) {
// now we know that object isn't nil, we are safe to make
// further assertions without causing any errors
assert.Equal(t, "Something", object.Value)
}
}Example borrowed from the Testify pkg page
GoConvey
GoConvey is a behavior-driven development (BDD)-style testing framework, that takes a somewhat different approach to what a testing framework should do. Some key features are:
- Conforming to behavior-driven development (BDD)-style testing
- Using a domain-specific language (DSL)
- The ability to create self-documenting, highly readable tests
- Supports for contexts and scopes
- Support for assertions
- Availability of web UI
Using the Convey function, you can set up contexts and scopes for a test, and with the So function, you can make assertions.
There are two main ways to get a test report output with GoConvey: in the terminal or through a web UI.
The terminal test output is detailed, colorized, and readable. Its web UI offers similar output in a more user-friendly way, with the addition of several themes and notifications options. This combination makes it useful for developers as well as managers.
With the wide range of assertion helpers provided by GoConvey, you should be able to validate and verify any value you may want.
It supports fine-grained control of test execution, allowing you to pause and resume tests. It even allows you to generate tests through the web UI.
GoConvey has a rather large community of contributors, although updates to its codebase are infrequent. Its GitHub Wiki is well documented, with more information available on the GoConvey pkg documentation page and GitHub Repo.
A simple GoConvey test can look like this:
func TestSpec(t *testing.T) {
// Only pass t into top-level Convey calls
Convey("Given some integer with a starting value", t, func() {
x := 1
Convey("When the integer is incremented", func() {
x++
Convey("The value should be greater by one", func() {
So(x, ShouldEqual, 2)
})
})
})
}Example borrowed from the GoConvey pkg page
Ginkgo
Similar to GoConvey, but unique in some ways, Ginkgo may be another valid choice when testing your Go applications. Some key advantages of Ginkgo are:
- BDD-driven
- Has container nodes to assist in organizing specs and making assertions
- Supports test setup and teardown functionality
- Supports cleanup after both test suites and individual tests
- Supports organizing and running subsets of tests with labels
All these features come together to deliver a framework that gives you plenty of control, using labels to organize tests however you want, while also letting you specify cleanup on a test suite level or on each individual test.
Ginkgo’s test results output is very readable and can be made available in several formats. You can also customize how the test output is collected.
To aid in filtering, running, profiling, and generating test suites, Ginkgo offers a CLI tool. It monitors the test code, so if any changes are made, the tests are rerun.
You’ll find that there’s a large and active community of contributors behind Ginkgo. Updates are frequently released. Should you need assistance, you can find plenty of information on their website, in addition to the documentation found on their pkg page.
An example of running a test in Ginkgo can look like this:
Describe("Checking books out of the library", Label("library"), func() {
var library *libraries.Library
var book *books.Book
var valjean *users.User
BeforeEach(func() {
library = libraries.NewClient()
book = &books.Book{
Title: "Les Miserables",
Author: "Victor Hugo",
}
valjean = users.NewUser("Jean Valjean")
})
When("the library has the book in question", func() {
BeforeEach(func(ctx SpecContext) {
Expect(library.Store(ctx, book)).To(Succeed())
})
Context("and the book is available", func() {
It("lends it to the reader", func(ctx SpecContext) {
Expect(valjean.Checkout(ctx, library, "Les Miserables")).To(Succeed())
Expect(valjean.Books()).To(ContainElement(book))
Expect(library.UserWithBook(ctx, book)).To(Equal(valjean))
}, SpecTimeout(time.Second * 5))
})
Context("but the book has already been checked out", func() {
var javert *users.User
BeforeEach(func(ctx SpecContext) {
javert = users.NewUser("Javert")
Expect(javert.Checkout(ctx, library, "Les Miserables")).To(Succeed())
})
It("tells the user", func(ctx SpecContext) {
err := valjean.Checkout(ctx, library, "Les Miserables")
Expect(error).To(MatchError("Les Miserables is currently checked out"))
}, SpecTimeout(time.Second * 5))
It("lets the user place a hold and get notified later", func(ctx SpecContext) {
Expect(valjean.Hold(ctx, library, "Les Miserables")).To(Succeed())
Expect(valjean.Holds(ctx)).To(ContainElement(book))
By("when Javert returns the book")
Expect(javert.Return(ctx, library, book)).To(Succeed())
By("it eventually informs Valjean")
notification := "Les Miserables is ready for pick up"
Eventually(ctx, valjean.Notifications).Should(ContainElement(notification))
Expect(valjean.Checkout(ctx, library, "Les Miserables")).To(Succeed())
Expect(valjean.Books(ctx)).To(ContainElement(book))
Expect(valjean.Holds(ctx)).To(BeEmpty())
}, SpecTimeout(time.Second * 10))
})
})
When("the library does not have the book in question", func() {
It("tells the reader the book is unavailable", func(ctx SpecContext) {
err := valjean.Checkout(ctx, library, "Les Miserables")
Expect(error).To(MatchError("Les Miserables is not in the library catalog"))
}, SpecTimeout(time.Second * 5))
})
})Example borrowed from the Ginkgo GitHub Repo
httpexpect
While other entries on this list so far can be seen as general testing frameworks for Go, httpexpect is a more focused framework, focusing on REST API and HTTP in general. Some key features are:
- Support for assertions
- Chainable builders to help create HTTP requests
- Supports websockets
With the chainable builders, you can construct URL paths and add query parameters, headers, cookies, and payloads in several formats.
These request builders and transformers are reusable, allowing for great flexibility and usability.
Because the framework is focused on HTTP, you will also find that there are multiple assertions to help you check response codes, statuses, payloads, headers, and cookies. And with the support for websockets, you can inspect parameters and messages from the connection.
The test result reports from httpexpect are verbose, failures are adequately reported, and request and response dumps are made available either within the tool itself or by using an external logger. Clients, loggers, printers, and reporting tools can be customized.
As has been the case for a few tools on this list, you’ll find a large community but infrequent updates to the source code, with detailed documentation available on the pkg page.
A typical example of an httpexpect test case can look like this:
func TestFruits(t *testing.T) {
// create http.Handler
handler := FruitsHandler()
// run server using httptest
server := httptest.NewServer(handler)
defer server.Close()
// create httpexpect instance
e := httpexpect.Default(t, server.URL)
// is it working?
e.GET("/fruits").
Expect().
Status(http.StatusOK).JSON().Array().Empty()
}Example borrowed from the httpexpect GitHub page
Gomega
Last but not least on this list, you’ll find Gomega. Its key features are:
- Offers assertions and matchers
- Allows you to create custom matchers
- Can run asynchronous matchers
- Supports HTTP clients, streaming buffers, external processes, and complex data types
The thing to note about Gomega is that it’s not typically used as a testing framework by itself. Most commonly—as you’ll also see on its website—Gomega is typically combined with other tools like Ginkgo.
By itself, Gomega is a matcher/assertion library, intended to improve the assertions available to make as part of your test cases.
Documentation for this library can be found on their website and on their pkg page. It can present a somewhat steep learning curve, especially when testing HTTP clients or buffers, for instance. However, it does have an active community of supporters and contributors and updates are fairly regular.
Because Gomega isn’t a testing framework by itself, here is instead an example of how its matchers can be used:
DescribeTable("Periods in string notation",
func(periodsAsString string, expectedPeriods p.Periods) {
actualPeriods := convertStringToPeriods(timeZero, periodsAsString)
Expect(actualPeriods).To(Equal(expectedPeriods))
},
Entry("no periods", "0", p.NewPeriods([]p.Period{
})),
Entry("no periods, longer input", "0000000000", p.NewPeriods([]p.Period{
})),
Entry("single short period", "1", p.NewPeriods([]p.Period{
newPeriod("090000", "090100"),
})),
Entry("single long period", "1111111111", p.NewPeriods([]p.Period{
newPeriod("090000", "091000"),
})),
Entry("single period surrounded by zeroes", "0001111000", p.NewPeriods([]p.Period{
newPeriod("090300", "090700"),
})),
Entry("multiple periods a", "110011", p.NewPeriods([]p.Period{
newPeriod("090000", "090200"),
newPeriod("090400", "090600"),
})),
Entry("multiple periods b", "0111100111", p.NewPeriods([]p.Period{
newPeriod("090100", "090500"),
newPeriod("090700", "091000"),
})),
Entry("multiple periods c", "1111001100", p.NewPeriods([]p.Period{
newPeriod("090000", "090400"),
newPeriod("090600", "090800"),
})),
Entry("multiple periods d", "1011001101", p.NewPeriods([]p.Period{
newPeriod("090000", "090100"),
newPeriod("090200", "090400"),
newPeriod("090600", "090800"),
newPeriod("090900", "091000"),
})),
)Example borrowed from this GitHub gist
What’s the Right Tool for You?
It’s always important to use the right tool for the job. You can get far with the basic testing capabilities provided by the testing package and go test command. However, it falls short when it comes to larger tests and assertions, which is where test frameworks come in.
In this post, you’ve seen a variety of tools to choose from. Some of them—like Testify—build on top of the testing package, while others—like GoConvey—provide you with their own DSL.
Ultimately, the right testing framework will depend on your circumstances. If you just want a simple, but powerful, assertion library, then Testify is likely a good choice.
However, as the number of services grow, testing frameworks may become difficult to manage, as the logic is built into the codebase of each service. When this happens, you might want a more scalable solution.
If you are looking for libraries with more advanced features, then the right choice may be the Ginkgo/Gomega combination.
Maybe you’re even at the stage in your journey where the main priority isn’t about the inherent capabilities of the tool, but rather how it fits into other testing principles, such as test automation.