Service virtualization vs. service mocking
Service mocking, API mocking, stubs, test doubles, fake APIs, service virtualization tools…the list goes on for testing tools with the purpose of mimicking real software components.
You’ve probably heard these terms and others to describe the concept of simulating backend systems for software testing. While it may seem like they can all be used interchangeably, there are actually some key differences that developers should be familiar with.
In this blog, we break down the differences, use cases, and key players in the space. Plus, we’ll cover why today’s software engineering teams are actually in need of a new solution better suited for cloud-native architectures.
Simulating services: A hierarchy
Just like bourbon is a type of whiskey (and not all whiskey is bourbon), virtual services are a type of service mock (but not all service mocks are a virtual service).
Many tenured folks in the space will argue the merits and use cases of a stubbing, service mocking, and service virtualization tool, but broadly speaking, they scale from simple to complex as shown in the diagram below:
When to use stubs, service mocks, and service virtualization
The degree of mimicking real software components depends on your test scenarios.
When to use stubs
A stub or test double is often used as simple, consistent returns to method calls, and are commonly used by an individual developer for unit testing (vs. shared with a team). Stubs typically return static test data and do not dynamically contain state. For example, let’s say one of your dependent services is an API for a stack, and there’s an endpoint to return whether the stack is empty. Although the real service’s data source is dynamic because there would be an actual stack running inside those 3rd party services, a stub would simply be hardcoded to return true or false if the stack is empty. In the case of special events like errors, error messages would also need to be manually hardcoded. Stubs are ideal if your testing environment only needs dummy data under specific conditions.
When to use service mocks
API mocking or service mocks usually happen over a network and have a more varied library of responses. But service mocks still have trouble keeping track of state beyond a simple unit test scenario. For example, eCommerce web services could have dependent components for the shopping cart. Real services for the shopping cart would have state management for events like adding or removing items from the cart. Mocking the dependent systems for the shopping cart, however, would struggle to track these events and their state. Typically, mocks do not include a virtual copy for data virtualization. When your test case is more demanding, you need virtual APIs for more complex workloads–like with particular performance testing or load testing cases.
When to use service virtualization
API service virtualization usually addresses this by “learning” as much behavior about specific software components by recording the network traffic. Service virtualization is more commonly used during functional tests, integration tests, and acceptance tests.
For a refresher on service virtualization principles, check out Part 1 of this blog series: What is Service Virtualization? An Introduction for Developers.
Key players
Stubs
Stubs are usually generated by hand in the language the application is written in (or whatever the software developer writing them is most comfortable with). Likewise, stubs are rarely leveraged beyond personal use, even within a dedicated testing team, as they require insider knowledge that is typically not well-documented.
Mocks
Service mocks can be reused by teams, but even a simple mock requires a high level of scripting and maintenance. Team members will need to continuously update the test suite to ensure the dependent services are being mocked to accurately simulate the real thing. Most service mocking and stubbing software is open source.
Virtualization
While API virtualization likewise requires scripting and maintenance, UI’s that allow for quick feedback and simple editing of virtualization work, as well as templates for reuse, are common.
Below, we’ve shared a roundup of some of the top vendors in the space.
Key Players | ||
Service Virtualization | Service Mocking | Stubs |
Broadcom Parasoft Tricentis Smartbear Microfocus Speedscale | MockServer Hoverfly Mountebank WireMock Traffic Parrot | JMock GoMock Mockito |
Case Study: Nylas
See how Nylas used 3rd party mocks to improve performance by 20x with Speedscale.
The need for more than service mocking
As application architecture moves from cloud VMs to containers, service mocks are often inadequate for a number of reasons:
- They are manually scripted and time-consuming to build
- Each request/response pair must be manually specified, and the matching logic is brittle
- The mocks are stateless and cannot track changes in a shopping cart/account balance
- Mocks are commonly run in an IDE as a process or server, requiring a code or config change to interface with the simulation
Microservice APIs are ubiquitous and the pace of change is high. Due to how tightly the services integrate, the data scenarios, messages and number of permutations are highly varied. The only way for service mocking or service virtualization to keep up is for the cost of building them to decrease exponentially.
The need for more than service virtualization, even!
Service virtualization also has a number of limitations that make it difficult to adapt the solution to cloud native architectures. The solution-space evolved during the service-oriented architecture (SOA) days, and many have not evolved since.
Limitations of service virtualization
- Code or config changes required for the recording process, to redirect the client API through a proxy listener, which then forwards the message onto the correct destination (see graphic below)
- All virtual services require editing to specify matching logic and broaden applicability
- Most UIs are tedious and mass editing of request/responses are difficult
- Services are hosted on a centralized server, again requiring code changes to talk to virtual services vs normal pods or containers in the cloud
- Server-based services cannot be deployed multiple times in multiple clusters in a cloud-native way
- Licensing models are not conducive to cloud usage
What’s next on the horizon: auto generated service virtualization with Speedscale
Future solutions will require virtual services to be cheap and quick to build. As the complexity of microservices increases, it becomes virtually impossible for humans to reason about. This is why platform engineering espouses the need for internal developer platforms (IDPs), and reduces cognitive load in day-to-day work.
Why Speedscale
Speedscale takes service virtualization a step further with production traffic replication. By recording traffic from your production environment, Speedscale can automatically detect your API dependencies and mimic the responses coming back from those external dependencies. Sensitive data is also automatically redacted so you don’t need to worry about personally-identifiable information (PII) being used, which is ideal for enterprise service virtualization. Plus, test data can be generated continuously and automatically from production traffic–so DevOps teams can integrate testing into a CI/CD pipeline to ensure high quality software.
Kubernetes Traffic Replay: How Speedscale Works
Record and replay real-world user traffic to simulate production conditions 5x faster–and build more resilient applications for less cost.
|
Capability |
Stubs |
Service Mocks |
Service Virtualization |
Speedscale |
|
Static responses |
X |
X |
X |
X |
|
Simulate web services |
X |
X |
X |
X |
|
Simulate message queues |
X |
X |
X |
X |
|
Dynamic responses |
|
X |
X |
X |
|
Statefulness |
|
|
X |
X |
|
Dynamic dates |
|
|
X |
X |
|
Dynamic auth tokens |
|
|
X |
X |
|
Proxy recording |
|
|
X |
X |
|
Route to real/mock option |
|
|
X |
X |
|
Auto update service option |
|
|
X |
X |
|
Attach CSV/DB for dataset |
|
|
X |
X |
|
Server hosted |
|
|
X |
X |
|
Container hosted |
|
|
|
X |
|
Transparent recording |
|
|
|
X |
|
Dependency mapping |
|
|
|
X |
|
Auto generation of services |
|
|
|
X |
|
Autoscaling to target TPS |
|
|
|
X |
|
One-click chaos responses |
|
|
|
X |
Summary
Both service virtualization and mocking provide valuable benefits but require a lot of initial effort and ongoing maintenance for development teams. Virtual services are feature-packed, and are constantly improving. Most service virtualization solutions are also offered by long-running vendors with professional services and customer support. Most service mocking solutions are open source with long lead times on updates.
If your development process releases on a fast cadence, with a high number of microservice APIs, you fit the ideal profile to benefit from these solutions. As your architecture moves to the cloud, however, existing service virtualization and mocking solutions don’t quite have the necessary ergonomics to deploy and use cloud-natively.
Next up: Top tools for service virtualization
In our next post in the series, we cover top tools for service virtualization, their advantages, limitations, and best use cases.