Postman is highly popular in the testing tools space for verifying API requests. While its use for general API testing is widely adopted, conducting load testing with Postman is not as straightforward.
In this post, we assume that you have some experience working with Postman and are familiar with the fundamentals of creating and sending requests. If you’re new to Postman, there are numerous resources available in the Postman Learning Center.
Introduction to Load and Performance Testing
Load testing is a critical component of software development that helps ensure the stability, reliability, and availability of an application or system. It involves simulating a large number of users accessing an application to measure its performance under heavy load. This type of performance testing is essential for big companies with platforms that support tons of users, as it helps to validate the performance of systems before a seasonal surge in traffic or unexpected viral success.
In real-world scenarios, load testing can be done to ensure the stability, reliability, and availability of a platform. It’s a crucial step in the development lifecycle to catch issues and errors early in the development cycle. By identifying potential issues before they become major problems, load testing helps to optimize a site or app for peak traffic days. It’s a proactive approach to ensure that a system can handle unexpected traffic or viral success.
Getting Started with Postman
Postman is a popular tool for API testing and development, and it also offers load-testing capabilities. To get started with load testing in Postman, create a Postman collection and add requests to it. You can so by clicking the + icon under the “Collections” tab and add your required requests.
Once you have your collection ready, use the Postman Collection Runner to run the collection and simulate a large number of virtual users accessing your API. To configure a load test in the Postman collection itself, you need to specify the load settings, such as the number of virtual users, test duration, and load profile. These settings help you define how the load and performance test will be executed.
During the load test, use the Performance tab to observe performance metrics in real-time, such as average response time, throughput, and error rate. By analyzing these metrics, you can identify bottlenecks and areas for improvement in the throughput of your API. You can run multiple collections in parallel in The Collection Runner, which is useful for simulating a large number of users accessing your API.
Overall, Postman is a powerful tool for load testing and other API performance during development, offering a range of features and capabilities to help you ensure the stability, performance, reliability, and availability of your application or system. By leveraging Postman’s load testing features, you can test your API’s performance under different loads and identify how many requests your API can handle per second.
Postman’s Load Testing Features
As a tool for performance testing APIs, Postman is designed to handle any type of API you might be working with, currently supporting three major protocols: REST, gRPC, and GraphQL. Additionally, it also supports SOAP for those who continue to use this protocol.
The most common use case for Postman is to simply test or verify individual API calls or requests. Postman Collections, in particular, is one of the optimal tools for testing API requests during development, as postman Collections allow you to group requests together, such as all possible requests for a given API.
However, Postman was initially only meant to test an API request one at a time. While this test scenario is still valid and useful, the team at Postman realized that many users were interested in a different use case: using Postman for performance testing. Because of this, some key features have been introduced, including Postman’s collection runner, which allows users to execute API requests in bulk and perform load testing. This tool helps monitor response times and detect bottlenecks during the testing process, enhancing the ability to verify both performance and resilience.
Generating Requests Sequentially
Load testing is an entire topic in itself. However, there is a fundamental feature common to all load-testing tools: the capacity to generate multiple requests simultaneously. To effectively perform a load test, it’s necessary to execute all tests concurrently to generate a substantial load on your API.
As you start working with load testing, you’ll find that the configurations can get very advanced and complex. However, without being able to generate a given number of requests at a time, it’s impossible to perform a load test.
Postman has implemented this functionality through the Postman Collection Runner, which enables you to execute requests from a collection as multiple users in parallel and in a specified order.
Creating Mock Servers
The use of mock servers can greatly enhance your load-testing experience. In fact, mock APIs can enhance your development experience in general.
Combining load testing and mocks is the best way to ensure that you’re focusing the load test on a single service. Imagine managing an e-commerce site that uses a microservice architecture, where one of the microservices is dedicated to the cart service.
This service will, of course, need to handle payments, and in this example scenario, your service uses Stripe. Although Stripe provides a sandbox API specifically for testing, you may still encounter rate-limiting issues, which are very likely to occur during a load test.
Later in this blog post, in the section about implementation, you’ll see how mocks are actually created in Postman.
Integrating with CI/CD
Being able to run a load and performance test locally is a great first step, and allows teams to verify the resilience of a service at a moment’s notice. However, efficient teams may consider integrating load tests in their CI/CD pipelines directly, ensuring with every commit that the service still performs as expected.
Postman provides two ways of accomplishing this. They’ve developed integrations directly for a number of CI/CD providers. If your CI/CD provider isn’t on this list, it’s also possible to get started by using the Newman library.
Limitations of Using Postman for Load and Performance testing
At this point, it should be apparent that performing load tests with Postman is by all means possible. However, there are certain limitations you need to consider before going through the effort of using Postman.
Checking response times
It can be argued that Postman doesn’t support “true” load testing, as it waits to check the response time for a request before sending the next one. This can result in insufficient stress tests, as the requests are being sent in isolated instances..
Manually managing mocks
While Postman does support mocking functionality, it’s far from an automated experience. In essence, Postman will generate a URL that will return whatever response you’ve configured.
Using our previous load test scenario of a cart service, you’ll have to configure the service (either directly or via environment variables) to use the new URL when contacting the Stripe API. It’s much better than not using mocks, but it does add some cognitive load to your load tests.
Combine this with the fact that mocks often have to be updated, especially if you’re mocking internal APIs, it can quickly become quite the task to manage.
No public cloud option
Perhaps the biggest limitation of Postman is the lack of a cloud option. With Newman, you can run it in your private cloud or internal data centers.
For one-off tests, this may not be the biggest issue, but when you want to integrate with CI/CD pipelines, this is additional infrastructure you will have to consider.
How to load test using Postman
Whether you’ve made the decision to move forward with Postman load testing or just curious to see how it works, here’s an overview of what a simple load test in Postman can look like.
To showcase this functionality, you’ll be using the RandomUser API, which is a free, open source API that can be used to get random test data. By utilizing Postman’s Collection Runner, you can execute API requests and perform load testing. The Postman Collection Runner enables you to set up and execute performance tests, monitor response times, and detect bottlenecks during the load-testing process.
1. Create and save requests to a Collection
First, you need to open up Postman and create a collection. Do this by clicking the + icon under the “Collections” tab.
Rename the collection to “Postman Load Test Tutorial” by clicking the pen to the right of the collection name.
Create a new HTTP Request by clicking the + button to the right of the Collection tab.
Now enter the URL https://randomuser.me/api/?gender=female&nat=US and click “Send” to execute the request.
At this point, you should see that the request executes properly, and you’re ready to save it to the collection. Do this by clicking “Save.”
In the modal window that pops up, name the request “Female US,” click on “Postman Load Test Tutorial” to select the collection, and, finally, click “Save.”
Let’s quickly add another request to the collection. Do this by changing the “gender” parameter to “Male,” clicking the arrow next to the “Save” button, then clicking on the “Save As…” button. After this, change the request name to “Male US” and click “Save.”
2. Verifying response
Verifying request responses in Postman is done by writing tests in JavaScript. To implement these tests, go to the “Tests” tab on the request and enter the following code snippet:
var data = JSON.parse(responseBody);
var user = data.results[0];
tests["A single user was returned"] = data.results.length ===1;
// Gender Tests
tests["Gender is male"] = user.gender ==="male";
// Nationality Tests
tests["The user is from the United States"] = user.nat === "US";The above snippet verifies that one user was returned, that they’re male, and that they’re from the US. After sending the request again, you should see in the “Test Results” tab at the bottom that all tests are passing.
3. Use the Postman collection runner to run a collection
At this point, you have a collection with a few requests and tests added to at least one request. Now it’s time to implement the actual point of this post: load testing.
To execute requests from a collection, you will first have to open the collection in the Postman UI. Do this by double-clicking the collection in the left-hand menu.
With the collection view open, click the “Run” button in the upper right-hand corner.
On the following page, choose from various run configurations, such as request selection, the number of iterations, and the manner in which it should be executed. For this first postman collections however, leave all options to default and click “Run Postman Load Test Tutorial.”
Given the low number of virtual users and duration of requests, the run will complete swiftly. Postman will then display the requests, results, and statistics, such as duration and average response times.
4. Configure a mock server
To configure a mock server with Postman, you first need to add an example to the requests saved in a Collection. To do so with the “Female US” request, select and execute it, ensuring that it shows a response.
Now that the response body is filled out, you can save the request as an example.
Next, click “Mock collection.”
Give the mock server a name and click “Create Mock Server.”
You should now be presented with a screen where you can copy the mock server’s URL.
Send a request to <mock-server-url>/api and see how it responds with the example response you’ve just saved in action.
You might notice that this response is being returned even though no parameters have been added. Postman uses a matching algorithm to determine which response is most suitable for a given request.
If you go back and add an example for the “Male US” request as well, you’ll notice that you are able to get different responses by changing the “gender” parameter. It is important to be aware of this, in case you’re expecting your mock to return an error code in case query parameters are misconfigured.
Mitigating Postman’s limitations
As you can see, getting Postman set up to send requests sequentially is fairly straightforward. However, it’s important to keep the previously mentioned limitations in mind, the biggest limitation being how Postman isn’t capable of performing “true” load tests.
You can see this more clearly in practice by adding a pre-request script to the “Female US” request then rerunning the collection.
Because Postman is waiting for one request to succeed before moving on, you’re unlikely to push your service to its limits and properly stress test it.
Load testing with Speedscale
Due to this limitation, it might be worth looking into what other testing tools are available. If you run your services in Kubernetes, it might be worth checking out how to load test with Speedscale.
Speedscale takes care of:
- Recording production traffic
- Replaying traffic
- Automatically creating mocks
- Generating many simultaneous requests
- Easy CI/CD integration
These key features in Speedscale allow you to push your services right to the edge, giving you confidence that your services will perform as expected in production. One Speedscale customer, Cimpress, not only got better insight, but reduced their load testing time from 4 weeks to 4 days.
Try Speedscale for free for 30 days or Schedule a Demo to see if our load testing solution is right for your organization’s needs.