Industry

eCommerce

Annual Revenue

$1.4B

Use Cases

Load Testing
Regression Testing
API Observability

Overview

Get started today

Replay past traffic, gain confidence in optimizations, and elevate performance.

Speedscale Customers

Sephora cuts troubleshooting time down 2 weeks with Speedscale

Company

Sephora is a multinational cosmetics retailer and eCommerce brand. Today, the company has over 750 retail locations in 21 countries.

The initiative: Improving performance testing in CI/CD

To give their software developers more agility, Sephora’s engineering leadership decided to migrate their monolith eCommerce platform to a new microservices architecture. As part of this initiative, the team is improving and automating performance testing in their CI/CD pipeline to minimize manual work.

For load and regression testing, the team used open source tools like JMeter, but found it lacked key capabilities, like reporting. “You can’t track regression easily with JMeter. You have to write some custom reporting tool to compare the reports. We liked that Speedscale actually provides this out of the box. You can set goals or thresholds that you don’t want to go above, like 100 milliseconds for this transaction, so you can easily know if the test passed or failed. JMeter doesn’t provide anything like that,” said Diana Manulik, Senior Performance Engineer at Sephora.

They also evaluated Tricentis NeoLoad and Speedscale, but ultimately chose Speedscale for its feature set and ability to grow and evolve with their performance testing needs. Specifically, they liked Speedscale for its:

They also appreciated Speedscale from a cost savings angle: “Because Speedscale can run tests directly in our Kubernetes cluster, there’s no need to set up load generators and provision VMs. When the test is done, the pod is gone, so we’re not wasting any money by keeping the load generators running.”

Seeing payloads with Traffic Viewer

Today, the team is working to fully set up a new CI/CD pipeline and integrate Speedscale into it, but one key feature they found immediate value in is Speedscale’s Traffic Viewer. With the level of detail that Traffic Viewer provides, like which microservices are called, how many calls have been made, and even PII-redacted payloads, the team has been able to solve performance issues faster.

In one instance, the team was testing with incorrect payloads and seeing unexpected results in production. To troubleshoot the issue, they can check PII-redacted payloads in Speedscale.

“There is no other tool that we use right now that provides the capability of letting us see payloads for troubleshooting. Without Speedscale, we would have to create a patch with debug logging, which could take as long as one or two weeks. It’s extremely slow and can be a nightmare, so Speedscale helps us speed up the investigation quite a bit.”

 

Generating mocks faster

Today, Sephora is yet to transition from WireMock to Speedscale, but Diana says the process is much faster with Traffic Viewer. Without Speedscale, she says she would’ve had to rely on developers to enable logins and provide payloads, which they usually wouldn’t have.

“It takes a week. I'm not like exaggerating. It can take a week to get all this data, but with Speedscale, I have it and I can just script with WireMock, it takes like less than a day to complete.”

 

Transforming and adding data

After environments, good test data is often a source of headache and delays.  At Sephora, Diana used to have to edit JSON files to manipulate and transform data during testing for additional scenarios and fuzzing. This process was a “nightmare” but she now uses Speedscale’s traffic transforms, saying, “Now it’s very easy to upload a CSV file to replace some data using a UI.” This helps Sephora quickly add a variety of data permutations for increased code coverage.

Transforming and adding data

 

Up next: Running load tests in CI/CD and holiday readiness testing

Throughout the engagement, the Sephora team has worked hand-in-hand with Speedscale’s Customer Support to get their first set of test scenarios up and running.

“The help we get from the Speedscale team has been really great. Speedscale is the only vendor we’ve seen that offers such hands-on support through Slack. It’s been amazing.”

While the team is still early in their implementation, they currently use Speedscale to test performance of several services in their CI/CD pipeline, including address validation, loyalty, and tax APIs. Lately, they have been rolling out Speedscale’s new comparison feature to help them pinpoint how their app’s performance changes with each new build.

 

Over the course of the year, they plan to transition more of their service-level testing from JMeter onto Speedscale, and introduce Speedscale into more of their holiday readiness testing.