APIs often represent the cutting edge of the technology space. This is especially true with Artificial Intelligence – as AI has evolved from speculative technology to mass adoption, it has shown up significantly in APIs as a modality and mechanism. However, as with all new technologies, using AI APIs comes with significant challenges. Connecting to AI models in a consumer-centric system often requires rethinking how you push your requests and data, adopting an internal model that more closely reflects third-party external services and integrations. Even when using local systems, this often requires considering the AI implementation of its API or service within a collection of more complex integrated requirements and considerations than a simple authentication server or traffic scaling implementation.
For this reason, software engineers have had to relearn a lot of development practices and rethink how they implement some core functions. One of the specific places this has shown itself readily is in building AI API mocks. Building a mock API is incredibly important, but it becomes quite a bit more complicated when considering the interplay of the AI system and the model that underpins it.
Today, we will look at the advantages of mocking AI APIs and the importance of building a mock, the general strategy that most developers will find helpful, and some specific tools and processes to make this process more effective and accurate.
The Importance of Mocking AI APIs
Mocking is an incredibly powerful and valuable part of the development lifecycle. In many ways, mocking can be one of the most useful tools in the developer toolset, opening up new methodologies for understanding and crafting your development at scale.
Mocking Demystifies the Process of Development
Firstly, mocking APIs demystifies the process of development quite significantly. Mocking an API – especially in the context of AI – allows you to simulate your API at scale in a controlled environment. This allows you to test different use cases and data flows without putting the actual system at risk, but it also allows you to test the constraints of the environment that might affect the utilization of the system. This can be especially important considering how resource-hungry AI is and how simple failures can make efficacy poorer.
Mocking Reduces Reliance on Third-Party Services During Development
Using mocks allows you to develop without having to worry about the lifetime and accessibility of your third-party services. This can have some significant benefits, but perhaps most importantly, it helps you build without the constraints of third-party systems reflecting in your code.
If you have a service, for instance, that fails somewhat randomly, you might write code that caches or resends requests or pauses on request to wait for a request to be completed. These constraints could be entirely artificial, and when building in a mock, you might find that your code doesn’t need these speed bumps, thereby creating a more effective system in totality.
Mocking Abstracts Unpredictable Behavior
Another huge benefit of adopting mocking for AI APIs is that it abstracts away the unpredictable behavior that can make using these systems difficult during development. For instance, AI systems can often hallucinate and might need systems such as Retrieval Augmented Generation to make their output more useful. These systems can fail for seemingly random reasons, making for complex operations.
All of this adds complexity, but at the end of the day, development doesn’t really care about the why as much as the state – if a call failed, there needs to be a process for that, and so abstracting away this unpredictable behavior actually makes it significantly easier to develop around it. In essence, it allows you to treat the unpredictability as an element of the AI itself and not of the product integrating it.
Mocking can massively reduce AI development costs
Lastly, leveraging mocks to simulate AI APIs can reduce integration costs by standing in for expensive responses in early phases of development. Running models and requesting prompt responses can use up unnecessary credits and run up quite the bill. By getting a few sample responses and generating mocks based on that, running the actual AI integration can wait until you’re ready to fine tune and do more sophisticated testing. Moreover AI responses can sometimes vary, and having consistent responses until you’re ready to toggle more variety can provide a stable development environment.
Overview of API Mocking Techniques
API mocking with respect to AI has a few specific considerations. First, let’s look at the traditional methodologies for mocking.
Stubs and Simulations
The classic approach to mocking is, of course, simple stubs and simulations. In this approach, the idea is to build your API and then mock those systems by using pre-configured stubs and scripts to deliver mock data as if the service is live. This can be useful, especially in cases where the data is static or the third-party service data is less important than the knowledge that the third-party service is actually there and returning a simple 200 code.
This process has been used at scale for many providers, but it carries some of its own drawbacks. There’s the obvious fact that the data is less important than the availability of the service – for APIs in which the opposite is true, where the status is less important but the real data is paramount, the need for having mocking that abstracts away the external system but still provides useful real-world data and realistic responses has put many providers and developers into a catch-22.
Additionally, there’s the reality that – even in a case where the freshness of the data itself is not as important – the nature of the response should mirror the real world. In some cases, you can do that – if you know your user will always hit the same endpoint, it’s quite easy to just mock that and call it good enough. When that’s not the case, however, and you need something more, you very quickly start to run into issues. This middle ground creates a scenario where your best mock might represent the most inaccurate form of your use case, making it highly useless for your specific scenario.
Modern Approaches
To resolve these fundamental issues, the modern approach to mocking has been to adopt a more dynamic response generation by simply adding more data to the potential pool. Speedscale has been at the forefront of this, allowing users to capture data for replay in the future. By capturing real user data, you can simulate real-world scenarios because you are replaying actual scenarios that have occurred in the real world.
Another huge improvement has been in the process of building mocks themselves. With services such as Speedscale, you can autogenerate mocks rather than hand code them, allowing for more accurate but cheaper mocking processes. This, in turn, can have a tack-on effect by reducing the overall cost of the mocking while improving the overall accuracy or alignment to real-world functionality that they require to be truly useful.
AI Moves Mocking to Automation
As more automated tooling has come into the fray, it has become more and more apparent that AI, in particular, will require more flexible tooling, more reflective of the real world, and ultimately more automated.
AI inverts a trend that has existed for a very long time in the mocking world. Mocking has always been a balance between the dynamic and the stable. Hand-created mocks return the same thing day in and day out, but real-world scenarios need something more dynamic. Dynamic services require a lot more freedom, but with enough data, you can accurately shift the dynamic into the stable.
For AI systems, however, this becomes more complicated – by their very nature, they are the most human-like you can get without becoming human, and as such, they are not just dynamic – they require dynamic systems that are flexible. For many, this seems to run afoul of mocking as a concept.
Introduction to Speedscale and proxymock
Speedscale, however, unlocks exactly this dynamism at scale. Let’s look at who Speedscale is, what their solution to the problem is, and how you can use it to build mock APIs.
What is Speedscale?
Speedscale is a leading tool in API performance testing and simulation, allowing development teams to build robust, reliable applications. By replicating real-world network conditions and load patterns, Speedscale’s tools empower organizations to identify bottlenecks and optimize API performance before deployment, to shift left while increasing accuracy and alignment to actual users, improving overall reliability, and unlocking comprehensive testing across a wide range of concerns like error handling, frontend development alignment, backend APIs and their integrations, and much more.
What is proxymock?
proxymock is Speedscale’s free desktop tool designed to simplify and enhance the process of API mocking on your local machine – especially for the complex world of AI-driven APIs. As AI APIs often exhibit dynamic and unpredictable behavior and heavily depend on external dependencies, proxymock offers a solution that allows developers to simulate these responses accurately, ensuring their applications are well-prepared for real-world interactions. In essence, proxymock seeks to mitigate the catch-22 that was noted above, allowing for stability and dynamism all at once.
Key Features of proxymock
proxymock is an excellent tool for simulating AI API behavior as well as serving as a mocking framework for API development in totality. Because APIs often have to develop their application code for a variety of different scenarios, it offers some pretty substantial flexibility and control. Let’s look at some of its key features.
Realistic Simulation of AI API Behavior
proxymock can closely mimic the intricate responses of AI systems, including varied data outputs, latency issues, and error scenarios. This realistic simulation ensures that developers can thoroughly test how their applications handle both typical and edge-case responses, allowing for greater testing flexibility as well as more resilient final builds.
Scalability for High-Volume Testing
proxymock is designed to handle large-scale scenarios. Accordingly, it supports high-volume testing by efficiently simulating thousands of concurrent API requests. This scalability is critical for ensuring that applications maintain performance requirements as they evolve.
This scalability doesn’t abstract away specific control, however – you can still target specific API responses and test edge cases for very direct testing purposes. For instance, if you need to test specific responses for a query that represents a quirk in your data model or even the specific query parameters for a new feature developed for an application’s frontend you can do so while continuing to do larger-scale testing using a holistic data set based around a larger body of collected HTTP requests and test data.
Seamless Integration with CI/CD Pipelines
proxymock was built with the intention of ensuring seamless integration across modern CI/CD development workflow processes, allowing teams to incorporate comprehensive API tests (including effective integration testing, load testing, and other advanced testing) into their automated pipelines. This means that any issues can be identified and resolved early in the development cycle, reducing downtime and ensuring consistent performance.
Flexibility to Mimic Various API Scenarios
Whether it’s simulating standard responses, network timeouts, or unexpected error conditions, proxymock offers extensive customization options that provide significant variability in testing and production environments. This flexibility enables developers to craft precise mock scenarios that reflect a wide range of potential real-world situations, leading to more robust and resilient applications.
Building AI API Mocks with proxymock
Now that you have an idea of the benefits of proxymock, let’s start using it. Starting up with proxymock is easy since the tool is designed with ease of installation in mind, allowing you to set up a robust mock API environment with minimal hassle.
The following section is adapted from the proxymock documentation Quickstart guide, which provides an excellent demonstration of building a mock API with proxymock centered around a simple IP testing service.
Installing proxymock
proxymock is designed with ease of installation in mind, ensuring you can set up a robust mock API environment with minimal hassle.
Installing proxymock is designed to be incredibly easy – you can simply install through homebrew as follows:
brew install speedscale/tap/proxymockAfter installation, you can initialize your instance and get some basic configuration prompts using the following command:
proxymock initCustomizing Your proxymock Setup
With proxymock in place, you will want to record some transactions from a app on your local desktop to define your mock API endpoints, simulate response behaviors, and even mimic network conditions.
To start recording, run the following command:
proxymock run
Note that this command will create a snapshot of recording data that can be modified. Note the snapshot ID in the printed output because you’ll want it for tracking and replay.
You can learn more about modifying data in the proxymock documentation, but you can set some key variables to mock your AI API system, including:
- Endpoint Definition:
Map specific API paths and methods (GET, POST, etc.) to desired responses. - Response Customization:
Configure response status codes, body content, and headers to mimic real API behavior. - Latency Simulation:
Use the delay parameter to introduce artificial latency, allowing you to test timeout and slow-response scenarios. - Error Simulation:
Simulate error conditions by setting HTTP error codes (e.g., 500) and custom error messages.
Using Proxymock to Build a Mock API
Once installed and customized, proxymock becomes a powerful tool to simulate the behavior of AI APIs. Here’s how to get your mock API up and running.
Launch proxymock
First, you need to start proxymock as a mock server. You can do so with the following command:
proxymock run --snapshot-id <id>
This will start a local mock server containing all the responses you recorded in the previous step. If you use the environment variables in the proxymock output you do not need to reset your endpoints.
Test Your Endpoints
With the mock server running, you can now send requests to your defined endpoints to get back your mock responses. Using proyxmock to capture previous API traffic to your AI endpoint, you can make a dynamic request that touches an internal endpoint that simply returns similar content – or even a basic response in status such as a 200 status.
At this point, you really have two options. You can mimic the expected behavior of the system, e.g., feeding back similar data that you have retrieved from your model in the past, or you can return basic codes to simulate a response success without passing the content itself. This will vary highly depending on your specific installation and use case, although offering content that is similar to the material captured during your traffic analysis will be most helpful for continued development.
Iterate and Expand
When working with something like an AI API, it’s important to remember that no one solution is going to be perfect, and the name of the game is iteration. To that end, while your first mock might be entirely focused on simply creating a mock service that responds somewhat accurately, you can spend more time and effort getting this mock to look as close to functional as possible over time.
You can do this by adopting a wide range of options. Firstly, you can look at adjusting parameters. Modifying delays and error codes in the configuration file allows you to simulate different network and API conditions, especially for scenarios where the AI model fails to return a result or is met with insufficient credits in a commercial tool.
You can also look to use templating systems to make for more dynamic response generation. This is especially important in an AI scenario, as those systems are highly dynamic and require a bit more flexibility than other mock systems.
Finally, you should test different integration pathways for the mocked system and the AI API itself. This will help to seamlessly embed proxymock within your CI/CD pipeline, ensuring that every code change is validated against realistic API scenarios.
Conclusion
Speedscale and proxymock are a dream come true for mocking AI APIs. By using a tool that unlocks dynamic power with stable endpoint mocking, you’re getting the best of both worlds and adopting an innovative development mindset.
What’s most important in this process is ensuring that you are connecting the right resources and processes to the end result – if your mock needs API dynamism, the last thing in the world you want is to have something that is resistant to change and is static at all times.
If you’re interested in using Speedscale, getting started is incredibly easy! If you’d like to look at proxymock as a solution, take a look at the documentation to see if it’s a good fit for your use case. Adopting innovative tools like these can be significantly beneficial to any development process and can make your development approach more dynamic and valuable!