How to create a sandbox for the OP Bank API

NZ
Nikolay Zenovkin

Data Scientist

6th May 2020

Writing tests for software that integrates with external APIs can become a real mess. Some APIs don't provide a sandbox at all. Some have strong limits on free requests, which make the possibility to run tests on a CI pipeline doubtful.

Today I'll show how to recreate the OP Bank sandbox and enhance it using the HTTP Mocking Toolkit (HMT).

OP Bank provides a great API and a completely free sandbox for it. The problem is that if you run the official example, it will fail. Only accounts with a v3 endpoint are available in the sandbox. And a database of the /accounts endpoint differs from the one used for /payments endpoints. This situation is common because maintaining API sandboxes could be a time consuming and challenging task. That's one of the reasons why we created HMT.

HMT is a tool for building mocks and API sandboxes. It's often hard to achieve satisfying sandbox behavior from a mock using either OpenAPI specifications and random data generation or recordings of API calls. So we decided to mix them both. We also added some consciousness to the data generation algorithms and provided a set of tools to customize behavior with minimal effort.

Let's recreate OP Bank's API behavior using HMT. All it should take is executing several commands and writing about a dozen lines of code in Python.

All of the code in this article is available on our example repository.

Table of Contents#

Installing HMT#

HMT is written in Python and available as a PyPi package.

To install HMT via pip, run:

Note: HMT requires Python 3.6+

To make sure it's installed properly, run:

Making API recordings#

Since we don't have an existing product that integrates with the OP Bank API, we'll have to create a sample client first.

Let's use the following class:

The above OPBankClient class can obtain a list of accounts with the get_accounts function. It can also make payments in two steps. First, the init_payment function initializes a payment and then confirm_payment confirms it.

Now let's check the response from the real API executing the following code:

To run this code, you'll need a valid API_KEY and Oauth2 token from OP Bank. Both are available for free when registering as an OP Developer.

You may notice in the script output that the list of accounts doesn't contain the IBANs that we used for the payment. We can make payments between two fixed accounts but we can't check if a payment actually changed anything. It limits our ability to test software, but it doesn't stop us from making recordings.

There are many ways to obtain API recordings using HMT. It has connectors to Kong API Gateway, a tool to convert tcpdump logs and a standalone reverse proxy. We'll use the reverse proxy.

We believe that a reverse proxy is a more convenient way to make recordings than a proxy. Not every client in every language respects the HTTP_PROXY environment variable or similar - while API hosts are usually configured somewhere and can be changed without code modifications. But we'll support both in the future.

To make recordings using the reverse proxy, we need to first start HMT in a recording mode:

This should create a file in the generated logs directory called sandbox.apis.op-palvelut.fi-recordings.jsonl. This contains recordings in the http-types format.

Now we can use the obtained recordings to build an OpenAPI spec:

This should create a file in the generated specs directory called openapi.json, which contains our spec. The --mode argument tells HMT to create an OpenAPI spec in the mixed mode during recording. The mixed mode means that a mock may return either recorded or generated data - depending on whether or not it could find a match for a request.

Now we have to change the target host to a HMT proxy host. By default, this is http://localhost:8000 followed by the target host in a path string.

Let's change it in the opbank_rec.py script:

If we run the test script with the new URL, we'll get the same results.

So now we can restart HMT in mock mode:

When we run the script again, it should print the same results. But this time, it uses the mock instead of the OP Bank Sandbox.

Defining callbacks#

Now that we have a mock, we can enhance it even further using callbacks.

Let's change account identifiers to something else we have in the list of accounts. Then, we can convert our test client script into a test case:

If we run the above code, it will fail on both the mock and the official sandbox. We can't do anything with the sandbox, but we can modify our mocks.

HMT allows us to define custom callbacks that can flexibly modify the response data, while also having access to a request data and internal key-value storage. Internal key-value storage allows us to maintain a state across a sequence of requests. To clean it, we need to call DEL /admin/storage in the HMT admin server. This admin server is launched automatically with the mock server.

Callbacks are Python functions decorated by the @callback decorator. HMT loads them automatically on start from any *.py file located under the ./callbacks directory.

Since we have access to internal storage, we may write a set of callbacks that:

  • Store payment data upon payment initialization.
  • Update difference between initial and current account balance.
  • Modify account balance according to stored differences upon account request.

It may sound complicated, but we can do all of this in less than 20 lines of code in Python:

When we launch the test script again, all of the test cases should pass.

What's next?#

The mock we made covers the basic features of the OP Bank API. However, a lot of developers probably want to be able to mock validation errors. We're going to explain how to manage error responses with HMT in a future tutorial.

We would appreciate any questions as well as feature requests. Please, leave a comment or create an issue on GitHub if you have something to tell us. You may also follow us on Twitter to stay in touch. Any feedback means a lot for us since we're developing a new and emerging tool.

Newer postOlder post

Don’t miss the next post!

Absolutely no spam. Unsubscribe anytime.

Company

About usContactPricingT&C