Finding and fixing bugs in your tests with fast-check

Carolyn Stransky

Software Engineer

27th Jul 2020

Recently, we published Property-based testing for JavaScript developers. In it, we introduced an issue to show the limitations of example-based testing. Throughout the rest of the guide, we provided examples of how to use fast-check and other property-based testing principles.

But we didn't offer a fix to that initial testing bug.

We've been told that this is unsatisfying, so consider this post our redemption.

πŸ’» The tests featured in this post are available in the corresponding GitHub repository.

Summary of the bug in our test#

Imagine you have an input where users write in a price - but this input is type="text" rather than type="number". So you need to create a function (getNumber) that converts the input string into a number and an accompanying test:

This passes πŸŽ‰

Now imagine your website also operates in Germany where the meaning of commas and decimals in numbers are switched (i.e. $1,200.99 in English would be $1.200,99 in German).

So you add another test case to address this:

But when you run the test, you hit an error:

Assuming that your website and input are equipped to handle various locales and localized strings, this would be a limitation of your tests - not a bug in your app.

There's something that can help though: Property-based testing.

Fixing this bug with fast-check#

To solve this problem, let's use fast-check, a JavaScript framework for generative test cases. Our goal is to test that our function turns any valid input string into a number, regardless of locale.

First, we need to pick the property (or properties) to test for in this situation. There a couple of options:

  • It works with strings that aren't formatted.
  • It works with strings that are formatted using one of the accepted locales (in this case, that would be German).
  • A combination of the two.

We'll tackle the latter: A combination of formatted and non-formatted strings.

Spoiler alert: Final solution#

Here's a peek at what our test will look like by the end of this post:

This is a lot all at once. So let's go through it step-by-step.

Setting up our test#

To begin, we need to set up our test:

Applying our properties#

To test a combination of formatted and non-formatted strings, we need to pass two different arbitraries to property.

The first will be float to generate the floating-point numbers:

The second is also a combination of sorts. To start, we need to write a constantFrom function that takes in multiple, equally probable values. We'll use this for our German locale: de-DE.

Then, we'll wrap constantFrom in the option function. With option, the complete arbitrary will return either null (a stand-in for the English default) or our values from constantFrom.

Prices in these locales only go to the second decimal place and we want to embody this in our test.

To do this, we'll take the testFloat value and use JavaScript's built-in toFixed method on it. The 2 value indicates how many decimal places we want.

This method returns a string, but we need a number for our final assertion. So we'll wrap it in another built-in function, parseFloat, and name this variable fixedFloat.

Finally, we'll pass two arguments to the property callback: testFloat (representing each generated floating-point number) and locale (representing the generated locale from our option and constantFrom combination).

Altogether, our property function will look like this so far:

Processing the strings#

Because we're testing both formatted and non-formatted strings, we need a way to differentiate these two string types.

We'll create a variable called floatString that checks if there is a locale value.

If locale !== null, then we need to create a localized string based on the generated float.

To do this, we'll use the Intl.NumberFormat constructor and pass in our locale value. On top, we'll chain on the format method with our fixedFloat value from earlier.

If locale is null, then we'll use the built-in toString method on fixedFloat, which returns it as a string.

The resulting floatString variable:

Adding your assertion#

At last, we need the most crucial part of any test: The assertion.

The following expect statement indicates that when you pass the floatString (formatted or not) to the getNumber function, you expect it to equal the fixedFloat number value:

Putting it all together#

After all that, you're back to that final result:

Now your tests are more thorough and resilient πŸŽ‰

Special thanks#

Nicolas Dubien, the creator of fast-check, was the first to flag this issue in the original guide. He also helped develop the solution. You're the best, Nicolas!

Have a better solution? Tweet us a gist with your suggestions or open a pull request.

Newer postOlder post

Don’t miss the next post!

Absolutely no spam. Unsubscribe anytime.


About usCareersContact