How Meeshkan works
Testing is the core of what Meeshkan does and it's important that we're transparent about how it all works. This section covers some of the Meeshkan fundamentals.
In this section:
Meeshkan uses an OpenAPI specification to understand your project's expected inputs and outputs. This OpenAPI specification provides a framework for Meeshkan to generate test properties. From there, Meeshkan runs property-based tests on your project.
If you don't already have an OpenAPI specification, we can generate one for you with our HTTP Mocking Toolkit (HMT). HMT collects recordings of server traffic and generates a JSON Lines file based on the logs. It then builds an OpenAPI schema from that file.
During the alpha release (which we're in now), there are two test schedules:
- Your project is out of the box compatible. Using the checks API, Meeshkan will automatically run on every commit where the app is installed.
- Your project needs some manually configuration by the Meeshkan team to build and run. Meeshkan will test all registered repositories on a weekly basis. We decided on weekly because property-based testing is very resource-intensive. Not to mention that the volume of bugs discovered in a single run can be significant.
We'll soon be introducing paid plans with options to test repositories more often. If you're interested in this, let us know.
We're currently experimenting with the most effective way to report bugs. By default, checks happen on a per commit basis using the GitHub Checks API.
Other options we're exploring include email digests, Slack notifications, and filing an issue on the registered repository.
In some cases, Meeshkan is able to suggest a fix for the reported bug. This is a sparse feature at the moment, but it's being actively developed. Please note that, in any case, these are suggestions and not a certain solution to the problem.
Here's an example of what they could look like (anonymized for privacy). The suggestion is the last sentence of the issue description: