'Flaky' tests: a short story
One of the hardest failing tests to debug are those which fail randomly, also known as “flaky” tests. You write your test cases, you run the tests in your environment (in random order), and see them all pass. Afterwards, you push your code, your CI server runs them and one test fails.
This is not an uncommon scenario, and one too common when using integration
tests which use JS, with
But if your failing test doesn’t communicate with an external API, doesn’t use
JS, and passes locally, it can be a bit nerve-wracking.
After you have identified the failing test, and it still passes after running it locally, one way to figure out why it’s failing is running its context multiple times.
To automate this process a bit, I like to use the following command:Read more »