Now using Firefox for functional/integration testing

55 seconds versus 20 seconds: this is the reason why I am switching from Chrome to Firefox for my automated tests.  I normally wouldn't write a blog post about what browser I use for testing but I found the difference between the new Firefox and Chrome so striking that I thought I should write about it. If you want to know the details, please read on.

To test Reeborg's World, I use two different testing strategies:

1. I run unit tests of individual javascript functions using tape; I plan at some point to write a blog post explaining why I think that this is the best Javascript testing framework for casual programmers like me who prefer Python over other languages.
2. I run functional/integration tests using Qunit. 

The way I use QUnit is to take the existing html file including all javascript and python code, hide all the normal UI by setting the css property to "display: none", insert some additional testing script which set up automatic programming execution in an attempt to test all robot commands and ensure that all existing programming challenges work.  I run these tests using a local Python browser (similar to python -m http.server) which, admittedly, is not the most efficient server. The integration tests *might* run faster if I were to use a different local web server ... but that would only make the difference between Chrome and Firefox even more striking.

For many years, I have been using Chrome as my browser of choice for development. However, after testing the new Firefox version, I am definitely making the change. Here are the numbers, and how I obtained them.  I ran each testing sequence 5 times.

After starting my local web browser, for each test with a given browser, I open a private window, paste the url in the address bar and press return.  Time to load the page until I see the display, as measured by my watch:

• Chrome (v 62, 64 bits): 12 seconds
• Firefox (v 58, 64 bits): 3 seconds.

Yes, measuring with my watch is not precise; however it is precise enough to distinguish between 3 and 12 seconds.

After loading the page, the tests start in earnest.  According to QUnit, the time taken to run the tests is:

• Chrome: between 42 and 43.5 seconds
• Firefox: between 16 and 17 seconds.

So, including the time to load (measured by my watch), it takes approximately 55 seconds to run the test suite using Chrome, and approximately 20 seconds using Firefox.

This is why I am switching to Firefox.

As I was curious, I also tried Microsoft Edge. While Edge can be used apparently without any problems on my public site, it hangs inexplicably when trying to run tests from my local (Python) server.