diff options
Diffstat (limited to 'docs/_posts/2016-12-15-v2.0-faster-tests.md')
| -rw-r--r-- | docs/_posts/2016-12-15-v2.0-faster-tests.md | 65 |
1 files changed, 65 insertions, 0 deletions
diff --git a/docs/_posts/2016-12-15-v2.0-faster-tests.md b/docs/_posts/2016-12-15-v2.0-faster-tests.md new file mode 100644 index 000000000..2e55cf7b3 --- /dev/null +++ b/docs/_posts/2016-12-15-v2.0-faster-tests.md @@ -0,0 +1,65 @@ +--- +layout: post +title: Version 2.0 – testing improvements +author: matthew +--- + +FixMyStreet has a large and hopefully comprehensive test suite that runs +through all aspects of the codebase, checking everything is working. This +makes it easier to change code and add new features, safe in the knowledge +that any breakages will be quickly highlighted. + +## Speeding up the tests + +Every time someone commits code to our GitHub repository, or opens a pull +request, the tests are automatically run for us by Travis CI. We're alerted to +success or failure with little green ticks or red crosses on GitHub, and by +notice in IRC. + +The tests seemed to have slowed down considerably in recent times, but we +couldn't identify any changes at the FixMyStreet side which might have caused +this. + +However, there had recently been some spam scraping of +[Gaze](https://gaze.mysociety.org/), our web service that provides population +density information to FixMyStreet (so that e.g. the [maps can try and guess an +appropriate zoom +level](https://www.mysociety.org/2012/08/14/mysociety-design-tips-how-we-choose-the-best-map-zoom-level/), +and so alerts can try and guess an appropriate radius), and rate limiting had +been added to try and help combat it. + +Dave spotted that this was being triggered by FixMyStreet test runs, leading to +pauses as the suite waited for the rate limiting to ease. Thankfully, all Gaze +calls were being routed through one function (that had been created in order to +cope gracefully with a Gaze failure) and so it was a simple matter for this +function to be stubbed out if being run as part of a test. + +[Before](https://travis-ci.org/mysociety/fixmystreet/builds/143317849): + + +[After](https://travis-ci.org/mysociety/fixmystreet/builds/143325800): + + +There are many tests that still rely on the internet (e.g. for some MapIt +lookups) and eventually it would be good to get to the point where they are all +stubbed out and the test suite can run completely offline, probably even more +quickly. + +## Multiple test running + +When running the tests, the suite creates a test database (in PostgreSQL terms, +it actually creates a temporary cluster) so that anything it does won't affect +your development database. Theoretically, this means you should be able to run +the test suite multiple times simultaneously – perhaps it's doing a full run, +but you want to try and fix (and retest) the first error while it carries on. +However, this was not working, and after some investigation it turned out that +each run was creating (and overwriting) a test configuration `.yml` file, which +meant the existing runs got all confused. Adding a process ID to the test +configuration file meant that each run is independent and can successfully +coexist with each other. + +## Keystroke saving + +Lastly, you used to have to run the full suite with `bin/run-tests t`, but now +if you run `bin/run-tests`, it will assume you meant `t`. A small thing, but it +might save a few seconds over the years. ;-) |