1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
|
---
layout: post
title: Version 2.0 – testing improvements
author: matthew
---
FixMyStreet has a large and hopefully comprehensive test suite that runs
through all aspects of the codebase, checking everything is working. This
makes it easier to change code and add new features, safe in the knowledge
that any breakages will be quickly highlighted.
## Speeding up the tests
Every time someone commits code to our GitHub repository, or opens a pull
request, the tests are automatically run for us by Travis CI. We're alerted to
success or failure with little green ticks or red crosses on GitHub, and by
notice in IRC.
The tests seemed to have slowed down considerably in recent times, but we
couldn't identify any changes at the FixMyStreet side which might have caused
this.
However, there had recently been some spam scraping of
[Gaze](https://gaze.mysociety.org/), our web service that provides population
density information to FixMyStreet (so that e.g. the [maps can try and guess an
appropriate zoom
level](https://www.mysociety.org/2012/08/14/mysociety-design-tips-how-we-choose-the-best-map-zoom-level/),
and so alerts can try and guess an appropriate radius), and rate limiting had
been added to try and help combat it.
Dave spotted that this was being triggered by FixMyStreet test runs, leading to
pauses as the suite waited for the rate limiting to ease. Thankfully, all Gaze
calls were being routed through one function (that had been created in order to
cope gracefully with a Gaze failure) and so it was a simple matter for this
function to be stubbed out if being run as part of a test.
[Before](https://travis-ci.org/mysociety/fixmystreet/builds/143317849):
[After](https://travis-ci.org/mysociety/fixmystreet/builds/143325800):
There are many tests that still rely on the internet (e.g. for some MapIt
lookups) and eventually it would be good to get to the point where they are all
stubbed out and the test suite can run completely offline, probably even more
quickly.
## Multiple test running
When running the tests, the suite creates a test database (in PostgreSQL terms,
it actually creates a temporary cluster) so that anything it does won't affect
your development database. Theoretically, this means you should be able to run
the test suite multiple times simultaneously – perhaps it's doing a full run,
but you want to try and fix (and retest) the first error while it carries on.
However, this was not working, and after some investigation it turned out that
each run was creating (and overwriting) a test configuration `.yml` file, which
meant the existing runs got all confused. Adding a process ID to the test
configuration file meant that each run is independent and can successfully
coexist with each other.
## Keystroke saving
Lastly, you used to have to run the full suite with `bin/run-tests t`, but now
if you run `bin/run-tests`, it will assume you meant `t`. A small thing, but it
might save a few seconds over the years. ;-)
|
