diff options
Diffstat (limited to 'docs/_posts')
48 files changed, 2325 insertions, 0 deletions
diff --git a/docs/_posts/2012-02-06-an-introduction.md b/docs/_posts/2012-02-06-an-introduction.md new file mode 100644 index 000000000..f0633b0eb --- /dev/null +++ b/docs/_posts/2012-02-06-an-introduction.md @@ -0,0 +1,33 @@ +--- +layout: post +title: An Introduction +author: struan +--- + +[mySociety](https://www.mysociety.org/) wrote +[FixMyStreet](https://www.fixmystreet.com/) in 2007 in order to make it easy for +people in the UK to report problems in their area. Since then over +[200,000 problems](https://www.mysociety.org/2012/01/24/fixmystreet-another-big-number/) +have been reported to UK councils using the site. As with all our +sites we want people everywhere to be able to benefit so we were delighted to +work with the Norwegian Unix User’s Group ([NUUG](http://www.nuug.no)) +last year to set up a [FixMyStreet in Norway](http://www.fiksgatami.no/). + +We want to build on that and help more people around the world install their +own copy of FixMyStreet. In the months since +[FiksGataMi](http://www.fiksgatami.no/) was launched we’ve made a number +of changes to FixMyStreet to make it easier to install and customise for other +countries. We’ve rewritten much of the code to use the +[Catalyst](http://www.catalystframework.org/) web framework which has +helped make it easier to customise and we’re now improving the +documentation and ironing out the installation procedure. + +However, we know that easy to install code is only part of the process. +Building a community around the software is far more valuable in encouraging +people to use and improve it. This site and the new +[mailing list](https://groups.google.com/a/mysociety.org/forum/#!forum/fixmystreet) +are our first steps along that road. + +So, if you’ve ever wanted to have a FixMyStreet for your town, state or +country then join the mailing list and we’ll help you along the way. + diff --git a/docs/_posts/2012-07-27-default-workings.md b/docs/_posts/2012-07-27-default-workings.md new file mode 100644 index 000000000..ce89b3f4a --- /dev/null +++ b/docs/_posts/2012-07-27-default-workings.md @@ -0,0 +1,43 @@ +--- +layout: post +title: Default Workings +author: matthew +--- + +In the past few weeks, a number of improvements have been made to the +FixMyStreet default set up, so that installation should provide you with a +working setup more easily and quickly, before you get on and make all the +necessary customisations you will want to for the service you are setting up. + +Firstly, we've tidied and consolidated the documentation on to this site, +putting everything you need in one place. We are using GitHub pages, which means +that the documentation is bundled along with the repository when checking out, +which might be useful. The installation guide now includes help for installing +on Mac OS X, and various other tweaks and improvements. + +Next, the codebase now automatically defaults to +[OpenStreetMap](http://www.openstreetmap.org/) maps and geocoding – these are +available, with more or less data, everywhere in the world, so you should be +able to test your installation and see working maps. + +Whilst an installation of [MapIt](http://global.mapit.mysociety.org/) may be +necessary for your FixMyStreet to work as you want – mapping locations picked to +the right authority might need some private boundary data, for example – the +code will now default to work as if everywhere is one administrative area. + +The code for sending reports has been refactored and modularised, enabling +proprietary options to be more easily added alongside the standard email, +Open311, and so on. + +We have removed any UK specific code from the default cobrand, moving it to +a UK cobrand (which is then in turn inherited by the various council cobrands +we have made in the UK). This should mean that you find you have less to override, +and more things should work by default. + + + +Lastly, the default cobrand now uses the new style that you can see on +<https://www.fixmystreet.com>. By default, we have picked a pretty yet garish +colour scheme, in order to remind you that you almost certainly want to change +the colours being used for your own installation :) + diff --git a/docs/_posts/2012-08-17-improving-configuration.md b/docs/_posts/2012-08-17-improving-configuration.md new file mode 100644 index 000000000..12ce5e34f --- /dev/null +++ b/docs/_posts/2012-08-17-improving-configuration.md @@ -0,0 +1,27 @@ +--- +layout: post +title: Improving Configuration +author: matthew +--- + +Now that a default install is a bit more straightforward to set up, our +thoughts turn to improving the customistation of that default install. +Currently, apart from the options already present in the main configuration +file, that involves knowing a bit of Perl, in order to create a Cobrand .pm +file containing the various customistations. So to reduce that dependency, +we've moved a number of these options into the main configuration file, so that +hopefully a standard customisation might not need a Cobrand .pm file at all. + +These changes range from simple text strings that are now in templates, through +to specifying what areas from MapIt you are interested in, or what languages +the site is available in. The general.yml-example file contains information on +each option, and we've updated our [customisation documentation](/customising/) +as well. + +Also, thanks to some testing of a current installation by +[Anders](https://github.com/kagee) for FiksGataMi, we've made more incremental +improvements to the installation, including fixing a couple of tests that +shouldn't run unless your configuration is set up in a particular way, making +sure inherited cobrands use the best templates, and including the +Catalyst::Devel module so running the development server is easier. + diff --git a/docs/_posts/2012-10-02-easy-install.md b/docs/_posts/2012-10-02-easy-install.md new file mode 100644 index 000000000..b047b1f71 --- /dev/null +++ b/docs/_posts/2012-10-02-easy-install.md @@ -0,0 +1,34 @@ +--- +layout: post +title: Easy Installation +author: matthew +--- + +Four months ago, someone raised [a +ticket](https://github.com/mysociety/fixmystreet/issues/302) on FixMyStreet's +GitHub account, asking for alternative ways of setting up an installation. +We certainly agreed this was a good idea, as we're well aware that there +are various different parts to FixMyStreet that might require quite a bit of +knowledge in setting up. + +We're now pleased to announce that we have created an [AMI](/install/ami/) +(Amazon Machine Image) containing an already set-up default installation of +FixMyStreet. You can use this to create a running server on an Amazon EC2 +instance. If you haven't used Amazon Web Services before, then you can get a +Micro instance free for a year. + +If you have your own server, then we have separately released the [install +script](/install/install-script/) that is used to create the AMI, which can be +run on any clean Debian or Ubuntu server to set everything up for you, from +the PostgreSQL database to nginx. + +If you prefer to do things manually, and already know how to set up your +database and web server, our [manual documentation](/install/) is still +available. + +An AMI and install script is also available for MapIt -- see our +[MapIt documentation](http://mapit.poplus.org/) for more details. +This should make it very straightforward to get something set up for testing +and development. + +Do let us know how you get on. diff --git a/docs/_posts/2013-02-22-bodies.md b/docs/_posts/2013-02-22-bodies.md new file mode 100644 index 000000000..1d65640eb --- /dev/null +++ b/docs/_posts/2013-02-22-bodies.md @@ -0,0 +1,43 @@ +--- +layout: post +title: Version 1.1 - Bodies +author: matthew +--- + +<img class="r" src="/assets/posts/big-head-small.jpeg" width="320" height="240" alt="Big Beautiful Face Statue in Tenerife by epSos.de"> + +Today we're releasing **version 1.1** of the FixMyStreet platform. The [AMI](/install/ami/) has +been updated and the [install script](/install/install-script/) will automatically now install +this version. + +The main change since version 1.0 is the addition of **bodies**. Historically, +FixMyStreet has assumed that the administrative areas that are returned from +MapIt are the same thing as the bodies to which the reports will be sent. This +has led over time to a number of workarounds when this hasn't been the case, +either in manual code changes in FixMyStreet or by adding new types to a MapIt +install, and dealing with it in that way. + +We have updated the code so that FixMyStreet holds its own records of bodies to +which reports can be sent, and the MapIt area -- or areas -- that they cover. +This is conceptually much clearer, as well as making it much easier to have a +body covering multiple areas, an area with multiple bodies, or a combination. + +Smaller functional changes in this release include admin improvements (it now +looks like the front end site, and has add a number of other additions), and a +couple of new configuration variables, `DO_NOT_REPLY_EMAIL` and +`SEND_REPORTS_ON_STAGING`, to make debugging a little easier, along with a +`--debug` option to send-reports. Also, we found on the [mailing +list](/community/) a couple of times that people ran into trouble because their +MapIt had debug turned on, and FixMyStreet didn't cope well with the debug +output MapIt included in its responses. This has now been fixed. + +Many others of the commits in the past few months have been for various +installations of the codebase, from the forthcoming FixMyBarangay in the +Philippines to local UK council installs such as Oxfordshire or Bromley. These +have in many cases led to small improvements and bugfixes to the central +codebase, which can then be used by any reusers of the code. + +Lastly, all the strings in the JavaScript are now translatable, along with a +few other strings that had previously been missed; do let us know if you +find any other strings that can't be translated and we'll look into it. + diff --git a/docs/_posts/2013-05-03-v1.2.md b/docs/_posts/2013-05-03-v1.2.md new file mode 100644 index 000000000..7d839a41e --- /dev/null +++ b/docs/_posts/2013-05-03-v1.2.md @@ -0,0 +1,34 @@ +--- +layout: post +title: Version 1.2 +author: matthew +--- + +<a href="http://www.flickr.com/photos/rbm/100165557/" title="broken lamp post by Rob Mac, on Flickr"><img class="r" src="http://farm1.staticflickr.com/28/100165557_6aff9acd7b.jpg" width="375" height="500" alt="broken lamp post"></a> + +Today we're releasing **version 1.2** of the FixMyStreet platform. The +[AMI](/install/ami/) has been updated and the [install +script](/install/install-script/) will automatically now install this version. + +The main items in this release are things prompted by requests on [our mailing +list](/community/) :-) + +* Postfix is now installed as part of the install script, or in the AMI. This + means email should work out of the box. For anyone already installed, you can + run the commands in [the GitHub ticket](https://github.com/mysociety/fixmystreet/issues/412). + +* A new configuration option `MAPIT_ID_WHITELIST` has been added, to restrict + usage to the IDs specified, if given. This means Claudio, who emailed last + week, could have `[ 239540 ]` as his `MAPIT_ID_WHITELIST`, and then reports + could only be made within the Marche region of Italy. We already use this new + option ourselves on https://www.zueriwieneu.ch/ where before it was + hard-coded in the code. + +* Other things include being able to zoom in further on OSM maps, and HTML + pages are now being gzipped. + +Lastly, as you can see this site has had a redesign to make it more friendly, +and we've added some more documentation about e.g. updating an AMI instance to +a newer version. If there's anything unclear, please do [ask on the mailing +list](/community/) or submit an issue or pull request on GitHub. + diff --git a/docs/_posts/2013-11-25-v1.3.md b/docs/_posts/2013-11-25-v1.3.md new file mode 100644 index 000000000..c94a63d79 --- /dev/null +++ b/docs/_posts/2013-11-25-v1.3.md @@ -0,0 +1,44 @@ +--- +layout: post +title: Version 1.3 +author: matthew +--- + +<a href="http://www.flickr.com/photos/jasonbacon/2281783615/" title="Yosemite tunnel tonemapped in qtpfsgui by nineball2727, on Flickr"><img class="r" src="http://farm3.staticflickr.com/2252/2281783615_d1cb6f7ccd.jpg" width="500" height="339" alt="Yosemite tunnel tonemapped in qtpfsgui"></a> + +**Version 1.3** of FixMyStreet is now out :-) The [install +script](/install/install-script/) and [AMI](/install/ami/) are both updated to +this version. + +I realise I haven't posted here with each point release during version 1.2, +sorry. You can see the changes in each release here on GitHub at +[https://github.com/mysociety/fixmystreet/releases](https://github.com/mysociety/fixmystreet/releases) and below is a list of all +the main things that have changed since version 1.2 (* means new since the +last version, 1.2.6, if you were keeping track): + +* OpenLayers upgraded to 2.13.1, giving e.g. animated zooming +* A fully functional Google Maps layer via OpenLayers +* *\** If you only specify one cobrand in the configuration file, the site will + always use it, rather than only if your hostname also matches. This is + probably what you would expect to happen! +* *\** A contact can be given multiple email addresses +* *\** A body can be marked as deleted, and then it will not be used by the + front end at all +* The admin interface has had a lot of inline documentation, hints and notices + added, along with a page showing the site's current configuration +* *\** The admin interface has some feature additions from coding volunteers, + such as a date picker on the stats page thanks to Andrew Black, and searching + by external ID thanks to Andy Lulham +* We've added an example Vagrantfile and improved the install scripts +* *\** The test suite should now run regardless of the contents of your + configuration file +* Translation improvements - some better wording of strings, some missed or + UK-only URLs/translations (thanks Jonas and Rikard), and a fix for the + long-standing issue where multiline strings were not being translated + (hooray) +* *\** Bug fixes, most notably sometimes when changing report state in the admin + interface, and an issue with the bottom navbar in Chrome (which we've + reported to the Chromium project) + +As always, do [ask on the mailing list](/community/) if you'd like more +information on any of the above, or submit an issue or pull request on GitHub. diff --git a/docs/_posts/2014-05-23-v1.4.md b/docs/_posts/2014-05-23-v1.4.md new file mode 100644 index 000000000..626312bcc --- /dev/null +++ b/docs/_posts/2014-05-23-v1.4.md @@ -0,0 +1,34 @@ +--- +layout: post +title: Version 1.4 +author: matthew +--- + +<div class="r" align="right"><a href="https://www.flickr.com/photos/rozabbotts/6675291613/" title="Yellow by RozSheffield on Flickr"><img src="https://farm8.staticflickr.com/7154/6675291613_3d4764abc5.jpg" width="379" height="500" alt="Yellow"><br>Yellow by RozSheffield on Flickr</a></div> + +Say hello to **version 1.4** of FixMyStreet. As usual, the [install +script](/install/install-script/) and [AMI](/install/ami/) have both been +updated to this version. + +Hopefully this release will run more smoothly on EC2 micro instances, with some +cron rejigging to alleviate memory problems. + +There have been a few minor user-facing improvements, such as automatically +selecting the reporting category if there's only one choice, and removing the +indenting of emails; some bug fixes, including changes for the new version of +Debian, a problem with language setting in email alerts, and removal of cached +photos; and a number of improvements for people reusing the code, including a +no-op send method, having reports on staging sites be sent to the reporter, +adding an external URL field to bodies, and making it easier to change the pin +icons. + +See the full changes over on GitHub at +[https://github.com/mysociety/fixmystreet/releases](https://github.com/mysociety/fixmystreet/releases). + +Thanks to Andy Lulham, Chris Mytton, Dave Arter, Dave Whiteland, Gerald, Hakim +Cassimally, Ian Chard, Jon Kristensen, Jonas Oberg, Kindrat, Matthew +Somerville, Rikard, Steven Day, and Struan Donald for contributing to this +release. + +As always, do [ask on the mailing list](/community/) if you'd like more +information on any of the above, or submit an issue or pull request on GitHub. diff --git a/docs/_posts/2014-07-15-v1.4.2.md b/docs/_posts/2014-07-15-v1.4.2.md new file mode 100644 index 000000000..72cd43528 --- /dev/null +++ b/docs/_posts/2014-07-15-v1.4.2.md @@ -0,0 +1,25 @@ +--- +layout: post +title: Version 1.4.2 +author: matthew +--- + +<div class="r" align="right"><a href="https://www.flickr.com/photos/theenmoy/8372727576/" title="Bicycle Rack by Theen Moy on Flickr"><img src="https://farm9.staticflickr.com/8083/8372727576_75d392409c.jpg" width="500" height="375" alt="Bicycle Rack"><br>Bicycle Rack by Theen Moy on Flickr</a></div> + +We’ve released **version 1.4.2**, a maintenance release, but also with a couple +of new features. The main reason for this release was to fix a couple of issues +with the installation script, which are hopefully now resolved. It also +upgrades our bundled copy of `cpanm` (to cope better when an external module +website is down), tidies up the template structure, copes with browser autofill +on the `/auth` sign in page, and adds links from the All Reports page to a +body’s open or fixed reports. + +The main new feature is the addition of SMTP username/password and SSL/TLS +options for your setup. You can read about the new options on +[the configuration settings page](/customising/config/#smtp_smarthost) + +See the full changes over on GitHub at +[https://github.com/mysociety/fixmystreet/releases](https://github.com/mysociety/fixmystreet/releases). + +As always, do [ask on the mailing list](/community/) if you'd like more +information on any of the above, or submit an issue or pull request on GitHub. diff --git a/docs/_posts/2014-11-20-v1.5.md b/docs/_posts/2014-11-20-v1.5.md new file mode 100644 index 000000000..69ffefb46 --- /dev/null +++ b/docs/_posts/2014-11-20-v1.5.md @@ -0,0 +1,39 @@ +--- +layout: post +title: Version 1.5 +author: matthew +--- + +<div class="r" align="right"> +<a href="https://www.flickr.com/photos/shutupyourface/5545353603" title="Resting Tahr 2 by Jason Armstrong, on Flickr"><img src="https://farm6.staticflickr.com/5091/5545353603_71e3822ce9.jpg" width="500" height="303" alt="Resting Tahr 2"><br>Resting Tahr by Jason Armstrong on Flickr</a> +</div> + +We’ve released **version 1.5** of FixMyStreet. This version fully supports the +new Long Term Support (LTS) version of Ubuntu, Trusty Tahr 14.04 (the code did +already run fine on Ubuntu Trusty if you set it up manually, but now the +install script will work and a few other bits have been tidied). + +This release comes with a few improvements to the admin interface, including +pagination of search results, validation of new categories, and some display +enhancements. + +We've moved the map sidebar to be flush with the window edge, which we think is +simpler and easier on the eye, and we've continued making the template +structure easier to change and override. + +We've also fixed some bugs, such as map submission not working with JavaScript +disabled or unavailable. As another example, we had a report of the Android +browser crashing when showing a map page, which we tracked down to the slightly +transparent map navigation controls – crashing wasn't worth this, so now on +mobile they're fully opaque. + +From [Transifex](https://www.transifex.com/projects/p/fixmystreet/) we've added +four new languages (as well as updating the existing ones): +Albanian, Bulgarian, Hebrew, and Ukranian. + +See the full changes +[over on GitHub](https://github.com/mysociety/fixmystreet/releases). + +As always, do [ask on the mailing list](/community/) if you'd like more +information on any of the above, or submit an issue or pull request on GitHub. + diff --git a/docs/_posts/2015-03-18-v1.5.4.md b/docs/_posts/2015-03-18-v1.5.4.md new file mode 100644 index 000000000..a7c6cd414 --- /dev/null +++ b/docs/_posts/2015-03-18-v1.5.4.md @@ -0,0 +1,27 @@ +--- +layout: post +title: Version 1.5.4 +author: matthew +--- + +<div class="r" align="right"> +<a href="https://www.flickr.com/photos/alanvernon/6131124997" title="Female Yellow Warbler by Alan Vernon, on Flickr"><img src="https://farm7.staticflickr.com/6061/6131124997_f9e893b37c.jpg" width="500" height="333" alt="Female Yellow Warbler"><br>Female Yellow Warbler by Alan Vernon on Flickr</a> +</div> + +We’ve released **version 1.5.4** of FixMyStreet. + +This includes a couple of new map layers, Bing Maps and Stamen's +[toner-lite](http://maps.stamen.com/toner-lite/), and nicer confirmation pages +for after you've made a report or update, along with other smaller improvements +and bug fixes. See the full list of changes +[over on GitHub](https://github.com/mysociety/fixmystreet/releases). + +For developers, it includes a few small improvements, to do with Mac +installation, making some things optional, and including a new configuration +variable for if you're running behind an SSL proxy. We've also added some test +URLs so that you can view confirmation pages without having to leave a new +report or update, e.g. see it in action on fixmystreet.com: +[https://www.fixmystreet.com/P/\_test\_](https://www.fixmystreet.com/P/_test_). + +As always, do [ask on the mailing list](/community/) if you'd like more +information on any of the above, or submit an issue or pull request on GitHub. diff --git a/docs/_posts/2015-07-10-v1.5.5.md b/docs/_posts/2015-07-10-v1.5.5.md new file mode 100644 index 000000000..b82c55738 --- /dev/null +++ b/docs/_posts/2015-07-10-v1.5.5.md @@ -0,0 +1,50 @@ +--- +layout: post +title: Version 1.6 +author: matthew +--- + +<div class="r" align="right"> +<a data-flickr-embed="true" href="https://www.flickr.com/photos/caffeina/2079673826/" title="security"><img src="https://farm3.staticflickr.com/2065/2079673826_c4edb07e4d.jpg" width="500" height="333" alt="security"></a><script async src="//embedr.flickr.com/assets/client-code.js" charset="utf-8"></script> +</div> + +We’ve released **version 1.6** of FixMyStreet (previously numbered 1.5.5). + +This release includes important security fixes: + +* A vulnerability in login email sending that could allow an account to + be hijacked by a third party; +* Alterations to token logging in and timeout behaviour; +* A dependency update to fix an issue with Unicode characters in passwords. + +More details on those items below. Other items in this release include a +Chinese translation, a bug fix with shrunken update photos, and some front end +improvements, such as a ‘hamburger’ menu icon and an easier Report button on +mobile, and resized map pins based on zoom level. + +See the full list of changes +[over on GitHub](https://github.com/mysociety/fixmystreet/releases). + +Security fixes +-------------- + +**Login email account hijacking:** +Due to the way parameters were passed into the token table in the database, it +was possible for someone to request a login email for one email address, but +have the login email sent to different address. This would allow a third party +to log in as someone else, letting them make reports or updates as that person. + +The code has been rewritten so all user parameter passing goes through central +functions that return only one parameter even if the user has passed multiple +parameters. [More details of this class of vulnerability](http://blog.gerv.net/2014/10/new-class-of-vulnerability-in-perl-web-applications/). + +**Email authentication tokens:** +Problem confirmation tokens had to be used within a month; this now applies to +all confirmation tokens, and email sign in tokens are valid for a day. Using +those tokens after confirmation will redirect correctly, but no longer log you +in; links in alert emails will no longer log you in. + +**Unicode characters in passwords:** +The package our code uses to encode database columns, +DBIx::Class::EncodedColumn, could have issues with Unicode characters provided +to it. This was fixed by upgrading the version we use. diff --git a/docs/_posts/2015-07-31-v1.6.1.md b/docs/_posts/2015-07-31-v1.6.1.md new file mode 100644 index 000000000..fdf7544d6 --- /dev/null +++ b/docs/_posts/2015-07-31-v1.6.1.md @@ -0,0 +1,22 @@ +--- +layout: post +title: Version 1.6.1 +author: matthew +--- + +<div class="r" align="right"> +<a data-flickr-embed="true" href="https://www.flickr.com/photos/bmiphone/5488751401/" title="Copenhagen trip"><img src="https://farm6.staticflickr.com/5052/5488751401_fe5bc84a49.jpg" width="500" height="333" alt="Copenhagen trip"></a><script async src="//embedr.flickr.com/assets/client-code.js" charset="utf-8"></script> +</div> + +We’ve released **version 1.6.1** of FixMyStreet. + +This release fixes a bug introduced in the previous release when setting +multiple areas for a body in the administration interface. + +It also includes improvements to the All Reports page, adding a fixed header +and tooltips, and stops the sidebar running over the footer on alerts pages. +The admin gets a variety of minor improvements, with better internal linking +and a mark as sent button. Plus a Danish translation :) + +As ever, see the full list of changes +[over on GitHub](https://github.com/mysociety/fixmystreet/releases). diff --git a/docs/_posts/2015-10-23-v1.7.md b/docs/_posts/2015-10-23-v1.7.md new file mode 100644 index 000000000..98035a921 --- /dev/null +++ b/docs/_posts/2015-10-23-v1.7.md @@ -0,0 +1,39 @@ +--- +layout: post +title: Version 1.7 +author: matthew +--- + +<div class="r" align="right"> +<a data-flickr-embed="true" href="https://www.flickr.com/photos/flyingturtle/837938254/" title="yellow lines"><img src="https://farm2.staticflickr.com/1074/837938254_c4ecb36323.jpg" width="500" height="375" alt="yellow lines"></a><script async src="//embedr.flickr.com/assets/client-code.js" charset="utf-8"></script> +</div> + +We’ve released **version 1.7** of FixMyStreet. + +This version adds some new features. First off is that the FixMyStreet design +is now *bi-directional*, providing an easy switch to flip the design to either +left-to-right or right-to-left. This was done with the kind support of the +National Democratic Institute. + +We have added *state and category filters* to the list pages, letting users +view only e.g. open reports in the potholes category, or all reports in the +graffiti category. Various design improvements have been made, including the +showing of the report on a questionnaire page and the email confirmation pages, +and we've added a nicer default OpenGraph image. + +Database *performance* has been improved in a number of areas, and the +accessibility of the map pages has been improved. + +This release also fixes a number of small *bugs*, including translating report +states in the admin index, dealing with DMARC email issues, and fixes for +Google Maps API users. + +For *developers*, we've made it easier to run gettext-extract if you're +performing your own translations, removed some confusing warnings, finally +removed the final few hardcoded "FixMyStreet" strings so it's easy to rename +your site, streamlined the navigation menu and list item CSS using a BEM style +naming scheme so it is easy to change and override, and lastly fixed a long +standing issue where errors were not always logged correctly. + +Plus quite a few other things; as always, see the full list of changes +[over on GitHub](https://github.com/mysociety/fixmystreet/releases). diff --git a/docs/_posts/2016-03-02-v1.8.md b/docs/_posts/2016-03-02-v1.8.md new file mode 100644 index 000000000..c79172515 --- /dev/null +++ b/docs/_posts/2016-03-02-v1.8.md @@ -0,0 +1,37 @@ +--- +layout: post +title: Version 1.8 +author: matthew +--- + +<div class="r" align="right"> +<a data-flickr-embed="true" href="https://www.flickr.com/photos/danbri/2520508005/" title="Thin yellow line"><img src="https://farm4.staticflickr.com/3078/2520508005_4d388294fc.jpg" width="500" height="334" alt="Thin yellow line"></a><script async src="//embedr.flickr.com/assets/client-code.js" charset="utf-8"></script> +</div> + +We’ve released **version 1.8** of FixMyStreet. + +The two main new features in this release are *Facebook login* – provide a +Facebook app ID and secret in your configuration and it will smoothly fit into +the creation and login flow – and *multiple photo support*, along with a modern +interface for previewing and uploading photos whilst you create your report. + +Smaller improvements include *highlighting the pin* when you hover over an item +in the sidebar, and vice-versa; fixing some small display bugs such as how +updates were displayed in Your Reports and preventing a chevron being stretched +in Firefox; improving the look of the 404 page, and making sure you can see an +update if you got to it via an in-page link. + +Memory *performance* has been improved, meaning cron jobs can take up to half +as much memory, and this release also fixes a number of small bugs, including +an embarrassing swap of latitude and longitude in the Google geocoder, making +sure you're signed up for updates if you used the app and were logged in, and +better internationalisation and display of numbers. + +For *developers*, we've added a generic static route handler, so that adding +new static HTML pages to your installation involves only creating a new file in +your template directory and nothing more; improved bounce handling; and fixed +the cobrand restriction handling on Your Reports and list pages. + +Plus quite a few other things; as always, see the full list of changes +[over on GitHub](https://github.com/mysociety/fixmystreet/releases). + diff --git a/docs/_posts/2016-03-23-v1.8.1.md b/docs/_posts/2016-03-23-v1.8.1.md new file mode 100644 index 000000000..2e028761a --- /dev/null +++ b/docs/_posts/2016-03-23-v1.8.1.md @@ -0,0 +1,22 @@ +--- +layout: post +title: Version 1.8.1 +author: matthew +--- + +We’ve released **version 1.8.1** of FixMyStreet, which fixes some bugs and +makes some improvements to the 1.8 release. + +Now there is multiple photo support, the display of those photos on a report +page is now a bit nicer, and if there's an error message on photo upload it +should now always be visible. Auto-scrolling of the sidebar when you hover over +a pin has been removed as it was confusing, and the site now remembers a user's +last anonymous state. Fixes have been made for running on more recent versions +of Perl, one of which was causing the geocoder to break. + +For developers, an easier way of adding cobrand-specific custom reporting +fields has been added. + +There are a couple of other minor changes; see the full list of changes +[over on GitHub](https://github.com/mysociety/fixmystreet/releases). + diff --git a/docs/_posts/2016-05-03-v1.8.2.md b/docs/_posts/2016-05-03-v1.8.2.md new file mode 100644 index 000000000..009cb856a --- /dev/null +++ b/docs/_posts/2016-05-03-v1.8.2.md @@ -0,0 +1,30 @@ +--- +layout: post +title: Version 1.8.2 +author: matthew +--- + +<div class="r" align="right"> +<a data-flickr-embed="true" href="https://www.flickr.com/photos/m0php/4001836593/" title="Security barrier"><img src="https://farm3.staticflickr.com/2553/4001836593_492263c6cd.jpg" width="500" height="332" alt="Security barrier"></a><script async src="//embedr.flickr.com/assets/client-code.js" charset="utf-8"></script> +</div> + +We’ve released **version 1.8.2** of FixMyStreet, along with versions 1.7.1 and 1.6.2. + +These releases include an important security fix, whereby a malicious user +could craft an image upload to the server that allowed them to run external +commands as the user running the site. Please update your installation as soon +as possible. + +Version 1.8.2 also contains other improvements and additions to existing +features: + +* Twitter social login, alongside the existing Facebook login; +* PNG and GIF image upload support; +* Some development improvements, including the final merging of `base` and +`fixmystreet` templates, storing any Open311 error in the database, and tidying +up some unused cobrands; +* A few bug fixes, such as showing the right body user form value for fixed +reports (thanks Jon Kristensen). + +See the full list of changes +[over on GitHub](https://github.com/mysociety/fixmystreet/releases). diff --git a/docs/_posts/2016-06-03-v1.8.3.md b/docs/_posts/2016-06-03-v1.8.3.md new file mode 100644 index 000000000..9544c4f89 --- /dev/null +++ b/docs/_posts/2016-06-03-v1.8.3.md @@ -0,0 +1,34 @@ +--- +layout: post +title: Version 1.8.3 +author: matthew +--- + +<div class="r" align="right"> +<a data-flickr-embed="true" href="https://www.flickr.com/photos/freedomiiphotography/5987366229/" title="Passing Light"><img src="https://c6.staticflickr.com/7/6121/5987366229_ecd877f4d8.jpg" width="500" height="333" alt="Passing Light"></a><script async src="//embedr.flickr.com/assets/client-code.js" charset="utf-8"></script> +</div> + +We’ve released **version 1.8.3** of FixMyStreet. + +For developers, the main improvement in this version is the great speed up to +CSS compilation by moving from sass to libsass. + +We've fixed some map issues, getting the Google Maps layer working again and +dealing with tap sensitivity on some devices. + +The admin interface gets a bit of love, adding common search boxes to the index +page, allowing change of email to an existing address, and speeding up photo +removal. + +### Upgrading notes + +If your templates aren't kept in the main repository, there are a couple of +things to be aware of with this release: + +* If you've used the `cf` class, you'll need to rename it to `clearfix`. +* If you have customised any of the email templates that use `<?=...?>` style + variables (e.g. alert emails, questionnaire, submit), you'll need to + rewrite them to use the standard `[%...%]` variables. + +See the full list of changes +[over on GitHub](https://github.com/mysociety/fixmystreet/releases). diff --git a/docs/_posts/2016-07-06-v1.8.4.md b/docs/_posts/2016-07-06-v1.8.4.md new file mode 100644 index 000000000..de4a7d3fd --- /dev/null +++ b/docs/_posts/2016-07-06-v1.8.4.md @@ -0,0 +1,23 @@ +--- +layout: post +title: Version 1.8.4 +author: matthew +--- + +<div class="r" align="right"> +<a data-flickr-embed="true" href="https://www.flickr.com/photos/ennid/5999264663/" title="Yellow Line (Mumbai)"><img src="https://c8.staticflickr.com/7/6002/5999264663_9384176094.jpg" width="500" height="375" alt="Yellow Line (Mumbai)"></a><script async src="//embedr.flickr.com/assets/client-code.js" charset="utf-8"></script> +</div> + +We’ve released **version 1.8.4** of FixMyStreet, along with versions 1.7.2 and 1.6.3. + +These releases include a security fix, closing two +cross-site scripting (XSS) vulnerabilities. Please update +your installation as soon as possible. + +Version 1.8.4 also contains other minor bug fixes, such as correctly +orientating preview images, wrapping emails better for differing screen sizes, +a bug with filter redirect when JavaScript wasn't available, a race condition +when starting a new report, and a couple of display bugs in IE7. + +See the full list of changes +[over on GitHub](https://github.com/mysociety/fixmystreet/releases). diff --git a/docs/_posts/2016-11-15-v2.0.md b/docs/_posts/2016-11-15-v2.0.md new file mode 100644 index 000000000..dfc430b5b --- /dev/null +++ b/docs/_posts/2016-11-15-v2.0.md @@ -0,0 +1,120 @@ +--- +layout: post +title: Version 2.0 +author: matthew +--- + +<div class="r" align="right"> +<a data-flickr-embed="true" href="https://www.flickr.com/photos/nauright/4676220337/" title="bureau of street traffic"><img src="https://c2.staticflickr.com/5/4057/4676220337_fd38770a98.jpg" width="500" height="313" alt="bureau of street traffic"></a><script async src="//embedr.flickr.com/assets/client-code.js" charset="utf-8"></script> +</div> + +We’re proud to announce the release of **version 2.0** of FixMyStreet. + +This version contains a wide array of new features, including HTML email and +multiple state/category filtering, a new admin user system with graduated +permissions, and various bugfixes and development improvements. + +Over the next few days and weeks, we will be writing a series of blog posts, +going into details about a number of the changes, which I hope will be of +interest. But before then, do [set up the code](/overview/) or update your +installation, and [ask us questions](/community/) :) + +## New front end features + +* <img src="/assets/posts/html-email-example.png" alt="" class="r"> +**HTML email:** +There is now the option for all emails sent by FixMyStreet to be HTML formatted +where previously they were plain text only. This includes confirmation and +questionnaire emails to the user, and report emails to the public body. These +emails include any image added to the report, plus a small static map of the +problem's location. +<br style="clear:both"> + +* <img src="/assets/posts/multi-select.png" alt="" class="r"> +**State/category filtering and sorting of list pages:** +When viewing a list of reports, you can now filter and sort them in pretty +much any way you choose, including sorting by most- or least-recently updated, +newest or oldest, or most commented. You can also select multiple categories +or states (e.g. "fixed"). +<br style="clear:both"> + +* <img src="/assets/posts/area-highlighting-after.jpeg" alt="" class="r"> +**Pretty area highlighting on body pages:** +The highlighting of areas on a body page has been inverted, so that the +unimportant parts of the map are shaded and you can interact more easily with +reports on the page. +<br style="clear:both"> + +* **Users can now update their own email address** +This was a frequent request from users and we're glad to report that they can +now do it themselves on their account page. + +* <img src="/assets/posts/report-selected.png" alt="" class="r"> +**Performance improvements:** +When looking at reports from a list page, the other report pins stay visible so +that it is easier to switch between them. The report itself is being pulled in +behind the scenes, meaning the whole page does not need to reload. The map no +longer extends underneath the sidebar and header, which makes things easier, +and a scroll wheel can now zoom the map in and out. +<br style="clear:both"> + +* <div class="r" style="height:280px;overflow:auto"><img src="/assets/posts/new-reporting-form.png" alt=""></div> +**Making privacy options clearer:** +The reporting form has been separated into public and private sections, to make +it clearer which parts of what you provide will be made visible on the site. + + **Showing the relevant recipient:** If you live in an area where there's +more than one body, the category you pick normally dictates which body we send +your report to. Now, when you select the category we update the name of the +body given at the top of the report page, if we know that the report will be +sent there. +<br style="clear:both"> + +## New admin user system + +Admin users can now use the same log-in right across the site - whether they're +making a report like a standard user, or logging in to make edits and moderate +the site. + +In the past, the distinction between admin and other users was black and white. +As an admin user, you had access to every part of the site, but users can now be +given individual permissions for various layers of access. These include: + +* **Proxy users** This layer grants the ability to create a report or update on + behalf of a body, or as another user. We envisage this being useful in a + body's contact centre, where they receive a report over a phone and enter it + into FixMyStreet as that user; +* **Report editors** Giving the power to edit a report's category, state, or + location. If the admin user changes the category, and that change means that + a different body is now responsible for the report, it will be re-sent; +* **List makers**, who can compile their own shortlist of reports they wish to + go and inspect. This may be useful for a contractor or team who wishes to + compile the day's tasks; +* **Quick responders** These users have access to response templates, allowing them + to edit and publish templated updates; +* **Prioritisers** These users may set different priorities on reports; +* **Trusted users** A simple reputation system, which e.g. potentially lets + reports from trusted users be actioned more quickly. + +The admin report edit form has also been greatly improved, including a map +to update a report's location (and re-sending the report if the body changes), +and much tidier layout. + +## Bugfixes and development changes + +Bugfixes include updating the top-level domain (TLD) list for email validation, +hiding authorities which don't exist any more on the `/reports` page, and +fixing the previously-broken photo preview display after form submission. We +have dropped support for Internet Explorer 6. + +If you're a re-user of the codebase, there are a number of changes that will +hopefully help you out. Geocoder results won't be cached in development, the +`UPLOAD_DIR` and `GEO_CACHE` variables are now relative to the project root, +we've added a server-side MapIt proxy, and you can add your own fields to the +contact form. Open311 support has been tidied up and improved. If you run the +tests, you can now run the test suite multiple times simultaneously, and the +tests have been sped up quite a bit. + +Plus quite a few other things; as always, see the full list of changes +[over on GitHub](https://github.com/mysociety/fixmystreet/releases). + diff --git a/docs/_posts/2016-11-29-v2.0-html-emails.md b/docs/_posts/2016-11-29-v2.0-html-emails.md new file mode 100644 index 000000000..aa4791886 --- /dev/null +++ b/docs/_posts/2016-11-29-v2.0-html-emails.md @@ -0,0 +1,220 @@ +--- +layout: post +title: Version 2.0 – HTML emails +author: matthew +--- + +FixMyStreet sends a number of automated emails, both to users (confirmation +emails, follow-up questionnaires) and to bodies (the reports themselves). +Previously these were plain text, but we have now introduced HTML emails, with +all the design possibilities that this implies. + +One reason for this is to make the site's communications look more +professional; another is an attempt to minimise one of the most time-consuming +admin tasks - dealing with users who reply to our automated emails. + +## Designing better emails + +In Version 2.0 we wanted to afford FixMyStreet emails the same design and +usability attention that we normally spend on the FixMyStreet website. + +Incorporating feedback from our users, our support team, and our council +partners, we quickly identified a few key places that HTML (graphical) emails +could improve the FixMyStreet experience for everyone: + +1. **Attaching a map and photo** of each problem to our “Has your problem been + fixed?” questionnaire email, to help people remember the report we’re + asking about. +1. **Styling calls to action as attractive, clickable buttons**, to make the + emails easier to quickly scan and comprehend, and to reduce user support + queries. +1. **Using photos and a clearer typographical hierarchy** to make the area + alert emails easier to scan, especially when there are multiple new + problems in your chosen area. +1. Letting our council partners and international reusers maintain their brand + image across the website and emails, by **easily customising the logo and + colour scheme** of emails sent to their users. + +The best way to see how we’ve improved FixMyStreet’s emails is to give +FixMyStreet a try, and receive the emails yourself! Try +[reporting a new problem near you](https://www.fixmystreet.com), or +[subscribing to email alerts about new reports in your area](https://www.fixmystreet.com/alert). + +If you’d like to know *how* we implemented some of our more technical changes, +read on… + +## Attaching the static map image + +One thing that we wanted to include in the email was a map showing the location +of the report (be that the one you've just made, or the one you're receiving an +alert or questionnaire for). The map that you see on the website is made up of +many 256x256px tiles stitched together in HTML/CSS to appear as one smooth map, +with separate pin images superimposed in the correct location; for the email, +we needed just one image covering whatever portion of the map was necessary, +plus the pin. + +{: .r } + +This is the point at which FixMyStreet's +[progressively-enhanced](https://gdstechnology.blog.gov.uk/2016/09/19/why-we-use-progressive-enhancement-to-build-gov-uk/) +base came in very useful. If JavaScript does not work for whatever reason, the +site has always displayed an alternative: a small map made up of four tiles in +a square, with the pin located appropriately on top. It also makes sure that +the pin location (the point at the bottom middle of the pin) is contained +within the central half of the tiled map (the red dashed area in our image +here), so that there's no chance the pin overlaps the edge of the map. + +This was perfect for the image to be used in an HTML email. The new static map +function fetches the same data used by the front end, requests the tile data +for those four tiles, stitches them together in one 512x512px image, composites the +pin on top in the correct location, and then cuts off the bottom 128 and top 64 +pixels – as the pin's central location means those will always be pin-less. The +image is then shrunk to 310px in width, providing output that looks like this, +appearing in the top right hand corner of the email: + +<div style="text-align:center"> +<img src="/assets/posts/html-email-map.png" alt=""> +</div> + +## Inline images + +The new HTML email - whether they're reports, alerts, confirmations or +questionnaires - normally have upwards of three images: the static map image, +an image of the report (or repots for alert emails), and a site logo. We wanted +to include all these images within the email itself, rather than use remote +images, as due to spam many people have remote images switched off. + +We also wanted to keep things as simple as possible when including images in +the email templates. Inline images in HTML emails use an image source of +`cid:UNIQUE_ID` within the HTML (or CSS), and then give a particular attachment +of the email the same `UNIQUE_ID` in its Content-ID header. Lastly, we may +potentially also have normal attachments. + +We settled on an `inline_image` function in the template, which can be +provided with either a path to an image file (for the static logo), or a +function that returns image data and a content type (for the static map or +report image). As the email template is processed, each call to this function +generates a unique ID for the image and stores the information in a list to be +added after the plain text/HTML email parts. + +There are a variety of ways to attach images to an email. Content types +are used to identify what is contained by each part of the email. As well as +the various types of image, and text/plain and text/html for the text and HTML +parts, there are various containers: `multipart/alternative` as a container for +parts containing the same information in different formats, `multipart/related` +as a container for parts that are linked in some way, and `multipart/mixed` as +a general container. + +The setups we decided on were as follows: + +* If the HTML email has no inline images or other attachments, then we create + a `multipart/alternative` email, containing the two textual parts. + +* If the HTML email has inline images but no other attachments (the most common + case), we create a `multipart/related` email, its first part being the + `multipart/alternative` as above, the subsequent parts the inline images. + + An alternative here would be to create a `multipart/alternative` email, with its + first part being text, and its second part being `multipart/related` containing + the HTML part and the inline images. This would mean that an email client + that didn't support HTML email might only see the textual part and not any of + the images. If your inline images were not 'important' to the email (e.g. an + email footer signature) then this might be a way to go; we thought that the + map and image should be visible to all users if possible. + +* If the HTML email has attachments but no inline images, + we create a `multipart/mixed` email, its first part being the + `multipart/alternative` as above, the subsequent parts the attachments. + +* If the HTML email has both inline images and other attachments, then + we create a `multipart/mixed` email. Its first part is a `multipart/related` + email (that again contains `multipart/alternative` and the inline images), + and then its other attachments form the later parts of the mixed email. If + you imagine the parts as envelopes with brackets, it would look like this: + + ``` + multipart/mixed ( + multipart/related ( + multipart/alternative ( + text/plain + text/html + ) + image/jpeg + image/png + ) + application/pdf + ) + ``` + +As part of this work, I discovered that the [oldest open +GitHub](https://github.com/rails/rails/issues/2686) issue of the Rails +framework was related to this topic – if you used Rails to create an email +containing both inline images and normal attachments, the normal attachments +were not accessible to most email clients (that support HTML email) as they had +embedded all the normal attachments inside a `multipart/related` part. I have +submitted a [pull request](https://github.com/rails/rails/pull/26445) to fix +this structure, which I hope will be accepted in some way. + +## Templating + +This work was also a good opportunity to move some text generation out of +some code into the templates (necessary because the text being generated +now needed some HTML around each entry), for all the alert emails. + +## Testing + +Finally, this post wouldn’t be complete without a few words about +email testing. + +Any of you who have built HTML emails in the past will agree that they are like +taking a time machine back to web development in the mid 1990s. Email clients +like Outlook, Gmail, and iOS Mail have dramatically different capabilities and +ways of rendering the same email code. + +One way of avoiding cross-client complications is to keep your HTML layout as +simple as possible; maybe add an `<img>` tag in for your site logo, some +`<strong>` or `<em>` tags for emphasis, and call it a day. + +But our plans for FixMyStreet required much more complex email layouts than +this. It was a hard requirement that the details of the report (the map, the +name, the photo) in our questionnaire emails, were given equal priority to the +introductory text and the call to action buttons. The requirements led us to a +two-column layout, which, inevitably, required two or three layers of nested +table cells. (Remember, we’re in our 1990s time machine!) + +To help us test these layout changes in all the required email clients, +we used [Litmus](https://litmus.com), which is like +[Browserstack](https://browserstack.com) but for emails. +You send a single email to Litmus, and it renders that email in +dozens of different email clients, grabbing screenshots of each one, and +presenting them to you in a handy dashboard. Building HTML emails this way +still requires a good deal of trial and error, and obscure CSS knowledge, but +at least with Litmus, the process of iterating on your design is made as short +and fast as possible. It’s an expensive service, but well worth the cost for +the peace of mind that your new layout works in even the most uncooperative +email clients. + +<div style="text-align:center"> +<img src="/assets/posts/html-email-litmus.png" alt=""> +</div> + +With the help of our Litmus checklists, we made a bunch of unintuitive +discoveries, including: + +* Using `<th>` rather than `<td>` elements, so that the Android 4.x mail client + can give them `block` styling in the small screen media query. +* Defining our font settings on every table cell, rather than simply inheriting + `font-family` from the `body`, so that sans-serif fonts are used in Outlook, + rather than Times New Roman. +* Using a three-column wrapper table to create a 620px centred content area + that also shrinks down on narrow screens. (Outlook doesn’t like + max-width, so this is the simplest alternative.) +* Enforcing a sensible (500px) min-width for the main content area, + on clients that don’t support media queries. +* Using giant borders on `<a>` elements, to make them into Outlook-friendly + buttons without resorting to less accessible alternatives like images. +* Aligning images with the deprecated `align` attribute, rather than CSS floats. +* Applying the email background colour to a wrapper element inside the `body`, + and thus leaving the `body` to keep its default white background, so that + replies sent from Outlook (which inserts the reply message *inside* the body + of the original message) will have a white background. diff --git a/docs/_posts/2016-12-05-v2.0-email-domain-name-validation.md b/docs/_posts/2016-12-05-v2.0-email-domain-name-validation.md new file mode 100644 index 000000000..1794ad77a --- /dev/null +++ b/docs/_posts/2016-12-05-v2.0-email-domain-name-validation.md @@ -0,0 +1,38 @@ +--- +layout: post +title: Version 2.0 – email validation +author: matthew +--- + +When someone enters an email address on FixMyStreet, we try and do some simple +validation before sending a confirmation email, to catch typos and the like. We +do the following: + +* Check the address is correctly formed (e.g. that it contains an @ sign and a + domain, and doesn't include irregular characters, etc); +* Check that the email address's domain is a valid top-level domain (TLD) (it's + not going to be delivered anywhere if it's not); and +* Check that the email address's domain has a valid DNS entry (so we'll be + able to try and deliver an email there). + +The middle point means using a Perl module, +[Net::Domain::TLD](https://metacpan.org/pod/Net::Domain::TLD), that contains a +list of all valid TLDs. This list has grown since we last updated it, and we +found ourselves incorrectly rejecting an email address that was perfectly valid +(a @domain.cymru address). It was at least easy to update our version of +Net::Domain::TLD to ensure that our list of top-level domains is current. + +## Changing your email address + +The confirmation email is itself another form of validation, and the only one +that can actually confirm that the user entering the email address has access +to that email inbox. + +Version 2 of FixMyStreet now allows users to change their own email address, +which threw up a complication – we can’t update the user’s table to their new +email address until it’s been confirmed. + +So we have to store their old email address with the token that is sent to the +new one. When the new address has been confirmed, we deal with it differently +depending on whether or not that address already exists in the database. If it +does, it's a matter of merging the two accounts. diff --git a/docs/_posts/2016-12-06-v2.0-multi-select.md b/docs/_posts/2016-12-06-v2.0-multi-select.md new file mode 100644 index 000000000..aaa5c9fa9 --- /dev/null +++ b/docs/_posts/2016-12-06-v2.0-multi-select.md @@ -0,0 +1,46 @@ +--- +layout: post +title: Version 2.0 – Multi-select dropdown jQuery plugin +author: zarino +--- + +There are several types of 'list page' on FixMyStreet: for example, when you +view all reports from a specific body, or when you log into your account to see +reports you have made. + +For quite a while now, users have been able to filter these by state (eg +'fixed' or 'in progress') or category (eg 'pothole', 'streetlight', etc), but a +recent suggestion from Oxfordshire County Council prompted us to look again at +this functionality and improve it. + +So now it's simple to filter by multiple states or categories: want to see all +unfixed potholes? All streetlights or flytipping reports that are fixed? No +problem. + +HTML's `<select multiple>` is not the nicest tool in the box, especially when +it comes to actually selecting multiple options. It usually involves holding +down a key, but that key is different depending upon your operating system. + +So I looked at existing JavaScript plugins and finding nothing suitable created +a very simple jQuery plugin which is available at +[https://github.com/mysociety/jquery-multi-select](https://github.com/mysociety/jquery-multi-select). + +By default, it converts a multiple select into a dropdown with checkboxes: + +<img src="/assets/posts/multi-select-people.png" alt=""> + +The options can be wrapped within a container to stop the dropdown overlapping other content: + +<img src="/assets/posts/multi-select-line-wrap.png" alt=""> + +With options in the constructor, you can specify text for when all/no options +are selected, and specify groups of options as presets that will be listed at +the top of the dropdown: + +<img src="/assets/posts/multi-select-groups.png" alt=""> + +On FixMyStreet, we have used this for both the state and category filters on list pages: + +<img src="/assets/posts/multi-select.png" alt=""> + +Let us know if you find the plugin useful! diff --git a/docs/_posts/2016-12-07-v2.0-new-user-system.md b/docs/_posts/2016-12-07-v2.0-new-user-system.md new file mode 100644 index 000000000..f9c6bdda1 --- /dev/null +++ b/docs/_posts/2016-12-07-v2.0-new-user-system.md @@ -0,0 +1,69 @@ +--- +layout: post +title: Version 2.0 – New user system +author: matthew +--- + +Version 2 launches a new user system for admins, including more granular +permissions and a variety of new features. + +## Logging in + +Admins can now use the same login system as the main front end. We highly +recommend running your site over HTTPS in order to keep credentials secure; +[LetsEncrypt](https://letsencrypt.org/) can supply free 3-month certificates. + +The main admin user type is the 'superuser'; a user with this flag set (which +can be done in the database, or by running the `createsuperuser` command) has +full access to all areas of the admin, just like admins under the old system. +By default, these are the only types of user allowed to log in to `/admin`. (It +is possible to change this, e.g. in the UK, non-superuser admin users +associated with a body can log in to `/admin` on their own cobrand.) + +## User permissions + +<img class="r" src="/assets/posts/admin-user-permissions.png" alt=""> + +Users associated with a body (called 'body users') can be given a variety +of different feature-based permissions; the screenshot here shows the list +of different options. + +* **Categories:** You can associate a user with a list of categories, which +e.g. pre-selects those categories when the user visits the All Reports page. + +* **Response priorities:** This allows you to set a list of different +priorities for a body, or again for a particular category in a body, +letting you note different priorities for different reports. + +* **Response templates:** You can create and edit templates associated with +your body, or with a particular category in that body, and then when leaving an +update you can select one of these templates to allow easy updating of reports. + +* You can give a user access to a front-end **report 'inspect' view**, which +lets a user edit a report's category, state, or location. If the category +change moves the report to a different body, it will be re-sent. Alternatively, +a user can be given *only* category edit or priority edit permission. +Here is a screenshot of the top of an inspect form view: + +<img src="/assets/posts/report-inspect.png" alt="The inspect form lets you change category, state, report location, and so on."> + +## Create as another/body + +This permission gives a user the ability to create a report or update on behalf +of a body, or as another user. We envisage this being useful in a body's +contact centre, where they receive a report over a phone and enter it into +FixMyStreet as that user. Below is a short animation showing this in action on +the Oxfordshire cobrand of FixMyStreet.com: + + + +## Shortlists + +<img class="l" src="https://cloud.githubusercontent.com/assets/739624/19122469/7fa927ba-8b22-11e6-8193-ef20d9ce496e.png" alt=""> +A user with the shortlist permission gains a shortlist button on each report; +clicking this adds the report to your own personal shortlist of reports, which +you can view in a section of Your Account. This may be useful for an +'inspector' type of admin user, who wishes to compile the day's list of +reports before going out and investigating them. You can also see if a report +is on someone else's shortlist, and take it off them if you need to. +<br style="clear:both"> diff --git a/docs/_posts/2016-12-08-v2.0-area-highlighting.md b/docs/_posts/2016-12-08-v2.0-area-highlighting.md new file mode 100644 index 000000000..5cd424fbd --- /dev/null +++ b/docs/_posts/2016-12-08-v2.0-area-highlighting.md @@ -0,0 +1,42 @@ +--- +layout: post +title: Version 2.0 – area highlighting +author: matthew +--- + +FixMyStreet's report pages for a particular body have always highlighted the +area of the body covered, by fetching the KML shape from the associated MapIt +and plotting it on a map. In version two we have made it look much nicer: + +<img src="/assets/posts/area-highlighting-before.jpeg" align="left" style="max-width:49%"> +<img src="/assets/posts/area-highlighting-after.jpeg" align="right" style="max-width:49%"> + +Until now we have coloured in the shape in a light orange, as pictured above +left. This is fine for an overall view of a body, but if you want to zoom in to +a particular area, it makes it quite hard to see the underlying map. + +It would be preferable if every part of the map *except* the body's area could +be shaded, highlighting the correct area without losing any clarity from the +part you're interested in, as in the new image above right. + +## Polygon make-up + +Polygons in KML and other similar formats are made up of multiple linear rings +– the first ring is the polygon's outline, and any subsequent entries are holes +within the polygon. You can therefore 'invert' a polygon by having the outer +ring be something surrounding the polygon, then having the polygon be a hole +within that. + +So the way we have implemented this in our new version is to fetch the area +from [MapIt](https://mapit.mysociety.org/), but before plotting it preface it +with a giant rectangle covering the whole world. (Thankfully, any holes within +the original shape are inverted too, though that is implementation dependent +and it would have been a lot harder if they weren't!). + +There were a couple of issues along the way. MapIt may return either a polygon +or multiple polygons, so we needed to deal with each differently (we turn +multiple polygons into one polygon, which also thankfully Just Works). And +OpenLayers has a hardcoded maximum pixel co-ordinate for SVG rendering, dating +from [2007 and an issue in Firefox 2](https://github.com/openlayers/ol2/commit/ba5b664a2f34691574e149c8d48ab4deb33c532b). +We hope that any such issues have been fixed by now. We haven't had any reports +of crashes yet, anyway... diff --git a/docs/_posts/2016-12-09-v2.0-html5-history.md b/docs/_posts/2016-12-09-v2.0-html5-history.md new file mode 100644 index 000000000..c2376fd80 --- /dev/null +++ b/docs/_posts/2016-12-09-v2.0-html5-history.md @@ -0,0 +1,62 @@ +--- +layout: post +title: Version 2.0 – HTML5 History +author: matthew +--- + +We've [mentioned +before](https://www.mysociety.org/2015/12/18/photo-upload-and-progressive-enhancement-for-fixmystreet/) +that FixMyStreet is built on a progressively-enhanced base, a concept explained +neatly [in a nice blog post from the UK +government](https://gdstechnology.blog.gov.uk/2016/09/19/why-we-use-progressive-enhancement-to-build-gov-uk/). + +This means that e.g. the slippy map is, underneath it all, an old-style +server-side image map that works out where you click; URLs are all shareable +and pages are functional even if JavaScript is not available; the front page +loads quickly and doesn't need to preload an entire application. + +None of this means that we don't use or like JavaScript, however. This post is +about adding JavaScript to FixMyStreet to provide a quicker experience to users +looking at reports on our map. + +When viewing a [list of +reports](https://www.fixmystreet.com/reports/Oxfordshire) on FixMyStreet, you +might want to look at a few reports one after the other, much as you can on a +Google Maps or OpenStreetMap results screen. + +So now, when you click a report in the list or a pin on the map, the report +page is pulled in via JavaScript. This updates the page, pin and URL in situ, +rather than loading a new page. Other pins remain visible, plus the page feels +(and probably is) a bit quicker as the page header doesn't reload, and it is +easy to switch back to the list view. + +This feature uses HTML5's History API, ie. `pushState` and `popState`, to +update the URL as the page changes. That means that when you share it, +the page that loads will always be the one that the user intended it to be. + +This improvement did not come without problems, however. Overriding the +browser's own behaviour when it comes to history and navigation means you have +to think carefully, and I'm sure we'll need to make further refinements to +ensure that everything works as the user would expect. + +There were small issues: for example, `pushState` stores the document title at +the point when it's called, for the 'Back' button list, meaning we had to make +sure any title change happened after that. Some browsers have a `popstate` on +page load, which can cause an issue if you assume it's only fired due to +history events. + +There was the complication of needing to tell the difference between someone +clicking back to the 'initial state' of the page, and an internal hashchange or +other less crucial event – as well as using a `replaceState` on page +load, we store the original URL and title for use in such a situation. + +Then we forgot that the code would be running on `/reports` lists as well as +`/around` which led to some confusion until we realised what was happening! And +of course, you have to make sure everything JavaScript-wise is set up +appropriately for content brought in via JavaScript. + +We also used `pushState` in the new report process, to update the URL as you +select the report's location, and on list pages when you select one of the +filters or sort. This has worked well, and is certainly much more preferable to +the 'hash-bang' technique used by some sites in previous years (and still now), +which is reliant on JavaScript functioning. diff --git a/docs/_posts/2016-12-13-v2.0-improved-forms.md b/docs/_posts/2016-12-13-v2.0-improved-forms.md new file mode 100644 index 000000000..efad888b7 --- /dev/null +++ b/docs/_posts/2016-12-13-v2.0-improved-forms.md @@ -0,0 +1,77 @@ +--- +layout: post +title: Version 2.0 – Improved forms +author: matthew +--- + +The new release of FixMyStreet includes a number of improvements to various +forms on the site. In this post, we will take a brief look at the notable +changes. + +## Public reporting form + +<img class="r" src="/assets/posts/new-reporting-form.png" alt=""> + +This form has been rejigged, in order to more obviously split out details that +will be public (e.g. photos, details) from those that will not be published on +the site (e.g. the user's email address and phone number). The category +selector has also moved to the top, and if the category chosen requires the +display of extra questions or information (e.g. through Open311 attributes or a +custom built asset layer), they will be shown immediately. + +If a report is made in an area that is covered by more than one body, the +category the user selects will normally dictate which one the report is sent +to. Now, when the category is selected, we update the list of bodies given at +the top of the report page, if we know that the report will be sent there. + +Talking about custom built asset layers, this is a good place to show how the +FixMyStreet codebase can be put to other uses, with a bit of development. + +Angus Council in Scotland provide a WFS layer (that is, vector format geographic +information) containing the locations of all their streetlights, which +we display if the street lighting category is selected within Angus on FixMyStreet. + +Importantly, it can display which lights Angus already knows are broken. If the +user is able to identify precisely which street light is affected, they can +click on it. But picking a street light isn't mandatory: we don't want to put +people off who aren't certain, or who are unable to select an individual light. +You can see an example of what this looks like on the Angus cobrand of +FixMyStreet.com below. + +<img src="/assets/posts/angus-streetlights.png" alt="fix.angus.gov.uk shows individual street lights when the Street lighting category is selected."> + +<br style="clear:both"> + +## Admin report editing + +<img class="r" src="/assets/posts/admin-report-form-before.png" alt=""> + +Previously, the form for administrators to edit a report was functional, but +certainly nothing more than that! Due to the work we've done on the new user +system, more admin users may well be accessing this form in future, and so +we've taken the opportunity to make it much tidier. + +It now looks much more like the front end of the site. We've added a map that +lets you move the location of the report, tidied up the various functions an +admin can perform, and so on. If a category change means the report should have +been sent to a different body, it will be re-sent. + +<img src="/assets/posts/admin-report-form-after.png" alt=""> + +## Admin category editing + +This was a historical oddity, in that the Add category form and the Edit +category form were completely separate creatures, though both contained the +same fields, and were used for basically the same purpose: + +<img src="/assets/posts/admin-category-add-before.png" alt="" class="l" style="max-width:45%"> +<img src="/assets/posts/admin-category-edit-before.png" alt="" class="r" style="max-width:45%"> +<br style="clear:both"> + +<img src="/assets/posts/admin-category-edit-after.png" alt="" class="r"> +You can see how they looked different in the screenshots above. They now share +an HTML template, which also makes it easier for us to update should it need +changing in future. + +<br style="clear:both"> + diff --git a/docs/_posts/2016-12-14-v2.0-exif-rotation-preview.md b/docs/_posts/2016-12-14-v2.0-exif-rotation-preview.md new file mode 100644 index 000000000..8cbb9549d --- /dev/null +++ b/docs/_posts/2016-12-14-v2.0-exif-rotation-preview.md @@ -0,0 +1,34 @@ +--- +layout: post +title: Version 2.0 – EXIF rotation in JavaScript +author: matthew +--- + +FixMyStreet has had a nice multiple image uploader since version 1.8. This uses +multiple `input type=file` fields, progressively enhanced to add +drag'n'drop, image preview, and uploading in the background whilst you fill in +the rest of the form. + +<img src="https://cloud.githubusercontent.com/assets/739624/11398421/76c7c856-9378-11e5-9c25-971de271bd0c.gif" alt=""> + +In version 1.8.4, we +[patched](https://github.com/mysociety/fixmystreet/commit/4023ee86708f825b64e5f10bef31519ebfb0c5f8) +the third party library we use, [dropzone](http://www.dropzonejs.com), to +correctly orient photos in the image preview. We did this by including a +cut-down version of [exif-js](https://github.com/exif-js/exif-js) to read in +the EXIF orientation data, and then make sure we rotated the image as +instructed in JavaScript before drawing the thumbnail preview. The rotation was +accomplished by moving the image so its centre was over (0,0), rotating the +appropriate amount, and then re-translating it back. + +<img src="https://cloud.githubusercontent.com/assets/8081877/16151581/b808b924-3495-11e6-8573-5e9828b3fa49.png" alt="" align="right" hspace="8"> + +For this new version, we had a different bug to fix. If the user had uploaded a +picture, submitted the form, and was shown the form again due to a server side +error of some sort (some validation not caught by client-side validation, for +example, or because you were logging in during the reporting process), the +image for the preview was then being loaded from the server (where it had +already been uploaded), not the client, and not displaying. We +[patched the exif-js library](https://github.com/mysociety/fixmystreet/commit/7684ee064c717722003e920baed08d0230eb0d9e)., +Now, if it is given a URL rather than a data: string, it will go off and fetch +the image so that it can read out the orientation data. diff --git a/docs/_posts/2016-12-15-v2.0-faster-tests.md b/docs/_posts/2016-12-15-v2.0-faster-tests.md new file mode 100644 index 000000000..2e55cf7b3 --- /dev/null +++ b/docs/_posts/2016-12-15-v2.0-faster-tests.md @@ -0,0 +1,65 @@ +--- +layout: post +title: Version 2.0 – testing improvements +author: matthew +--- + +FixMyStreet has a large and hopefully comprehensive test suite that runs +through all aspects of the codebase, checking everything is working. This +makes it easier to change code and add new features, safe in the knowledge +that any breakages will be quickly highlighted. + +## Speeding up the tests + +Every time someone commits code to our GitHub repository, or opens a pull +request, the tests are automatically run for us by Travis CI. We're alerted to +success or failure with little green ticks or red crosses on GitHub, and by +notice in IRC. + +The tests seemed to have slowed down considerably in recent times, but we +couldn't identify any changes at the FixMyStreet side which might have caused +this. + +However, there had recently been some spam scraping of +[Gaze](https://gaze.mysociety.org/), our web service that provides population +density information to FixMyStreet (so that e.g. the [maps can try and guess an +appropriate zoom +level](https://www.mysociety.org/2012/08/14/mysociety-design-tips-how-we-choose-the-best-map-zoom-level/), +and so alerts can try and guess an appropriate radius), and rate limiting had +been added to try and help combat it. + +Dave spotted that this was being triggered by FixMyStreet test runs, leading to +pauses as the suite waited for the rate limiting to ease. Thankfully, all Gaze +calls were being routed through one function (that had been created in order to +cope gracefully with a Gaze failure) and so it was a simple matter for this +function to be stubbed out if being run as part of a test. + +[Before](https://travis-ci.org/mysociety/fixmystreet/builds/143317849): + + +[After](https://travis-ci.org/mysociety/fixmystreet/builds/143325800): + + +There are many tests that still rely on the internet (e.g. for some MapIt +lookups) and eventually it would be good to get to the point where they are all +stubbed out and the test suite can run completely offline, probably even more +quickly. + +## Multiple test running + +When running the tests, the suite creates a test database (in PostgreSQL terms, +it actually creates a temporary cluster) so that anything it does won't affect +your development database. Theoretically, this means you should be able to run +the test suite multiple times simultaneously – perhaps it's doing a full run, +but you want to try and fix (and retest) the first error while it carries on. +However, this was not working, and after some investigation it turned out that +each run was creating (and overwriting) a test configuration `.yml` file, which +meant the existing runs got all confused. Adding a process ID to the test +configuration file meant that each run is independent and can successfully +coexist with each other. + +## Keystroke saving + +Lastly, you used to have to run the full suite with `bin/run-tests t`, but now +if you run `bin/run-tests`, it will assume you meant `t`. A small thing, but it +might save a few seconds over the years. ;-) diff --git a/docs/_posts/2016-12-16-v2.0-javascript-improvements.md b/docs/_posts/2016-12-16-v2.0-javascript-improvements.md new file mode 100644 index 000000000..fa82fef6f --- /dev/null +++ b/docs/_posts/2016-12-16-v2.0-javascript-improvements.md @@ -0,0 +1,123 @@ +--- +layout: post +title: Version 2.0 – JavaScript performance +author: matthew +--- + +The JavaScript on FixMyStreet has gradually evolved over many years (we +launched in 2007, remember!), and while we were working on other features in +this area (such as HTML5 History) it was a good opportunity to tidy up the +JavaScript, making it clearer and simpler. Below I'm going to go through most +of the steps I took, not necessarily in the order I took them, which hopefully +might prove useful to your own websites. And there are exciting pictures at the +end, I promise! + +It also let us add the Content-Security-Policy header to FixMyStreet, which is +a method in browsers to prevent cross-site scripting (XSS), clickjacking and +other code injection attacks by specifying the valid sources for script +execution. + +## Separate scripting and styling + +When you report an issue on FixMyStreet on mobile, the map `/around` page is +full-screen to make reporting easier. This was being done in JavaScript using +some jQuery `css()` calls when setting up (or ending) the mobile view. It was +straightforward to move this CSS to a `mobile-reporting-map` class and have the +JavaScript do no more than add or remove this class, making both parts clearer. +(Later during this process, I also added an `only-map` class to prevent the map +being scrolled until it is clicked, when the rest of the form can then be +shown.) + +Similarly, the JavaScript was requisitioning the desktop-width green banner to +provide a different mobile banner; updating this to use a separate mobile +banner made the code clearer and shorter. + +## Tidy up the JavaScript set up + +The main setting up of our JavaScript was taking place, for historical reasons, +in two files, both confusingly called `fixmystreet.js`. Whilst in the future we +may want to split this up into separate files more tailored to the particular +pages where the code is used (though our main use of JavaScript is our map +page, which already has its JavaScript separate), for now it made most sense to +combine these in one file, and tidy up all the setup functions into a list of +feature-based functions each called in turn on page load. + +We were bundling a copy of Modernizr (for media query detection) that contained +html5shiv and yepnope. But html5shiv is only needed in old versions of Internet +Explorer, and yepnope is only used on FixMyStreet's front page (to try and +preload the map page JavaScript mentioned above), so I could move html5shiv +into an IE conditional comment, and included yepnope.js only on the front page, +reducing Modernizr to only Modernizr itself. + +## Move vital JavaScript as early as possible + +Now that I had a `mobile-reporting-map` class, I wanted this class activated +as soon as possible as the page is loading, not only when the document had been +parsed. There were also a couple of site-wide variables, `page` (the type of +page, e.g. `around` or `new`), and `cobrand` (the branding of the site you're +on). Lastly, I wanted to be able to set a class on the document if JavaScript +was activated, so that CSS using that class would be instantly active, +preventing a flash of style change or content. + +To achieve all this, I created a `header.js` file that performed the above +three tasks, setting a class on <html>, setting two variables on our +global `fixmystreet` variable, and if we're on a small width (using Modernizr) +and perhaps a map page, setting the appropriate classes. I then minimized and +inlined this script in the header of each page, so that we don't have to wait +for any external script to load. + +## Move JavaScript to the end of the HTML, remove inline JavaScript + +As [recommended by Jake Archibald +here](https://www.html5rocks.com/en/tutorials/speed/script-loading/), I moved +all the JavaScript to the end of the HTML. To make this easier, and also to +make adding a Content-Security-Policy header easier, I removed all the inline +JavaScript from FixMyStreet (I didn't think FixMyStreet had that much inline +JavaScript, and it didn't, but it still had more than I had remembered!). This +was most commonly being used to set up some JavaScript variables with +server-side data, so the easiest way to replace this was to place this data as +attributes on HTML elements (preferably semantically related elements), and set +up the JavaScript variables in an external script. + +## Minify JavaScript + +I didn't want to make this mandatory, as we have done with SCSS/CSS, but I +wanted the option available, so I added an option to our templating code that +means it will prefer an .auto.min.js file in preference to a .js file of the +same basename. This lets you compile your JavaScript in a deploy process, for +example, should you wish to. We do this with the standard Closure Compiler from +Google; I haven't yet been brave enough to try and check/get the JavaScript +working with the advanced option of the Closure Compiler :) + +## Activate Content-Security-Policy + +The Content-Security-Policy header lets you specify domains from which +JavaScript will run, plus lets you choose to run inline JavaScript either en +masse or only if you provide a specific unique nonce ID in the <script> +tag that matches the same ID in the CSP header. That latter option is how we +kept our inline header JavaScript running without having to externalise it +again. + +"nonce" was only added in the second version of the CSP spec, so you may note +our header also specifies `unsafe-inline`. Any browser that supports version 2 +will ignore this when it sees the nonce header, but it is needed in order for +the inline script to still run in any browser only supporting version 1. + +## Conclusion + +In Google Page Speed Insights, with manual minification of the main JS files, +this moves the front page from 68/84 to 85/92ish (filmstrip from +webpagetest.org, top is live site, bottom is my dev site): + +[](https://cloud.githubusercontent.com/assets/154364/17670115/58b741f6-6308-11e6-9510-82c23fbd5c35.png) + +These are requests from the US: most of the initial delay is in that initial +download. Now here's a report page going from 58/77 to 85/86ish ("ish" because +e.g. live site will have analytics that my dev site doesn't): + +[](https://cloud.githubusercontent.com/assets/154364/17670122/680c9a16-6308-11e6-9bfb-721272ff1f71.png) + +Due to page elements displaying more quickly than might previously have been +expected, there are still a few things to tidy up. For example, when an element +displays before a bit of JavaScript kicks in and makes it look slightly +different... but hopefully nothing major. diff --git a/docs/_posts/2016-12-16-v2.0.1.md b/docs/_posts/2016-12-16-v2.0.1.md new file mode 100644 index 000000000..629e25e61 --- /dev/null +++ b/docs/_posts/2016-12-16-v2.0.1.md @@ -0,0 +1,27 @@ +--- +layout: post +title: Version 2.0.1 +author: matthew +--- + +<div class="r" align="right"> +<a data-flickr-embed="true" href="https://www.flickr.com/photos/s2art/189574055/" title="drowning"><img src="https://c8.staticflickr.com/1/67/189574055_37e836feaa.jpg" width="500" height="375" alt="drowning"></a><script async src="//embedr.flickr.com/assets/client-code.js" charset="utf-8"></script> +</div> + +Today we have released **version 2.0.1** of FixMyStreet, a bugfix release. +The issues fixed are: + +* Chrome 55 changes the way it handles mouse/touch events, [unifying them as + pointer events](https://developers.google.com/web/updates/2016/10/pointer-events), + and OpenLayers, the JavaScript mapping library we use, needed patching to + deal with this. Thanks to the Chrome team for helping debug this. +* strftime output (e.g. used to display dates) was sometimes being double + decoded, giving incorrect output in some languages. +* If a category was filtered, it was not being properly carried through to the + new report form. +* The body/area inputs in the admin were being fixed to a too-small height. + +Some bugfixes and improvements have also been made to the admin inspector +report view, along with a couple of development improvements – the tests coping +better if run on a network that intercepts NXDOMAINs, and a better way of +showing the git version in the admin config. diff --git a/docs/_posts/2017-02-03-v2.0.2.md b/docs/_posts/2017-02-03-v2.0.2.md new file mode 100644 index 000000000..0047f8120 --- /dev/null +++ b/docs/_posts/2017-02-03-v2.0.2.md @@ -0,0 +1,37 @@ +--- +layout: post +title: Version 2.0.2 +author: matthew +--- + +<div class="r" align="right"> +<a data-flickr-embed="true" href="https://www.flickr.com/photos/amyeanderson/29212751225/" title="Lake Annecy"><img src="https://c1.staticflickr.com/9/8503/29212751225_67b5579e60.jpg" width="500" height="453" alt="Lake Annecy"></a><script async src="//embedr.flickr.com/assets/client-code.js" charset="utf-8"></script> +</div> + +Today we have released **version 2.0.2** of FixMyStreet, a bugfix release along +with some new admin improvements. + +The main bugfixes are to mark two missing strings for translation, to make sure +email is lowercase when signing in via `/auth`, and to make sure a language +subdomain can be included in calls to `base_url_for_report`. There are also +some CSS fixes to the homepage and an improved print layout for report list +pages. Fixes to the admin interface include fixing filtering on the shortlist +page, and fixing the ‘save with public update’ toggle. + +For international users, there have been a few string renames, but we have +maintained translations as the concept remains unchanged, so please only feel +free to update if you consider it necessary: + +* ‘unable to fix’ has been renamed to ‘no further action’ +* ‘marked as a duplicate report’ has been changed to ‘closed as a duplicate report’ + +An offline fallback page has been added using appcache, which inspectors can +use for offline updating of reports in their shortlist. + +Admin improvements include allowing response templates to be associated with a +state (so that template is used by default when the state is changed), allowing +shortlist addition/removal from report lists, and shortlist reordering. Users +with a new permission can see the body user who left a `contribute_as_body` +report or update. + +[Full changelog](https://github.com/mysociety/fixmystreet/releases/tag/v2.0.2) diff --git a/docs/_posts/2017-03-31-v2.0.3.md b/docs/_posts/2017-03-31-v2.0.3.md new file mode 100644 index 000000000..cfbe75c52 --- /dev/null +++ b/docs/_posts/2017-03-31-v2.0.3.md @@ -0,0 +1,33 @@ +--- +layout: post +title: Version 2.0.3 +author: matthew +--- + +<div class="r" align="right"> +<a data-flickr-embed="true" href="https://www.flickr.com/photos/mssuziecue/2637008185/" title="Untitled"><img src="https://c1.staticflickr.com/4/3071/2637008185_6c4bd4d6a0.jpg" width="440" height="500" alt="Untitled"></a><script async src="//embedr.flickr.com/assets/client-code.js" charset="utf-8"></script> +</div> + +Today we have released **version 2.0.3** of FixMyStreet, +a bugfix release along with some other improvements. + +The map on a mobile report page can now be made full screen, and if you are +using Google Maps you can supply a custom map styling. There's also now a +loading indicator whilst data is being fetched for the map. + +Various missing translations have been added, for moderation, social login, +and offline usage. + +We've upgraded our email sending to deal with issues sending with SSL, dealt +with IE11 caching report Ajax calls too aggressively, and with Safari 5/ +Android 4 not showing our questionnaire answer buttons. + +Performance improvements include moving admin-related JavaScript to its own +file (so normal users don't need to download it), and reducing the amount of +disk stats the code performs. + +Lastly, all the test suite can now run offline, and amusingly I found a bug in +a test that only happened if the test was run c. 55 hours before daylight +savings time began :) + +[Full changelog](https://github.com/mysociety/fixmystreet/releases/tag/v2.0.3) diff --git a/docs/_posts/2017-04-13-v2.0.4.md b/docs/_posts/2017-04-13-v2.0.4.md new file mode 100644 index 000000000..709e0aa01 --- /dev/null +++ b/docs/_posts/2017-04-13-v2.0.4.md @@ -0,0 +1,27 @@ +--- +layout: post +title: Version 2.0.4 +author: matthew +--- + +<div class="r" align="right"> +<a data-flickr-embed="true" href="https://www.flickr.com/photos/pieczonysnieg/8635754135/" title="11"><img src="https://c1.staticflickr.com/9/8537/8635754135_4aa39ac521.jpg" width="338" height="500" alt="11"></a><script async src="//embedr.flickr.com/assets/client-code.js" charset="utf-8"></script> +</div> + +Today we have released **version 2.0.4** of FixMyStreet, a bugfix release along +with a couple of other improvements. + +The bugfixes are updating our Facebook library so that Facebook login works +again, stopping an error if you had a devolved body and its contact both +lacking a send method, and an issue in the multi-select front end if you had +characters such as brackets in your category name. + +Strangely, while the backend has always allowed multiple email addresses for a +contact, separated by commas, the admin interface hasn't allowed them to be +entered - this has now been corrected. + +Lastly, body pages under `/reports` will now limit the reports shown to those +within the visible map, which should make moving and zooming around much more +intuitive. + +[Full changelog](https://github.com/mysociety/fixmystreet/releases/tag/v2.0.4) diff --git a/docs/_posts/2017-07-18-v2.1.md b/docs/_posts/2017-07-18-v2.1.md new file mode 100644 index 000000000..f3579f813 --- /dev/null +++ b/docs/_posts/2017-07-18-v2.1.md @@ -0,0 +1,52 @@ +--- +layout: post +title: Version 2.1 +author: matthew +--- + +<div class="r" align="right"> +<a data-flickr-embed="true" href="https://www.flickr.com/photos/winnie_quan/2295947472/" title="Untitled"><img src="https://farm4.staticflickr.com/3048/2295947472_ed1a3fe0e1.jpg" width="500" height="281" alt="Untitled"></a><script async src="//embedr.flickr.com/assets/client-code.js" charset="utf-8"></script> +</div> + +Today we have released **version 2.1** of FixMyStreet, with some new features +and a variety of improvements for users and reusers. + +New features include allowing users to hide their own name on reports/updates, +and a new graph-based /reports page, which you can see in operation on our +[main UK site](https://www.fixmystreet.com/reports). We also now resize photos +client-side before uploading, given the increase in very large images these +days. + +Admin improvements include adding an 'inactive' state to categories, so you can +prevent a category being used for new reports but still have it available in +filters, various improvements to the inspect form and the new report process +for inspectors, and an easier way for inspectors to shortlist all reports in +their view. + +Geolocation is back on the alert page, lost sometime last year, and some random +blank spaces are no longer clickable, hooray! + +Development-wise, FixMyStreet now supports Debian stretch and perl 5.24, badly +configured SMTP options should now be spotted, and we have refactored and +simplified the CSS used for header/content/navigation. This may mean you can +simplify overrides in your cobrand. If you wish to have a static front page +rather than the normal report page, that's now possible by creating an +`about/homepage.html` template in your cobrand, and cobrands can now easily +change the new report pin colour. + +The test suite now runs each file in a transaction which means it can be run in +parallel and sped up dramatically; on the other hand, on Travis we added code +coverage which slowed it down again. + +Two **backwards incompatible** changes that may require changes to your own +templates if you have customised them: + +* The `nav-wrapper-2` class has been removed. If you have a custom footer + template, replace that class with 'container'. + +* The `/reports` page now uses different generated data. If you have a custom + `reports/index.html` template and wish to keep it, you may need to call + `update-all-reports` with a `--table` argument to generate the old style + data. + +[Full changelog](https://github.com/mysociety/fixmystreet/releases/tag/v2.1) diff --git a/docs/_posts/2017-08-03-v2.1.1.md b/docs/_posts/2017-08-03-v2.1.1.md new file mode 100644 index 000000000..b1b487fa2 --- /dev/null +++ b/docs/_posts/2017-08-03-v2.1.1.md @@ -0,0 +1,24 @@ +--- +layout: post +title: Version 2.1.1 +author: struan +--- + +<div class="r" align="right"> +<a data-flickr-embed="true" href="https://www.flickr.com/photos/drinksmachine/435344339/" title="Untitled"><img src="https://farm1.staticflickr.com/176/435344339_84fd7e4705_z_d.jpg" width="640" height="480" alt="Untitled"></a><script async src="//embedr.flickr.com/assets/client-code.js" charset="utf-8"></script> +</div> + +Today we have released **version 2.1.1** of FixMyStreet, a bugfix release with +a few small improvements. + +The map in HTML emails is now a link through to the report page, and we've +fixed a bug causing the wrong pin size to be shown there. We've also fixed a +bug that could cause a different language from the requested one to be shown on +about pages. + +Admin improvements include resending reports if changing the category +has changed the `send_method` and displaying the reporter's phone number +on the inspector form. Text deleted during moderation is no longer +replaced with `[...]`. + +[Full changelog](https://github.com/mysociety/fixmystreet/releases/tag/v2.1.1) diff --git a/docs/_posts/2017-09-14-v2.2.md b/docs/_posts/2017-09-14-v2.2.md new file mode 100644 index 000000000..b4c0aa804 --- /dev/null +++ b/docs/_posts/2017-09-14-v2.2.md @@ -0,0 +1,53 @@ +--- +layout: post +title: Version 2.2 +author: matthew +--- + +<div class="r" align="right"> +<a data-flickr-embed="true" href="https://www.flickr.com/photos/_chrisuk/6621444233/" title="Pothole"><img src="https://farm8.staticflickr.com/7155/6621444233_d109d69b26_z.jpg" width="640" height="360" alt="Pothole"></a><script async src="//embedr.flickr.com/assets/client-code.js" charset="utf-8"></script> +</div> + +Today we have released **version 2.2** of FixMyStreet, with some new features, +some bugfixes, and other improvements. + +* At long last, **body and category names can be translated** in the admin +interface, letting you easily run sites in multiple languages without having to +perform various amounts of custom work; + +* **Report states can now be edited** (and translated) in the admin interface, if +you wish to have different options for the specific workflow of your site. If +you want to have a closed state of "object removed", now you can; + +* Also through the admin interface you can now **add extra fields** to be + shown in the reporting form, letting you ask custom questions for your +installation without needing to edit any code or templates. + +All the above improvements were funded by [NDI](https://www.ndi.org/) as part +of their FixMyCommunity project; thanks very much to them. + +<hr> + +Staff users can now create reports on behalf of people anonymously, and have +more filters on report list pages; new configuration options allow you to limit +your site to logged-in users only, or prevent new users from using the site. + +The email alerts page has had some work to improve its clarity, and some work +has gone into improving the performance of various pages. Some deep diving into +SQL queries led to some notable cases where things could be sped up, and we +also found one place (listing nearby duplicate reports for staff users) where +the database was being queried at a radius of 1000km, rather than 1km :) + +One notable bugfix is if you logged in during the reporting process, or had a +server side error, your photo thumbnails weren't being redisplayed correctly +(though the photo itself was stored fine) due to change in the image library +we use – everything should now be okay. + +Lastly, if you're developing on the code, we have added a debug toolbar +(similar to the Django Debug Toolbar, if you're familiar with that) which lets +you see which templates have been used, what SQL queries have been made, what +external requests, and so on. This is automatically enabled when you run the +development server. + +[Full changelog](https://github.com/mysociety/fixmystreet/releases/tag/v2.2) + diff --git a/docs/_posts/2017-12-18-v2.3.md b/docs/_posts/2017-12-18-v2.3.md new file mode 100644 index 000000000..f769aeb2b --- /dev/null +++ b/docs/_posts/2017-12-18-v2.3.md @@ -0,0 +1,56 @@ +--- +layout: post +title: Version 2.3 +author: matthew +--- + +<div class="r" align="right"> +<a data-flickr-embed="true" href="https://www.flickr.com/photos/ursonate/9072504686" title="IMG_20130617_072808.jpg"><img src="https://farm6.staticflickr.com/5491/9072504686_8f948925c4.jpg" width="375" height="500" alt="IMG_20130617_072808.jpg"></a><script async src="//embedr.flickr.com/assets/client-code.js" charset="utf-8"></script> +</div> + +Today we have released **version 2.3** of FixMyStreet. + +The major new feature in this release is adding <strong>confirmation by phone +text</strong> instead of email, via Twilio, to allow optional verification of +reports and updates, and logging in. + +Alongside that, there have been numerous front end improvements and bugfixes. +Front end improvements include <strong>paginating reports</strong> everywhere, +making sure all maps can be expanded on mobile, the pin loading indicator no +longer covering the whole map whilst pins are loading, and improved location +disambiguation on small screens. + +Some effort has gone into <strong>improving performance</strong> of various +pages, especially the front page, reducing the amount of JavaScript loaded, +replacing our image sprite with more SVG assets, and switching to modern +prefetch. <a href="https://www.fixmystreet.com/">fixmystreet.com</a> goes a +step further by inlining critical CSS on the front page so no external requests +are needed to show the start of the site. You can read more about this process +<a href="https://www.mysociety.org/2017/11/24/peak-performance/">on the +mySociety blog</a>. + +We now support <strong>Open311 category groups</strong>, and if we fetch +updates via Open311, we can use auto-response templates to fill in updates +without their own description. + +Lastly, the <strong>dashboard and statistics</strong> have been streamlined and +improved, and are all now available under <code>/dashboard</code>. This now +includes lookup by date range, ward, category or state, and the CSV export uses +machine-readable dates and is accessible via token-based authentication. + +### Bugfixes + +The number of updates on <a +href="https://www.fixmystreet.com/">fixmystreet.com</a> hit a million! So we +made sure large numbers don't overflow on the homepage. Also, multiple 'Expand +map' links should no longer trouble your maps on mobile; with JavaScript off, +the pins should no longer be double the size they're meant to be; and also some +bad interaction between the list filters and the back button has been fixed. + +Staff users got a number of bugfixes to their interfaces, including making sure +reports could always be removed from your shortlist (even if they've switched +body), only creating one update when changing category, and making sure the +text-only contacts output was indeed text, not HTML. + +[Full changelog](https://github.com/mysociety/fixmystreet/releases/tag/v2.3) + diff --git a/docs/_posts/2018-02-12-v2.3.1.md b/docs/_posts/2018-02-12-v2.3.1.md new file mode 100644 index 000000000..075590ee0 --- /dev/null +++ b/docs/_posts/2018-02-12-v2.3.1.md @@ -0,0 +1,39 @@ +--- +layout: post +title: Version 2.3.1 +author: matthew +--- + +<div class="r" align="right"> +<a data-flickr-embed="true" href="https://www.flickr.com/photos/luiscolas/9347625116/in/photolist-ff22hh-822Qzz-hUT-6XAG2R-Aj9tco-4AMCjJ-5JAudk-eRiDAZ-5gSMZS-9aCTMh-8tNEZa-hcWs5g-4HvS9p-aUjRov-LvqZbz-rDivP3-dRRVqb-bxrZTr-gBoCym-bP6HSF-85bACu-sAqHa1-o51A6W-8mVQtB-p4G8w1-GLHRtK-hcV8fg-3yn56F-apTU2w-86dnyQ-8mF2JJ-a5A1P6-xb9Ahd-adJTdh-86dny7-hCEwQ1-A4NHwV-eCvr5q-8Whq3z-81QKHC-bQTwSK-DrizbF-8V6js5-5KScte-fvZGqs-6G8mPr-5zyJXC-fJcHcV-M6HqY9-8V3dTs" title="Keep Behind This Line"><img src="https://farm6.staticflickr.com/5451/9347625116_8f3c8bf5e7.jpg" width="500" height="333" alt="Keep Behind This Line"></a><script async src="//embedr.flickr.com/assets/client-code.js" charset="utf-8"></script> +</div> + +Today we have released **version 2.3.1** of FixMyStreet. + +This is a minor release, with a number of bug fixes but also a number of +smaller front end and admin improvements. + +The site should be clearer when things are loading now, be that on initial page +load or when an asset layer is incoming. Our phone number library has been +updated, so national phone numbers should be displayed in a nicer format as +long as you've set the `PHONE_COUNTRY` variable. + +FixMyStreet now asks for the current password (or sends an email) on password +change, sets a minimum password length and checks against a list of commons +passwords. Superusers can have optional two-factor authentication to protect +their accounts. + +We fixed an error in sending `requires_inspection` reports, issues with +multiple select-multiples on a page, a questionnaire CSS snafu, showing deleted +bodies in the dashboard list, alongside other minor bugfixes. + +For staff users, 'report as another user' now allows phone number without +email, and the inspector form shows any extra fields again. + +Admins can now anonymize/hide all a user's reports, log a user out, or remove a +user's account details. We've provided a script to expire old sessions in the +database (plus this script can be run with `--init` to set up the database for +the new "log a user out" feature). + +[Full changelog](https://github.com/mysociety/fixmystreet/releases/tag/v2.3.1) + diff --git a/docs/_posts/2018-05-31-v2.3.2.md b/docs/_posts/2018-05-31-v2.3.2.md new file mode 100644 index 000000000..9bbb444be --- /dev/null +++ b/docs/_posts/2018-05-31-v2.3.2.md @@ -0,0 +1,43 @@ +--- +layout: post +title: Version 2.3.2 +author: matthew +--- + +<div class="r" align="right"> +<a data-flickr-embed="true" href="https://www.flickr.com/photos/photosak/6214596375" title="London Loop Section 3"><img src="https://farm7.staticflickr.com/6041/6214596375_ca94e02b93.jpg" width="500" height="333" alt="London Loop Section 3"></a><script async src="//embedr.flickr.com/assets/client-code.js" charset="utf-8"></script> +</div> + +Today we have released **version 2.3.2** of FixMyStreet. + +This is a minor release, but one with quite a number of bug fixes and other +smaller front end and admin improvements. + +The questionnaire process has been improved, with “Don’t know” now an option in +the email, and recording the answer as soon as the link is clicked. Smaller +front end fixes include increasing the size of “sub map links” (hide pins, +permalink, etc), clicking the "Click map" instruction banner now begins a new +report, and improved cursor/display of the new report pin. + +Front end bug fixes include improving chart display in old IE versions, a CSS +padding bug in the sidebar “drawer”, a race condition when making a new report +quickly, some small RTL text display issues, and making sure the loading +spinner is always shown when it should be. + +We now store a user’s creation and last active times, and have provided scripts +so you can use tis new information to anonymize inactive users or reports, +email inactive users, and/or close reports to new updates. + +At long last, the contents of a report’s extra field is now fully displayed in +the admin. + +Open311 has had a number of improvements and bug fixes – we can now fetch +problems over Open311, send multiple photos via an Open311 extension, and have +the ability to have automated attributes that can be filled in but not shown to +the user (e.g. asset IDs). + +For development, we’ve added an HTML email previewer at `/dev/_email/` which +our designers love, added some Cypress browser-based testing so that some of +our JavaScript is also tested, and upgraded our Vagrantfile to use Ubuntu Xenial. + +[Full changelog](https://github.com/mysociety/fixmystreet/releases/tag/v2.3.2) diff --git a/docs/_posts/2018-06-06-v2.3.3.md b/docs/_posts/2018-06-06-v2.3.3.md new file mode 100644 index 000000000..be0a9b668 --- /dev/null +++ b/docs/_posts/2018-06-06-v2.3.3.md @@ -0,0 +1,19 @@ +--- +layout: post +title: Version 2.3.3 +author: matthew +--- + +Today we have released **version 2.3.3** of FixMyStreet. + +This fixes a few bugs found in the past week of the last release, including one +potential data leak. It was possible to obtain from a crafted URL the textual +content of an update, and the name of the user who left that update (only if +they had made the update non-anonymously), even if that update was unconfirmed +or hidden. No other details of the update or user were made available. + +Other fixes include incorrect behaviour when selecting report pins on mobile, +bad admin navigation links in multi-language installs, and an issue with the +map display when an inspector clicked back from a report page on mobile. + +[Full changelog](https://github.com/mysociety/fixmystreet/releases/tag/v2.3.3) diff --git a/docs/_posts/2018-06-07-v2.3.4.md b/docs/_posts/2018-06-07-v2.3.4.md new file mode 100644 index 000000000..09b9bb436 --- /dev/null +++ b/docs/_posts/2018-06-07-v2.3.4.md @@ -0,0 +1,12 @@ +--- +layout: post +title: Version 2.3.4 +author: matthew +--- + +A very quick release, only a day after 2.3.3, but **version 2.3.4** fixes a bug +introduced in 2.3.3 as part of the fix for selecting pins on mobile; whilst +that was fixed, the fix then prevented pins being selected on non-`/around` +pages, whoops. + +[Full changelog](https://github.com/mysociety/fixmystreet/releases/tag/v2.3.4) diff --git a/docs/_posts/2018-09-06-v2.4.md b/docs/_posts/2018-09-06-v2.4.md new file mode 100644 index 000000000..435eba699 --- /dev/null +++ b/docs/_posts/2018-09-06-v2.4.md @@ -0,0 +1,42 @@ +--- +layout: post +title: Version 2.4 +author: matthew +--- + +<div class="r" align="right"> +<a data-flickr-embed="true" href="https://www.flickr.com/photos/matthamm/896309437/" title="Fattoria Palazetta, Field of Sunflowers, Cecina, Tuscany, Italy"><img src="https://farm2.staticflickr.com/1199/896309437_33a2c84a4e.jpg" width="500" height="375" alt="Fattoria Palazetta, Field of Sunflowers, Cecina, Tuscany, Italy"></a><script async src="//embedr.flickr.com/assets/client-code.js" charset="utf-8"></script> +</div> + +Today we have released **version 2.4** of FixMyStreet. + +This release contains a security fix to prevent the potential leak of a user +account's phone number, if you knew their email address; please upgrade your +installation as soon as you can, or if that is not possible, please apply +commit `abcb1f86` to your installation in the meantime. We have also released +version 2.3.5 which is identical to 2.3.4 except with this fix applied. + +The front end has removed its input placeholders, for better accessibility; +improved the report button in the navigation bar to be more context-aware and +allow for easier reporting in the same location; will only show 6 months of +reports on the around page by default, to hopefully deal with the issues of too +many or too few reports being shown; and removes the need for a separate +per-category ajax call by returning all category data up front. + +The admin now trims spaces from search input (at last!), lets you edit a +category group, and has a 'send login email' button on a user's edit page, so +you don't have to ask them to do it themselves. + +Bugfixes include an issue with the category filter if a category contained a +comma, fixing duplicate category display on Your Account page, and making sure +the Home link on mobile is always clickable. Staff workflows had various fixes +as well, such as inspectors being able to unset priority, seeing all fixed +issues with the map page filter, and fixing the pin dragging. + +For developers, there are new hooks for e.g. custom search results, extra login +conditions, and changing the details placeholder; a new `/_dev/` URL +(accessible to superusers) for previewing confirmation/submission pages; and +the client can set bodis a repot must not be sent to (e.g. if asset selection +means it has that knowledge). + +[Full changelog](https://github.com/mysociety/fixmystreet/releases/tag/v2.4) diff --git a/docs/_posts/2018-10-01-v2.4.1.md b/docs/_posts/2018-10-01-v2.4.1.md new file mode 100644 index 000000000..e39c3f0c5 --- /dev/null +++ b/docs/_posts/2018-10-01-v2.4.1.md @@ -0,0 +1,32 @@ +--- +layout: post +title: Version 2.4.1 +author: matthew +--- + +<div class="r" align="right"> +<a data-flickr-embed="true" href="https://www.flickr.com/photos/bradhoc/6341956589/" title="Docks"><img src="https://farm7.staticflickr.com/6097/6341956589_97f39d3f3b.jpg" width="500" height="304" alt="Docks"></a><script async src="//embedr.flickr.com/assets/client-code.js" charset="utf-8"></script> +</div> + +Today we have released **version 2.4.1** of FixMyStreet. + +This release contains a wide variety of new features for use by reusers of our software, +funded by [NDI](https://www.ndi.org/) as part of their FixMyCommunity project; thanks +very much to them. It also includes a number of smaller features and bugfixes. + +FixMyStreet now comes with a Dockerfile and example Docker Compose setup for +use by those tools, and we provide a hosted Vagrant box that can be used also. +You can choose to store photos in Amazon S3 rather then the local filesystem, +use an external memcache server, + +We've added a script, `cobrand-checks`, that will list/diff template changes in core +that might need applying to a cobrand, to help with upgrading, and added more checks +to our Pull Requests on GitHub to spot included cobrands that need checking with each +PR made. + +We have sped up fetching full lists of bodies to help installs that have +thousands of bodies, sped up the display of `/_dev` pages, fixed the display of +pins on a body page when using Bing or TonerLite maps, and fixed an issue where the +`from_body` field was lost on a user when edited by a non-superuser admin. + +[Full changelog](https://github.com/mysociety/fixmystreet/releases/tag/v2.4.1) diff --git a/docs/_posts/2018-10-08-s3-photo-migration.md b/docs/_posts/2018-10-08-s3-photo-migration.md new file mode 100644 index 000000000..2f28dd1d3 --- /dev/null +++ b/docs/_posts/2018-10-08-s3-photo-migration.md @@ -0,0 +1,182 @@ +--- +layout: post +title: Migrating to S3 photo storage +author: davea +--- + +The [recent release of FixMyStreet v2.4.1](/2018/10/01/v2.4.1/) brought with +it the ability to store photos for reports & updates in a bucket on Amazon's +S3 instead of locally on disk. + +Getting a new site configured to store newly uploaded photos in S3 is explained +in the [configuration pages](/customising/config/#photo_storage_backend), and +is a matter of tweaking a few keys in your `general.yml` file. But what about +if you have an existing FixMyStreet installation that you would like to +migrate to storing all photos in S3? This post outlines a process for +achieving just that - without any downtime. + + +## Prerequisites + + - A running FixMyStreet site with at least one report or update with a photo + - An account on Amazon AWS, and access/secret keys for a role that can create/manage S3 buckets + - The [AWS CLI](https://aws.amazon.com/cli/) tool installed on your FixMyStreet server, and [configured](https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-getting-started.html) with access credentials + + +## Creating the bucket + +First off, we need to create a bucket to hold all the photos. If you've already done this, skip ahead to the **[Migrating photos](#migrating-photos)** section. + +We'll show two ways to create a bucket: via the S3 web console and using the `aws` command line tool. + +### Using the S3 web console + +Log in to [S3 web console](https://s3.console.aws.amazon.com/s3/home), and click the 'Create Bucket' button + + + +Enter a name for the bucket, e.g., `my-fixmystreet-photos`, and choose the region closest to your FixMyStreet server ([more about buckets and regions](https://docs.aws.amazon.com/AmazonS3/latest/dev/UsingBucket.html)): + + + +Your newly created bucket should then appear in the list of buckets. Make sure public access isn't allowed (photos are served to users via your FixMyStreet server, not directly from S3): + + + +If everything looks OK, skip to the **[Migrating photos](#migrating-photos)** section. + +### Using the `aws` CLI tool + +You can also create buckets from the command line using the `aws` tool. + +If you've not already done so, now is the time to [install](https://aws.amazon.com/cli/) and [configure](https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-getting-started.html) the `aws` CLI tool. + +To create a bucket named `my-fixmystreet-photos` in the `eu-west-1` region, run the following: + +```bash +$ aws s3 mb s3://my-fixmystreet-photos --region eu-west-1 +``` + +The following success message should be shown: + +``` +make_bucket: my-fixmystreet-photos +``` + +The bucket's region can be inspected: + +```bash +$ aws s3api get-bucket-location --bucket my-fixmystreet-photos +{ + "LocationConstraint": "eu-west-1" +} +``` + +The bucket should be created as private by default, but you may wish to check its access policy +on the [S3 web console](https://s3.console.aws.amazon.com/s3/home). + + +## Migrating photos + +Now that the bucket has been created, the next step is to fill it with all the photos +from your existing FixMyStreet installation. For this we'll need the `aws` CLI tool, +so make sure you've [installed](https://aws.amazon.com/cli/) and [configured](https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-getting-started.html) it. + +Existing photos are stored on disk in the directory specified by the `UPLOAD_DIR` +(or `PHOTO_STORAGE_OPTIONS.UPLOAD_DIR`) configuration key in `conf/general.yml`. +For this example, let's assume that's `/home/fixmystreet/photos`. + +Running the following (with the path and bucket name altered to suit your configuration) +will make copy-pasting the subsequent commands a little easier: + +```bash +$ export UPLOAD_DIR=/home/fixmystreet/photos +$ export S3_BUCKET=my-fixmystreet-photos +``` + +We'll use `aws s3 sync` to copy all photos to the new S3 bucket. To get a preview of +the operations it will perform without actually copying anything, run: + +```bash +$ aws s3 sync $UPLOAD_DIR s3://$S3_BUCKET --dryrun +``` + +You should see output similar to the following, one line for each file: + +``` +(dryrun) upload: /home/fixmystreet/photos/069a5d216321061757fe30a6d7f862669eb46d7d.jpeg to s3://my-fixmystreet-photos/069a5d216321061757fe30a6d7f862669eb46d7d.jpeg +(dryrun) upload: /home/fixmystreet/photos/0d7db3b2cd615e27c50345c8144b6b9782d7ff4a.jpeg to s3://my-fixmystreet-photos/0d7db3b2cd615e27c50345c8144b6b9782d7ff4a.jpeg +(dryrun) upload: /home/fixmystreet/photos/137dfb5fe88c0207e2a339b02fcbc5e812b4e68b.jpeg to s3://my-fixmystreet-photos/137dfb5fe88c0207e2a339b02fcbc5e812b4e68b.jpeg +[...] +``` + +If the output looks correct, re-run without the `--dryrun` flag to actually copy +everything to the S3 bucket: + +```bash +$ aws s3 sync $UPLOAD_DIR s3://$S3_BUCKET +upload: /home/fixmystreet/photos/069a5d216321061757fe30a6d7f862669eb46d7d.jpeg to s3://my-fixmystreet-photos/069a5d216321061757fe30a6d7f862669eb46d7d.jpeg +upload: /home/fixmystreet/photos/0d7db3b2cd615e27c50345c8144b6b9782d7ff4a.jpeg to s3://my-fixmystreet-photos/0d7db3b2cd615e27c50345c8144b6b9782d7ff4a.jpeg +upload: /home/fixmystreet/photos/137dfb5fe88c0207e2a339b02fcbc5e812b4e68b.jpeg to s3://my-fixmystreet-photos/137dfb5fe88c0207e2a339b02fcbc5e812b4e68b.jpeg +[...] +``` + +All your photos are now in S3, so the next step is to tell FixMyStreet about the new bucket. +(See the note in **[the final section](#final-steps--tidying-up)** for reports made between now +and when the FixMyStreet config has been updated to use S3.) + + +## Configuring FixMyStreet to use S3 + +We need to make a few changes to `conf/general.yml` to configure FixMyStreet to +use the new S3 bucket for photo storage. + +Your `conf/general.yml` probably has a section similar to the following: + +```yaml +PHOTO_STORAGE_BACKEND: 'FileSystem' +PHOTO_STORAGE_OPTIONS: + UPLOAD_DIR: '../photos' +``` + +Or, if you set the site up before v2.4.1: + +```yaml +UPLOAD_DIR: '../photos' +``` + +Either way, you need to replace that section with the following: + +```yaml +PHOTO_STORAGE_BACKEND: 'S3' +PHOTO_STORAGE_OPTIONS: + BUCKET: 'my-fixmystreet-photos' + ACCESS_KEY: 'AKIA12345' + SECRET_KEY: '1234/1234' +``` + +Make sure you set `BUCKET`, `ACCESS_KEY` and `SECRET_KEY` to the same values +as when you configured the `aws` CLI tool and created the bucket. + +These changes are all that's needed to switch your FixMyStreet installation to using +S3 for photos. Once you're happy with the changes you've made to `conf/general.yml`, +restart the FixMyStreet app server ([example](/updating/#restart-the-server)) and +check the output for any errors. + +Once the server has restarted, try making a new report and uploading a couple of photos. +You should see new files appear in the S3 bucket (you can view them via the S3 web console) +as soon as you've drag-dropped them into the new report (or update) form. Check that report +photos appear on the site for newly-created reports too. **NB** photos for existing reports +are likely already cached on disk by FixMyStreet, so do check a new report. + + +## Final steps & tidying up + +Because there was a small window of time between syncing existing photos to the S3 bucket +and switching over the `conf/general.yml` config, there's a chance that a report with +photos was made whose photos weren't synced to S3. +Now that FixMyStreet is using S3, you may wish to re-run the `aws s3 sync` command from above +in order to make sure any such photos have been transferred to S3. + +The old directory containing photos on disk isn't needed any more, so once you're happy +the S3 storage is working as expected, you can delete this. diff --git a/docs/_posts/2018-11-06-v2.4.2.md b/docs/_posts/2018-11-06-v2.4.2.md new file mode 100644 index 000000000..9560d9b96 --- /dev/null +++ b/docs/_posts/2018-11-06-v2.4.2.md @@ -0,0 +1,27 @@ +--- +layout: post +title: Version 2.4.2 +author: matthew +--- + +<div class="r" align="right"> +<a data-flickr-embed="true" href="https://www.flickr.com/photos/eperales/2588601813/" title="Water & Oil"><img src="https://farm4.staticflickr.com/3070/2588601813_eb5e97557f.jpg" width="500" height="375" alt="Water & Oil"></a><script async src="//embedr.flickr.com/assets/client-code.js" charset="utf-8"></script> +</div> + +Today we have released **version 2.4.2** of FixMyStreet. + +This is mainly a bugfix release, but does contain a couple of new features - +the admin lets you see and manage a user’s alerts, and the dashboard has update +CSV export. + +Bugfixes include an issue when setting up a new Vagrant-based installation, +updating the areas column when a report is moved by a staff user, giving +superusers access to update staff dropdown, and an issue with subcategories +when visiting /report/new directly. + +We have renamed a few SCSS variables for clarity; the old names still work, but +if you wish to update your cobrand, then change `$map_nav_bg` to +`$nav_background_colour`, `$nav_fg` to `$nav_colour`, and `$nav_fg_hover` to +`$nav_hover_background_colour`. + +[Full changelog](https://github.com/mysociety/fixmystreet/releases/tag/v2.4.2) diff --git a/docs/_posts/2018-12-21-v2.5.md b/docs/_posts/2018-12-21-v2.5.md new file mode 100644 index 000000000..de25b59d0 --- /dev/null +++ b/docs/_posts/2018-12-21-v2.5.md @@ -0,0 +1,47 @@ +--- +layout: post +title: Version 2.5 +author: matthew +--- + +<div class="r" align="right"> +<a data-flickr-embed="true" href="https://www.flickr.com/photos/catinatree/8299877655/" title="White christmas!!! =D"><img src="https://farm9.staticflickr.com/8501/8299877655_ea1faf5bb3.jpg" width="500" height="375" alt="White christmas!!! =D"></a><script async src="//embedr.flickr.com/assets/client-code.js" charset="utf-8"></script> +</div> + +Today we have released **version 2.5** of FixMyStreet; happy solstice! + +This release contains a number of front end improvements, especially to the +sign in flow. This was the result of a lot of work to simplify the whole +process, and we’ve written more about it on the +[mySociety blog](https://www.mysociety.org/2018/12/20/open-sesame-simpler-log-in-forms-on-fixmystreet/). + +Other front end improvements range from a more prominent display of "state" on +report page, making sure you don’t need two taps on a report list entry on a +touchscreen, and clearer relocation options while you’re reporting a problem. + +Moderation has had some work done – it can now potentially edit a report’s +category, should you wish it to do so; it stores more of a report’s original +data upon moderation, and also now stores all moderation history, making that +visible in the report/update admin interface. + +Bugfixes include one reported by FixaMinGata (back in early 2017!) about better +map zoom behaviour when clicking on pins and going back; not escaping HTML +entities in report titles pulled in by Ajax; and show missing reopening/fixed +questionnaire responses when tey lacked their own separate update. We also +fixed some issues with our Open311 contact group handling, and improved the +validation of fetched reports timestamps. + +We now strip Exif data on uploaded photos, and have added a new config +variable, `SYMLINK_FULL_SIZE`, that can be used to symlink your full size +photos out of the photo cache rather than copy them, if your static files are +being served by your web server. + +[Full changelog](https://github.com/mysociety/fixmystreet/releases/tag/v2.5) + +## Upgrading + +Due to the sign-in and banner changes, this release changes a number of the +base templates. If you have overridden any related templates in your cobrand +and not fed those changes upstream, you may need to make adjustments to match; +please use our `bin/cobrand-checks` script to help make comparisons between +your changes. diff --git a/docs/_posts/2019-04-29-v2.6.md b/docs/_posts/2019-04-29-v2.6.md new file mode 100644 index 000000000..6e74b87f5 --- /dev/null +++ b/docs/_posts/2019-04-29-v2.6.md @@ -0,0 +1,37 @@ +--- +layout: post +title: Version 2.6 +author: matthew +--- + +<div class="r" align="right"> +<img src="/assets/posts/v2.6-image.jpeg" alt=""> +Image © <a href="https://twitter.com/StillWalkers">Ben Waddington</a> +</div> + +Today we have released **version 2.6** of FixMyStreet. + +This release fixes a cross-site scripting security issue where someone could +create a report through the site with a specially constructed query parameter, +and then viewing that report on the admin report edit page would allow the +report creator to run their own JavaScript. We have also released version 2.5.1 +which is identical to 2.5 including this fix. + +There's a new, optional, feature to auto-suggest similar nearby problems while +reporting, to discourage duplicate reports; and the map state is now updated in +the URL to make sharing links easier. A bit more work has been done on +moderation, spotting conflicts and showing moderation history to staff on +report pages, as well as in the admin. + +Mostly this release is bugfixes, please see the +[changelog](https://github.com/mysociety/fixmystreet/releases/tag/v2.6) for +full details. + +## Upgrading + +The admin body and user sections have been refactored – if you have custom +templates/code, you may need to update links to those. + +If you wish the default for the showname checkbox to be checked, you can add +`sub default_show_name { 1 }` to your cobrand file. + |