diff options
Diffstat (limited to 'docs/install')
| -rw-r--r-- | docs/install/ami.md | 105 | ||||
| -rw-r--r-- | docs/install/database.md | 37 | ||||
| -rw-r--r-- | docs/install/docker.md | 91 | ||||
| -rw-r--r-- | docs/install/fixmystreet-with-sms.md | 267 | ||||
| -rw-r--r-- | docs/install/index.md | 27 | ||||
| -rw-r--r-- | docs/install/install-script.md | 79 | ||||
| -rw-r--r-- | docs/install/manual-install.md | 251 | ||||
| -rw-r--r-- | docs/install/testing.md | 47 | ||||
| -rw-r--r-- | docs/install/troubleshooting.md | 116 | ||||
| -rw-r--r-- | docs/install/vagrant.md | 70 |
10 files changed, 1090 insertions, 0 deletions
diff --git a/docs/install/ami.md b/docs/install/ami.md new file mode 100644 index 000000000..fd283945a --- /dev/null +++ b/docs/install/ami.md @@ -0,0 +1,105 @@ +--- +layout: page +title: AMI for EC2 +--- + +# FixMyStreet AMI for EC2 + +<p class="lead"> + To help people to get started with the FixMyStreet platform, we have + created an AMI (Amazon Machine Image) with a basic installation of + FixMyStreet, which you can use to create a running server on an Amazon + EC2 instance. +</p> + +Note that this is just one of [many ways to install FixMyStreet]({{ "/install/" | relative_url }}). + +## Installing on Amazon's Web Services + +If you don't have your own server, or simply prefer to use an external one, you +can use Amazon Web Services (AWS) instead. They provide difference scale +servers, called instances. The smallest instance, the Micro, will be [free +for a year](http://aws.amazon.com/free/). + +### Using our pre-built AMI + +The AMI we've prepared for you can be found in the **EU West (Ireland)** +region, with the ID `ami-05386aa7b2b4faee9` and name "FixMyStreet installation +full 2018-12-21". You can launch an instance based on that AMI with +[this link](https://console.aws.amazon.com/ec2/home?region=eu-west-1#launchAmi=ami-05386aa7b2b4faee9). + This AMI is based on the [latest tagged release](https://github.com/mysociety/fixmystreet/releases) + and contains everything you need to get a base install up and running. + +When you create an EC2 instance based on that AMI, make sure that you +choose Security Groups that allow at least inbound HTTP, HTTPS and +SSH, and perhaps SMTP as well for email. + +When your EC2 instance is launched, you will be able to log in as the +`admin` user. This user can `sudo` freely to run commands as root. +However, the code is actually owned by (and runs as) the `fms` user. +After creating the instance, you may want to edit a configuration +file to set a couple of parameters. That configuration file is +`/home/fms/fixmystreet/conf/general.yml`, which can be edited with: + + admin@ip-10-58-191-98:~$ sudo su - fms + fms@ip-10-58-191-98:~$ cd fixmystreet + fms@ip-10-58-191-98:~/fixmystreet$ nano conf/general.yml + +You should set +<code><a href="{{ "/customising/config/#contact_email" | relative_url }}">CONTACT_EMAIL</a></code> +and +<code><a href="{{ "/customising/config/#do_not_reply_email" | relative_url }}">DO_NOT_REPLY_EMAIL</a></code> +or whatever you wish to use. postfix is installed so that outgoing email will +work, but this may need further configuration. + +Then you should restart the Catalyst FastCGI server with: + + fms@ip-10-58-191-98:~/fixmystreet$ logout + admin@ip-10-58-191-98:~$ sudo service fixmystreet restart + +If you find the hostname of your EC2 instance from the AWS console, +you should then be able to see the site at http://your-ec2-hostname.eu-west-1.compute.amazonaws.com + +By default, the admin part of the website (`/admin`) requires a user with +superuser permission to log in. In order to use this +interface, you will need to create a username and password for one or +more superusers. To add such a user, you can use the `createsuperuser` +command, as follows: + + admin@ip-10-58-66-208:~$ sudo su - fms + fms@ip-10-58-191-98:~$ cd fixmystreet + fms@ip-10-58-191-98:~/fixmystreet$ bin/createsuperuser fmsadmin@example.org password + fmsadmin@example.org is now a superuser. + +This basic installation uses the default cobrand, with a +(deliberately) rather garish colour scheme. + +### Building your own AMI + +You may have specific requirements that mean you want to create your own +customised AMI so we provide access to the Packer configuration files we +use to generate our images in our [Public Builds repository](https://github.com/mysociety/public-builds) +together with some guidance on how you can customise the build process. + +#### Integrating with other AWS services + +One option that we get asked about is how to integrate the AMI with other services +within AWS such as RDS. This is more challenging as we can't know about each +potential environment, but the Public Builds repository contains a recipe that can +be used to create a slim version of the AMI that omits the Postgres database +and memcache instance, similar in some respects to [the Docker build we provide](/install/docker/). + +In order to use this effectively you will need to provide a Postgres database +with the FixMyStreet schema loaded and a memcached endpoint, perhaps using RDS +and Elasticache. You would then need to build a custom slim AMI and seed the +relevant configuration files. + +See the [FixMyStreet specific notes](https://github.com/mysociety/public-builds/blob/master/docs/fixmystreet.md) +in the Pubic Builds repository for more information. + +## Installation complete... now customise + +You should then proceed +to [customise your installation](/customising/). + +Please also see the instructions for [updating your installation](/updating/ami/). diff --git a/docs/install/database.md b/docs/install/database.md new file mode 100644 index 000000000..5b5ffa19f --- /dev/null +++ b/docs/install/database.md @@ -0,0 +1,37 @@ +--- +layout: page +title: Database troubleshooting +--- + +# Database user access control + +<p class="lead"> + These instructions are for Debian — do consult the PostgreSQL + documentation if you are having trouble at this stage. +</p> + +At this point you might need to configure PostgreSQL to allow password-based +access to the `fms` database as the user `fms` from using Unix-domain sockets. +Edit the file `/etc/postgresql/9.6/main/pg_hba.conf` (note that the version +number in this path will vary depending on the version of Debian you are +using) and add as the first line: + + local fms fms md5 + +You will then need to restart PostgreSQL with: + + $ sudo service postgresql restart + +If you want to access the database from the command line, you can add +the following line to `~/.pgpass`: + + localhost:*:fms:fms:somepassword + +Then you should be able to access the database with: + + $ psql -U fms fms + +## Configuration + +When you've got everything working, you'll need to update the +[database config settings]({{ "/customising/config/#fms_db_host" | relative_url }}). diff --git a/docs/install/docker.md b/docs/install/docker.md new file mode 100644 index 000000000..3c716a5da --- /dev/null +++ b/docs/install/docker.md @@ -0,0 +1,91 @@ +--- +layout: page +title: Docker +--- + +# FixMyStreet with Docker + +<p class="lead"> + You can use Docker and Docker Compose to get up and running quickly + with FixMyStreet. +</p> + +This is just one of [many ways to install FixMyStreet]({{ "/install/" | relative_url }}). + +## Public images + +As well as providing a `Dockerfile` which you could use as the basis of your own +customised build, we provide public [images on Docker Hub](https://hub.docker.com/u/fixmystreet/) +with a full FixMyStreet installation for each of our tagged releases. + +## Docker Compose + +If you have Docker and Docker Compose installed, then the following should +set up a working FixMyStreet installation, with containers for the application, +database, memcached and webserver: + + docker-compose up + +You can then layer your own [cobrand-specific code](/customising/) +on top, update the configuration, or log in and make changes. + +A superuser is automatically created, with email `superuser@example.org` +and password given in `docker-compose.yml`. + +This basic installation uses the default cobrand, with a +(deliberately) rather garish colour scheme. + +## Adding your own cobrand + +If you want to map your own cobrand data into the Docker container, have a +repository/directory that contains the following (all items optional): + + my-cobrand-repo/ + templates/web/(cobrand)/ + templates/email/(cobrand)/ + perllib/FixMyStreet/Cobrand/(CoBrand.pm) + web/cobrands/(cobrand)/ + +Create a docker-compose.override.yml file containing: + + version: '3' + + services: + fixmystreet: + volumes: + - /path/to/your/general.yml:/var/www/fixmystreet/fixmystreet/conf/general.yml + - /path/to/my-cobrand-repo:/var/www/fixmystreet/cobrand + +Now if you run `docker-compose up` it should automatically include that cobrand +within the running container. + +This is a new facility, so please do feed back your thoughts. + +## Database configuration + +The example Docker Compose environment includes a [slightly customised Postgres container](https://github.com/mysociety/public-builds/tree/master/docker/postgres) +based on [the official image](https://hub.docker.com/_/postgres/) and localised for `en_GB`. + +This will be configured the first time it is started and its data stored in a +Docker volume for persistence. The password for the `postgres` user should be set +in the `POSTGRES_PASSWORD` environment variable and made available to both the +database and application containers; along with the various `FMS_DB_*` environment +variables this will be used to ensure the correct users, permissions and databases +are created when the container starts for the first time. + +### Using an external database + +If you wish to host the database in an external service you can do so by updating +the various `FMS_DB_*` environment variables used by the application container and +in `general.yml`. You should not provide a `POSTGRES_PASSWORD` variable to the +application container in this case. + +The application container will attempt to create the database if it doesn't already exist, so +you can either provide the user with the `CREATEDB` privilege or simply provide +an empty database and the application container will load the schema when it starts +for the first time. + +## Installation complete... now customise + +You should then proceed +to [customise your installation](/customising/). diff --git a/docs/install/fixmystreet-with-sms.md b/docs/install/fixmystreet-with-sms.md new file mode 100644 index 000000000..4d6ce9436 --- /dev/null +++ b/docs/install/fixmystreet-with-sms.md @@ -0,0 +1,267 @@ +--- +layout: page +title: Using FixMyStreet with SMS +--- + +# Using FixMyStreet with SMS + +<p class="lead"> + We're sometimes asked about running FixMyStreet with + <a href="{{ "/glossary/#sms" | relative_url }}" class="glossary__link">SMS</a>. + With SMS, users can report problems by text message instead of using the web + interface. This page describes how it can work, and what you must consider + before going ahead. +</p> + +Adding SMS capability to your FixMyStreet site is, on the face of it, a great +idea. In theory, it allows people with non-internet enabled handsets, or with +no data network coverage, to report problems. Such people may represent a large +and important section of the local population who might otherwise be unable to +use the service. However, it's important to appreciate what the limitations of +SMS are, and why we always recommend you establish and run your site on the web +*before* you try to add SMS capability to it. + +## Limitations of SMS + +### Main issues: duplicate reports, no location information + +One important aspect to FixMyStreet is that most +<a href="{{ "/glossary/#report" | relative_url }}" class="glossary__link">problem reports</a> are about problems with specific locations. On the web, this +means we can present the user with a map to click on. + +This interface, encouraging people to click on a map, serves a couple of functions: + + * it deters duplicate reports being sent for the same problem, because it's obvious if a problem has already been reported (a pin will already be there) + + * it provides an accurate + <a href="{{ "/glossary/#latlong" | relative_url }}" class="glossary__link">lat-long</a> + position of the location of the problem + +The main problem with SMS — although there are others — is that +both of these benefits are lost. So although SMS may be beneficial because it +doesn't require a more powerful device or interface capability, these are two +significant drawbacks. Both of them require your team to process the incoming +reports -- either to detect and intecept duplicate reports, or to identify the +precise location of a problem where this is not clear. + +<div class="attention-box info"> + In effect, this means that adding SMS capability to FixMyStreet <em>increases + the burden on the support team running the site</em>. Or, if it doesn't, then + this burden is passed directly on to the authorities who are receiving the + reports — and you should not run your site in this way. +</div> + +This is not to say it should not be done, just that if you do it you must +appreciate that you will not be making your team's lives easier. Of course, you +might be making things easier for the citizen users, but this will only happen +if your team is able to handle the work it may generate. It might not be a lot +of work -- that depends on how the system is used, and how widely -- but it +will certainly not be less work. + +We have also encountered misunderstanding about using geolocation on mobile +devices. The technical issue here is secondary — the main point is that +FixMyStreet doesn't need the location of the *handset*; it needs the location +of the problem being reported. This distinction is handled by the web +interface, but not when using SMS. See +*[Geolocating via SMS](#geolocating-via-sms)* below for more details. + +So, we have two approaches to this, and we always recommend anyone looking to +add SMS reporting capability to FixMyStreet tries the first one (method 1, +below) first. + +In both cases, you start by advertising a number (possibly a shortcode which is +easier for people to remember) for people to send SMS reports to. This means +you should +[customise your templates]({{ "/customising/templates" | relative_url }}), because (when it's running live) the number should presumably appear on the website +too. + +## Method 1: use a mobile phone (yes, really!) + +<div class="attention-box info"> +Have all incoming message go to a mobile phone your team has access to. They +then create problem reports in the FixMyStreet using their staff login, on behalf +of the reporter. +</div> + +The main reason for doing this is you get to discover just how badly needed SMS +input is. If nobody sends reports to you, you've discovered that it's not in +demand, and you haven't invested any time in integrating SMS into the system. +And if they do, you get a good idea of the kind and quality of the reports you +will be receiving. + +The downside of this is that the reports you create are not in the name (or +account) of the sender -- this may be more of a problem than you might realise, +because it means that the body (the council) who eventually gets the report +cannot reply directly (unless you also pass on the phone number -- note that +you typically won't know the name of the citizen either, as you've probably only +got their number). + +It's likely that you will need to reply to some incoming SMSs to clarify the +nature of the report (especially to pinpoint the location if it's not clear), +which you can do from the mobile phone. Of course there is an implicit cost to +this because you're sending an outgoing messages from the mobile phone each time. + +## Method 2: use an SMS gateway + +<div class="attention-box info"> +Use "Message Manager" integrated with an SMS gateway. Your staff can access +the messages using their staff login, and create problem reports on behalf +of the reporter. +</div> + +We have a web application called +<a href="{{ "/glossary/#message-manager" | relative_url }}" class="glossary__link">Message Manager</a> that accepts incoming messages from an SMS gateway, and makes them available to +<a href="{{ "/glossary/#staff-user" | relative_url }}" class="glossary__link">staff users</a> +within the FixMyStreet website. A staff user on a FixMyStreet site that is +using Message Manager can see incoming SMS reports, reply to them, and turn +them into reports by clicking on the map. This effectively integrates SMS +messages to FixMyStreet but it's very important to understand this does not +automate the creation of reports. We do not do that, because of the subtleties +described above: the location of the problem is often vague, and the submission +may be a duplicate report anyway. + +The advantage of using Message Manager is that the reports are easier for +trained staff to manage, because it's all happening in the web browser. Reports +submitted in this way are marked as such ("via mobile") on the site. It also +tracks the conversation threads for any replies. Commonly, your staff may need +to clarify details with the original reporter before the report can be +accurately made. Message Manager allows those replies to be sent from within +the browser too, and maintains them as threads. It also supports a certain +amount of processing based on codes within the message, for example, incoming +messages can be automatically classified by looking for prefix codes (e.g., +"to report problems in the Foo district, text 'FOO pothole outside +Example Shop in Demo Street"). + +So, Message Manager makes sense if you're getting a lot of SMS messages and +your team is willing to handle them. + +But there are two big caveats: firstly, it is not straightforward to set up, +because it needs to be integrated with an SMS gateway that is using the +advertised number or short code. This integration almost certainly requires +custom coding depending on the characteristics of whichever SMS gateway you use +(for example, +[this is the code](https://github.com/mysociety/message-manager/blob/master/app/Console/Command/NetcastShell.php) +we wrote so that Message Manager could run with the NetCast SMS gateway in the +Philippines). Also, of course there is a cost involved in using an SMS gateway, +typically both for its setup (for receiving messages), its ongoing service, and +possibly charges for outgoing messages. + +And remember, accepting SMS messages through an SMS gateway with Message +Manager does not lessen the burden of work on your team, it adds to it. It just +makes that work a little easier because it's possible to handle everything +entirely in a web browser, and there's a little less typing to do. + +## Process/comparison of methods + +<table class="table"> + <tr> + <th></th> + <th> + Method 1:<br>mobile phone receiving SMS directly + </th> + <th> + Method 2:<br>Message Manager via SMS gateway + </th> + </tr> + <tr> + <td> + Set up + </td> + <td>Practically none</td> + <td> + Potentially complex: + <br> + Install MM and create MM admin user(s), + configure with FMS (including CORS), + integration with SMS gateway (requires local testing), + authorise nominated staff accounts, + configure prefixes if used + </td> + </tr> + <tr> + <td> + Incoming messages + </td> + <td>arrive in phone's inbox</td> + <td>appear in message list in FMS, when logged in as staff</td> + </tr> + <tr> + <td> + Report creation (staff user) + </td> + <td>click on map, copy text manually from phone, select category</td> + <td>click on message, click on map: message text appears in the report, select category</td> + </tr> + <tr> + <td> + How location is determined + </td> + <td> + staff user clicks on map + </td> + <td> + staff user clicks on map + <br> + Area can be automatically detected by prefix + in SMS message + </td> + </tr> + <tr> + <td> + Reply to user (i.e., send SMS) + </td> + <td>by replying to incoming message (staff user sees the texter's phone number)</td> + <td>via the SMS gateway (can be configured not to reveal phone numbers to staff users)</td> + </tr> + <tr> + <td> + FMS records this as coming via SMS + </td> + <td> + not automatically + </td> + <td> + yes + </td> + </tr> + <tr> + <td> + automatic notification of status change + (e.g., when fixed) + </td> + <td>no<br> </td> + <td>no<br>(but theoretically possible)</td> + </tr> +</table> + +<img src="/assets/img/fms_with_sms_flowchart.png"> + + +## Geolocating via SMS + +Finally, a note about geo-location via SMS: some people get enthusiastic about +being able to automatically detect the location an SMS was sent from. It's +important to remember, in the context of FixMyStreet, that even if you can +determine the location from which an SMS was sent (technically, SMS itself does +not provide such a capability, but some network providers may offer a service +which does a similar thing) this isn't automatically *useful*. + +The problem is this: very often a report to FixMyStreet is not being made from +the location of the problem. That means you don't care where the phone is; you +care where the problem (e.g., the pothole) is. This works fine with the web +interface where the user can (if they want, and if the device supports it) use +their current location which the device can automagically supply. But there's +no such interface with SMS, and even if there was a reliable way to identify +the location of the sender, we would warn agains sending reports on to the body +responsible for fixing them when there's no way of knowing if the geolocation +data is or is not the correct location. FixMyStreet would fail as a service if +the authorities it is reporting to cannot trust the location information it is +sending to be useful. + + + + + + + + diff --git a/docs/install/index.md b/docs/install/index.md new file mode 100644 index 000000000..e0feadea7 --- /dev/null +++ b/docs/install/index.md @@ -0,0 +1,27 @@ +--- +layout: page +title: Installing +--- + +# Installing FixMyStreet Platform + +<p class="lead"> + There are several options for installing the FixMyStreet platform, + including Docker, an AMI for Amazon EC2 or an installation script. +</p> + +## Ways to install + +* [Use Docker]({{ "/install/docker/" | relative_url }}) +* [Use an install script for Debian or Ubuntu servers]({{ "/install/install-script/" | relative_url }}) +* [Use a FixMyStreet AMI for Amazon EC2]({{ "/install/ami/" | relative_url }}) +* [Vagrant installation]({{ "/install/vagrant/" | relative_url }}), for development +* [Install the software manually]({{ "/install/manual-install/" | relative_url }}) + +FixMyStreet is a web application written in Perl, using the Catalyst framework. +Installation deploys the core code and also manages its dependencies. + +If you're not technical, this can be a little daunting — if you haven't +already done so, [get in touch]({{ "/community" | relative_url }}) and ask for help. + +Please also see the instructions for [updating your code](/updating/) once it's installed. diff --git a/docs/install/install-script.md b/docs/install/install-script.md new file mode 100644 index 000000000..124434b34 --- /dev/null +++ b/docs/install/install-script.md @@ -0,0 +1,79 @@ +--- +layout: page +title: Install script +--- + +# FixMyStreet Install Script + +<p class="lead"> + If you have a new installation of Debian wheezy or Ubuntu precise, + you can use an install script to set up a basic installation of + FixMyStreet on your server. +</p> + +Note that this is just one of [many ways to install FixMyStreet]({{ "/install/" | relative_url }}). + +## Warning: installation changes your setup! + +*Warning: only use this script on a newly installed server -- it will +make significant changes to your server's setup, including modifying +your nginx setup, creating a user account, creating a database, +installing new packages, and so on.* + +## Running the script + +The script to run is called [`install-site.sh`, in our `commonlib` repository](https://github.com/mysociety/commonlib/blob/master/bin/install-site.sh). +That script's usage is as follows: + + Usage: ./install-site.sh [--default] <SITE-NAME> <UNIX-USER> [HOST] + HOST is only optional if you are running this on an EC2 instance. + --default means to install as the default site for this server, + rather than a virtualhost for HOST. + +The `<UNIX-USER>` parameter is the name of the Unix user that you want +to own and run the code. (This user will be created by the script.) + +The `HOST` parameter is a hostname for the server that will be usable +externally -- a virtualhost for this name will be created by the +script, unless you specified the `--default` option.. This parameter +is optional if you are on an EC2 instance, in which case the hostname +of that instance will be used. + +For example, if you wish to use a new user called `fms` and the +hostname `fixmystreet.127.0.0.1.xip.io` (xip.io is a very helpful service for +development, allowing easy domain/wildcard domain usage without having to edit +your hosts file), creating a virtualhost just for that hostname, you could +download and run the script with: + + curl -L -O https://github.com/mysociety/commonlib/raw/master/bin/install-site.sh + sudo sh install-site.sh fixmystreet fms fixmystreet.127.0.0.1.xip.io + +Or, if you want to set this up as the default site on an EC2 instance, +you could download the script, make it executable and then invoke it +with: + + sudo ./install-site.sh --default fixmystreet fms + +Please be aware that the last part of the installation process, +installing Perl modules, may take a long time to complete. + +When the script has finished, you should have a working copy of the +website, accessible via the hostname you supplied to the script. + +By default, the admin part of the website (`/admin`) requires a user with +superuser permission to log in. In order to use this +interface, you will need to create a username and password for one or +more superusers. To add such a user, you can use the `createsuperuser` +command, as follows: + + ubuntu@ip-10-58-66-208:~$ sudo su - fms + fms@ip-10-58-191-98:~$ cd fixmystreet + fms@ip-10-58-191-98:~/fixmystreet$ bin/createsuperuser fmsadmin@example.org password + fmsadmin@example.org is now a superuser. + +The script will install postfix to allow outgoing email; you can change the +`SMTP_SMARTHOST` and other `SMTP` variables in `conf/general.yml` to use a +different SMTP server. + +Please also see the instructions for [updating your installation](/updating/ami/). + diff --git a/docs/install/manual-install.md b/docs/install/manual-install.md new file mode 100644 index 000000000..5ddf8a30f --- /dev/null +++ b/docs/install/manual-install.md @@ -0,0 +1,251 @@ +--- +layout: page +title: Installing +--- + +# Manual installation + +<p class="lead"> + This page describes how to install the FixMyStreet platform manually. + You can use this if you're used to setting up web applications — + but the other installation options may be easier:</p> + +Note that this is just one of +[many ways to install FixMyStreet]({{ "/install/" | relative_url }}) +(the other ways are easier!). + + +## Manual installation + +If you prefer to set up each required component of FixMyStreet +yourself, proceed with the instructions below. + +### 1. Get the code + +Fetch the latest version from GitHub: + +{% highlight bash %} +$ mkdir FixMyStreet +$ cd FixMyStreet +$ git clone --recursive https://github.com/mysociety/fixmystreet.git +$ cd fixmystreet +{% endhighlight %} + +### 2. Install prerequisite packages + +#### a. Debian / Linux + +If you're using Debian or Ubuntu, then the packages to install required +dependencies are listed in `conf/packages.generic`. To install all of them you +can run e.g.: + +{% highlight bash %} +$ sudo xargs -a conf/packages.generic apt-get install +{% endhighlight %} + +A similar list of packages should work for other Debian-based distributions. +(Please let us know if you would like to contribute such a package list or +instructions for other distributions.) + +#### b. Mac OS X + +Install either MacPorts or HomeBrew (you might well have one already), and then +use the command below to install a few packages that FixMyStreet needs, for +which it's much simpler to install via a packaging system. + +##### i. MacPorts + +{% highlight bash %} +$ port install gettext p5-locale-gettext p5-perlmagick jhead postgresql91-server +{% endhighlight %} + +##### ii. HomeBrew + +{% highlight bash %} +$ brew install gettext perlmagick jhead postgresql +$ brew link gettext --force +{% endhighlight %} + +<div class="attention-box"> +gettext needs to be linked for the Locale::gettext Perl module to install; you +can unlink gettext once everything is installed. +</div> + +#### c. Other + +You need Perl 5.8, ImageMagick with the perl bindings, and gettext. +If you're expecting a lot of traffic it's recommended that you install memcached: <http://memcached.org/> + +### 3. Create a new PostgreSQL database + +The default settings assume the database is called fms and the user the same. +You can change these if you like. Using the defaults, create a user and +database using the following (do not worry if the `CREATE LANGUAGE` step gives +an error that it already exists, it might well do depending on how your +PostgreSQL was installed): + +{% highlight sql %} +$ sudo -u postgres psql +postgres=# CREATE USER fms WITH PASSWORD 'somepassword'; +CREATE ROLE +postgres=# CREATE DATABASE fms WITH OWNER fms; +CREATE DATABASE +postgres=# \c fms +postgres=# CREATE LANGUAGE plpgsql; +postgres=# \q +$ +{% endhighlight %} + +You should be able to connect to the database with `psql -U fms fms` -- if not, +you will need to investigate [how to allow access to your PostgreSQL database]({{ "/install/database" | relative_url }}). + +### 4. Install required dependencies, and other setup + +FixMyStreet uses a number of Perl modules from CPAN; to install them, +and perform other set up functions, run: + +{% highlight bash %} +$ script/setup +{% endhighlight %} + +This should tell you what it is doing as it goes. It takes some time, so +feel free to continue with further steps whilst it's running. + +<div class="attention-box helpful-hint"> +<!-- Below hopefully not needed as installed p5-locale-gettext above +<p>Note, with MacPorts you might have to specify some compilation PATHs:</p> +<pre><code>C_INCLUDE_PATH=/opt/local/include LIBRARY_PATH=/opt/local/lib bin/install_perl_modules</code></pre> +--> +<p>It is possible you may need to install some source packages to allow some of +the included modules to be built, including expat (libexpat1-dev), postgresql +(postgresql-server-dev-8.4), or the GMP math library (libgmp3-dev).</p> +</div> + +As well as installing dependencies, this script compiles our CSS (using +`bin/make_css`), installs the database schema (using `bin/update-schema`), +and compiles any translation `.mo` files (using `commonlib/bin/gettext-makemo`). + +### 5. Set up config + +The settings for FixMyStreet are defined in `conf/general.yml` using the YAML +markup language. There are some defaults in `conf/general.yml-example` which +you should copy to `conf/general.yml`; note that if you are using the Vagrant +environment, a simple `conf/general.yml` file should already have been +configured for you. + +{% highlight bash %} +$ cp conf/general.yml-example conf/general.yml +{% endhighlight %} + +The bare minimum of settings you will need to fill in or update are: + +* [FMS_DB_PASS]({{ "/customising/config/#fms_db_pass" | relative_url }}) -- this is the password for the database. (Also obviously change any other database settings you did differently here.) +* [BASE_URL]({{ "/customising/config/#base_url" | relative_url }}) -- for using the development server, set to `'http://localhost:3000/'`. This is the URL of the homepage of your FixMyStreet installation. +* [MAPIT_URL]({{ "/customising/config/#mapit_url" | relative_url }}) -- for the development server, set to `'http://localhost:3000/fakemapit/'`. This would be the URL of a MapIt installation, as and when you use one. + +Some others you might want to look at, though the defaults are enough for it to run: + +* [EMAIL_DOMAIN]({{ "/customising/config/#email_domain" | relative_url }}) -- the email domain that emails will be sent from +* [CONTACT_EMAIL]({{ "/customising/config/#contact_email" | relative_url }}) -- the email address to be used on the site for the contact us form. +* [DO_NOT_REPLY_EMAIL]({{ "/customising/config/#do_not_reply_email" | relative_url }}) -- the email address to be used on the site for e.g. confirmation emails. +* [STAGING_SITE]({{ "/customising/config/#staging_site" | relative_url }}) -- if this is 1 then all email (alerts and reports) will be sent to the contact email address. Use this for development sites. +* [PHOTO_STORAGE_OPTIONS.UPLOAD_DIR]({{ "/customising/config/#upload_dir" | relative_url }}) -- this is the location where images will be stored when they are uploaded. It should be accessible by and writeable by the FixMyStreet process. +* [GEO_CACHE]({{ "/customising/config/#geo_cache" | relative_url }}) -- this is the location where Geolocation data will be cached. It should be accessible by and writeable by the FixMyStreet process. + +If you are using Bing or Google maps you should also set one of +[BING_MAPS_API_KEY]({{ "/customising/config/#bing_maps_api_key" | relative_url }}) or +[GOOGLE_MAPS_API_KEY]({{ "/customising/config/#google_maps_api_key" | relative_url }}). + +### 6. Set up some required data + +You need to generate the data used for the `/reports` page (this is run by the +crontab, but to have it working from the start, we can run the script +manually). + +{% highlight bash %} +$ bin/update-all-reports +{% endhighlight %} + +### 7. Run + +The development server can now hopefully be run with: + +{% highlight bash %} +$ script/server +{% endhighlight %} + +The server will be accessible as <http://localhost:3000/>, and will +automatically restart if you update the code or `general.yml`. + +### Post-install: Things you might want to change + +#### Next Steps + +* Create a superuser with the `bin/createsuperuser` script to access admin site. +* [Customise your install using Templates, CSS and a Cobrand module](/customising/). +* Add contact details for authorities and categories using the admin interface. + +#### Tile server + +You will also need a tile server to serve up map tiles. FixMyStreet can +currently use tile servers such as Bing, OpenStreetMap and Google, defaulting +to OpenStreetMap. + +#### Geocoding + +Finally, you will need a geolocation service to turn addresses into longitude +and latitudes. FixMyStreet currently includes code to use Bing, Google, and +OpenStreetMap geolocation services, again defaulting to OpenStreetMap. + +#### Cron jobs + +There is an example crontab in `conf/crontab-example`. You can use that as a +base for your own user crontab. + +### Deployment + +For <a href="{{ "/glossary/#production" | relative_url }}" class="glossary__link">production</a> +use of FixMyStreet, we suggest you use Apache or nginx, with either FastCGI +or Plack/PSGI. + +#### Apache + +There is an example Apache vhost configuration file in +`conf/apache-vhost.conf.example` and `conf/httpd.conf-example`, which contain a +sample configuration and the required redirect rules. + +The sample configuration will need the following modules enabled: + +* mod_rewrite +* mod_proxy +* mod_expires +* mod_fastcgi + +For most Linux distributions you should be able to install these using the +distribution's packaging system. + +#### nginx + +There is an example nginx configuration in `conf/nginx.conf.example`. With +nginx, you need to run the application service separately - the +`conf/sysvinit.example` or `conf/systemd.example` files are example scripts you +could use to run it as a daemon. + +#### Check it's working + +At this point you should be able to restart the webserver and see your +FixMyStreet installation at the configured URL. See our +[testing page]({{ "/install/testing/" | relative_url }}) on running the test +suite. + +## Problems? + +See some [troubleshooting hints]({{ "/install/troubleshooting/" | relative_url }}) if +something's not working for you. + +## When you've finished + +Please see the instructions for [updating your code](/updating/) once it's installed. + +If you want to know which bits of FixMyStreet are in which directory, see this +[summary of the directory structure](/directory_structure). diff --git a/docs/install/testing.md b/docs/install/testing.md new file mode 100644 index 000000000..b63dfffcc --- /dev/null +++ b/docs/install/testing.md @@ -0,0 +1,47 @@ +--- +layout: page +title: Testing +--- + +# Testing + +<p class="lead"> + This page describes how to run FixMyStreet’s test suite. +</p> + +## Server testing + +You can run the test suite for the backend codebase by running the following +command in the `fixmystreet` directory: + +{% highlight bash %} +$ script/test +{% endhighlight %} + +The `master` branch of the repository should always be passing all tests for +our developers and on mySociety's servers. + +## Client testing + +To run the front-end tests, you will need to install +[Cypress](https://cypress.io) using `npm` (not direct download), and the +`cypress` command needs to be on your `PATH`. Then you can run the front-end +tests headlessly using: + +{% highlight bash %} +$ bin/browser-tests run +{% endhighlight %} + +This uses its own test server and database, not affecting your development +database. If you wish to run the tests interactively for debugging, use: + +{% highlight bash %} +$ bin/browser-tests open +{% endhighlight %} + +If you're running FixMyStreet in a Vagrant box, you can use this script to run +the test server in the VM and Cypress outside of it: + +{% highlight bash %} +$ bin/browser-tests --vagrant run +{% endhighlight %} diff --git a/docs/install/troubleshooting.md b/docs/install/troubleshooting.md new file mode 100644 index 000000000..fdae90a07 --- /dev/null +++ b/docs/install/troubleshooting.md @@ -0,0 +1,116 @@ +--- +layout: page +title: Installation troubleshooting +--- + +# Installation troubleshooting + +<p class="lead"> + If you've installed FixMyStreet using an automated method such as the + <a href="{{ "/install/install-script" | relative_url }}">installation script</a> + or the + <a href="{{ "/install/ami" | relative_url }}">AMI</a>, you should be good to go. + However, if you've done a + <a href="{{ "/install/manual-install" | relative_url }}">manual install</a>, + sometimes you might bump into a problem if your system is different from + what FixMyStreet expects. These hints might help. +</p> + +## Common Problems + +* [locale](#locale): must be installed +* [template caching](#template-caching) +* [Image::Magick perl module](#image-magick) +* [missing Perl modules](#missing-perl-modules) +* [No styling (CSS)](#no-styling) +* [Bad YAML format in config settings](#bad-yaml): no response or 500 error +* [Change of config being ignored](#requires-restart): requires restart + + +<a name="locale"> </a> + +### locale: must be installed + +By default, FixMyStreet uses the `en_GB.UTF-8` +<a href="{{ "/glossary/#locale" | relative_url }}" class="glossary__link">locale</a>. +If it is not installed then it may not start. You need this locale on your +system even if you're planning on running your site in a different language. + +If you've changed the language or languages you're supporting, you must have +the appropriate locales installed for each of those too. + +Check to see what locales your system currently supports with: + +<pre><code>locale -a +</code></pre> + +<a name="template-caching"> </a> + +### Template caching + +FixMyStreet caches compiled +<a href="{{ "/glossary/#template" | relative_url }}" class="glossary__link">templates</a> +alongside the source files, so the `templates/web/` directory needs to be writable +by the process that is running FixMyStreet. + +When everything is running OK, you'll see compiled template files with a +`.ttc` suffix appearing alongside the `.html` ones. + +<a name="image-magick"> </a> + +### Image::Magick perl module + +If your OS has a way to install a binary version of `Image::Magick` then we recommend +that you do that rather than install via CPAN. + +<a name="missing-perl-modules"> </a> + +### Missing Perl modules + +We think we've included all the modules you should need to run and develop +FixMyStreet on your machine but if we've missed one, please let us know. If you +need a new module for something you're developing, please get in touch as +adding things to carton (the mechanism FixMyStreet uses to manage Perl +dependencies) is currently not as simple as we would like. + +If you tried to run a script in the `bin` directory manually, it failed with +missing Perl module errors, you can try running it using `bin/cron-wrapper`. +This sets up the FixMyStreet environment for any script that doesn't do it +itself (though all provided scripts should do). + +<a name="no-styling"> </a> + +### No styling (CSS) + +FixMyStreet's stylesheets are built using SASS -- the `.css` files themselves +must be created (they are not shipped as compiled files in the repo). The +installation process does this for you, but if you did a manual install +and forgot to do it, or you've changed the `.scss` files and those changes +aren't showing up, do `bin/make_css` to create them. + +<a name="bad-yaml"> </a> + +### Bad YAML format in config settings: no response or 500 error + +When you change any of the [config settings]({{ "/customising/config/" | relative_url }}) +in `conf/general.yml` (which you will do!), make sure you don't break the YAML +format. If FixMyStreet can't read that file cleanly, it may time out, causing fastcgi +to eventually respond with a 500 Internal Server Error. + +There are YAML lint tools available for checking the well-formedness of YAML files. +Remember that your config settings may include sensitive information (your database +password, for example) so be sensible before using online validators. + +Simple things to be wary of: always quote strings that themselves contain a colon, +be sure to close any open quotes, and avoid using tabs. + +<a name="requires-restart"> </a> + +### Change of config being ignored: requires restart + +If you make changes (such as editing the `conf/general.yml` file) while FixMyStreet +is running, you may need to restart the process for the change to take effect. + +For example, restart the Catalyst FastCGI server with: + + $ sudo service fixmystreet restart diff --git a/docs/install/vagrant.md b/docs/install/vagrant.md new file mode 100644 index 000000000..98f846644 --- /dev/null +++ b/docs/install/vagrant.md @@ -0,0 +1,70 @@ +--- +layout: page +title: Vagrant +--- + +# FixMyStreet using Vagrant + +<p class="lead"> +Vagrant provides an easy method to set up virtual development environments — for +further information see <a href="http://www.vagrantup.com">the Vagrant website</a>. +We bundle an example Vagrantfile in the repository, which runs the +<a href="{{ "/install/install-script/" | relative_url }}">install script</a> for you. +</p> + +Note that this is just one of [many ways to install FixMyStreet]({{ "/install/" | relative_url }}). + +<div class="attention-box warning"> + Vagrant is only suitable for use as a + <a href="{{ "/glossary/#development" | relative_url }}" class="glossary__link">development</a> + server — <strong>do not</strong> use it in + <a href="{{ "/glossary/#production" | relative_url }}" class="glossary__link">production</a>! +</div> + +This pages describes how to use Vagrant to create a development environment +where you can run the test suite and the development server, and make changes +to the codebase. + +The advantage of using Vagrant is that it lets you run FixMyStreet within a +virtual machine (VM), with its own system software and environment set up +entirely independently of the actual machine it's running on. This means you +don't have to worry about it interfering with your own machine's operating +system. The main disadvantage is that the virtual machine runs somewhat slower, +and makes more demands on the processor, than FixMyStreet running natively. + +The basic process is to create a base virtual machine, and then provision it +with the software packages and setup needed. There are several ways to do this, +including Chef, Puppet, or the existing FixMyStreet install script (which is +the method used in the example below). The supplied scripts will create a +Vagrant VM based on the server edition of Ubuntu 12.04 LTS. This contains +everything you need to work on FixMyStreet. + +1. Clone the repository, `cd` into it and run vagrant. This will provision the + system and can take some time. + + git clone --recursive https://github.com/mysociety/fixmystreet.git + cd fixmystreet + vagrant up --no-color + +## Working with the Vagrant box + +You've now got a local FixMyStreet development server to work with. You can +edit the files locally (which means you can use your favourite text editor, for +example) and the changes will be reflected on the virtual machine. + +To start the dev server: + + vagrant ssh + + # You are now in a terminal on the virtual machine + cd fixmystreet + + # run the dev server + script/server + +The server will now be running and you can visit it at the address +`http://fixmystreet.127.0.0.1.xip.io:3000/` + +The username and password to access the admin (at address +`http://fixmystreet.127.0.0.1.xip.io:3000/admin/`) will have been shown at the +end of the `vagrant up` output. |