diff options
Diffstat (limited to 'doc/INSTALL.md')
| -rw-r--r-- | doc/INSTALL.md | 297 |
1 files changed, 221 insertions, 76 deletions
diff --git a/doc/INSTALL.md b/doc/INSTALL.md index 3a911cbc8..04cdb1352 100644 --- a/doc/INSTALL.md +++ b/doc/INSTALL.md @@ -1,4 +1,123 @@ -These instructions assume Debian Squeeze or Ubuntu 10.04 LTS. +# Installation Script and AMI + +The easiest options for installating Alaveteli for evaluation +are to use our install script or to use the AMI (Amazon Machine +Image) to create an instance on Amazon EC2. These options are +described below. If you would prefer to install the site +manually, please go to the Manual Installation section below. + +## Installing from an AMI (Amazon Machine Image) + +To help people try out Alaveteli, we have created an AMI (Amazon +Machine Image) with a basic installation of Alaveteli, which you +can use to create a running server on an Amazon EC2 instance. +This creates an instance that runs in development mode, so we +wouldn't recommend you use it for a production system without +changing the configuration. + +Unfortunately, Alaveteli will not run properly on a free Micro +instance due to the low amount of memory available on those +instances; you will need to use at least a Small instance, which +Amazon will charge for. + +The AMI can be found in the EU West (Ireland) region, with the +ID ami-8603f4f1 and name “Basic Alaveteli installation +2014-01-29”. You can launch an instance based on that AMI with +[this link](https://console.aws.amazon.com/ec2/home?region=eu-west-1#launchAmi=ami-8603f4f1). + +When you create an EC2 instance based on that AMI, make sure +that you choose Security Groups that allows at least inbound +HTTP, HTTPS, SSH and, if you want to test incoming mail as well, +SMTP. + +When your EC2 instance is launched, you will be able to log in +as the `ubuntu` user. This user can `sudo` freely to run +commands as root. However, the code is actually owned by (and +runs as) the `alaveteli` user. After creating the instance, you +may want to edit a configuration file to customize the site's +configuration. That configuration file is +`/var/www/alaveteli/alaveteli/config/general.yml`, which can be +edited with: + + ubuntu@ip-10-58-191-98:~$ sudo su - alaveteli + alaveteli@ip-10-58-191-98:~$ cd alaveteli + alaveteli@ip-10-58-191-98:~/alaveteli$ nano config/general.yml + +Then you should restart the Thin webserver with: + + alaveteli@ip-10-58-191-98:~/alaveteli$ logout + ubuntu@ip-10-58-191-98:~$ sudo /etc/init.d/alaveteli restart + +If you find the hostname of your EC2 instance from the AWS +console, you should then be able to see the site at +`http://your-ec2-hostname.eu-west-1.compute.amazonaws.com` + +If you have any problems or questions, please ask on the +[Alaveteli Google Group](https://groups.google.com/forum/#!forum/alaveteli-dev) +or [report an issue](https://github.com/mysociety/alaveteli/issues?state=open). + +## Installing with the Installation Script + +If you have a clean installation of Debian squeeze or Ubuntu +precise, you can use an install script in our commonlib +repository to set up a working instance of Alaveteli. This is +not suitable for production (it runs in development mode, for +example) but should set up a functional installation of the +site. + +**Warning: only use this script on a newly installed server – it +will make significant changes to your server’s setup, including +modifying your nginx setup, creating a user account, creating a +database, installing new packages etc.** + +To download the script, run the following command: + + curl -O https://raw.github.com/mysociety/commonlib/master/bin/install-site.sh + +If you run this script with `sh install-site.sh`, you'll see its +usage message: + + Usage: ./install-site.sh [--default] <SITE-NAME> <UNIX-USER> [HOST] + HOST is only optional if you are running this on an EC2 instance. + --default means to install as the default site for this server, + rather than a virtualhost for HOST. + +In this case `<SITE-NAME>` should be `alaveteli`. `<UNIX-USER>` +is the name of the Unix user that you want to own and run the +code. (This user will be created by the script.) + +The `HOST` parameter is a hostname for the server that will be +usable externally – a virtualhost for this name will be created +by the script, unless you specified the `--default` option. This +parameter is optional if you are on an EC2 instance, in which +case the hostname of that instance will be used. + +For example, if you wish to use a new user called `alaveteli` +and the hostname `alaveteli.127.0.0.1.xip.io`, creating a +virtualhost just for that hostname, you could download and run +the script with: + + sudo sh install-site.sh alaveteli alaveteli alaveteli.127.0.0.1.xip.io + +([xip.io](http://xip.io/) is a helpful domain for development.) + +Or, if you want to set this up as the default site on an EC2 +instance, you could download the script, make it executable and +then invoke it with: + + sudo ./install-site.sh --default alaveteli alaveteli + +When the script has finished, you should have a working copy of +the website, accessible via the hostname you supplied to the +script. + +If you have any problems or questions, please ask on the +[Alaveteli Google Group](https://groups.google.com/forum/#!forum/alaveteli-dev) +or [report an issue](https://github.com/mysociety/alaveteli/issues?state=open). + +# Manual Installation + +These instructions assume Debian Squeeze (64-bit) or Ubuntu 12.04 LTS (precise). [Install instructions for OS X](https://github.com/mysociety/alaveteli/wiki/OS-X-Quickstart) are under development. Debian Squeeze is the best supported deployment platform. @@ -9,12 +128,12 @@ As an aid to evaluation, there is an [Amazon AMI](https://github.com/mysociety/alaveteli/wiki/Alaveteli-ec2-ami) with all these steps configured. It is *not* production-ready. -# Get Alaveteli +## Get Alaveteli To start with, you may need to install git, e.g. with `sudo apt-get install git-core` -Next, get hold of the Alaveteli source code from github: +Next, get hold of the Alaveteli source code from github: git clone https://github.com/mysociety/alaveteli.git cd alaveteli @@ -25,19 +144,23 @@ master branch (which always contains the latest stable release): git checkout master -# Package pinning +## Package pinning -You need to configure [apt-pinning](http://wiki.debian.org/AptPreferences#Pinning-1) preferences in order to prevent packages being pulled from the debian testing distribution in preference to the stable distribution once you have added the testing repository as described below. +You need to configure [apt-pinning](http://wiki.debian.org/AptPreferences#Pinning-1) preferences in order to prevent packages being pulled from the debian wheezy distribution in preference to the stable distribution once you have added the wheezy repository as described below. -In order to configure apt-pinning and to keep most packages coming from the Debian stable repository while installing the ones required from testing and the mySociety repository you need to run the following commands: +In order to configure apt-pinning and to keep most packages coming from the Debian stable repository while installing the ones required from wheezy and the mySociety repository you need to run the following commands: echo "Package: *" >> /tmp/preferences - echo "Pin: release a=testing">> /tmp/preferences + echo "Pin: release a=squeeze-backports">> /tmp/preferences + echo "Pin-Priority: 200" >> /tmp/preferences + echo "" >> /tmp/preferences + echo "Package: *" >> /tmp/preferences + echo "Pin: release a=wheezy">> /tmp/preferences echo "Pin-Priority: 50" >> /tmp/preferences sudo cp /tmp/preferences /etc/apt/ rm /tmp/preferences - -# Install system dependencies + +## Install system dependencies These are packages that the software depends on: third-party software used to parse documents, host the site, etc. There are also packages @@ -48,7 +171,8 @@ If you are running Debian, add the following repositories to `/etc/apt/sources.list` and run `apt-get update`: deb http://debian.mysociety.org squeeze main - deb http://ftp.debian.org/debian/ testing main non-free contrib + deb http://ftp.debian.org/debian/ wheezy main non-free contrib + deb http://backports.debian.org/debian-backports squeeze-backports main contrib non-free The repositories above allow us to install the packages `wkhtmltopdf-static` and `bundler` using `apt`; so if you're running @@ -65,24 +189,16 @@ Some of the files also have a version number listed in config/packages - check that you have appropriate versions installed. Some also list "|" and offer a choice of packages. -# Install Ruby dependencies +## Install Ruby dependencies -Install rubygems 1.6.2 (we're not using the Debian package because we -need an older version; see "Troubleshooting" below for an -explanation): - - wget http://rubyforge.org/frs/download.php/74445/rubygems-1.6.2.tgz -O /tmp/rubygems-1.6.2.tgz - tar zxvf /tmp/rubygems-1.6.2.tgz -C /tmp/ - sudo ruby1.8 /tmp/rubygems-1.6.2/setup.rb - -To install Alaveteli's Ruby dependencies, we also need to install +To install Alaveteli's Ruby dependencies, we need to install bundler. In Debian, this is provided as a package (installed as part of the package install process above). You could also install it as a gem: sudo gem1.8 install bundler - -# Install mySociety libraries + +## Install mySociety libraries You will also want to install mySociety's common ruby libraries and the Rails code. Run: @@ -91,7 +207,7 @@ code. Run: to fetch the contents of the submodules. -## Packages customised by mySociety +### Packages customised by mySociety Debian users should add the mySociety debian archive to their `/etc/apt/sources.list` as described above. Doing this and following @@ -118,7 +234,7 @@ use the Debian package compiled by mySociety (see link in [issue 305](https://github.com/mysociety/alaveteli/issues/305)) -# Configure Database +## Configure Database There has been a little work done in trying to make the code work with other databases (e.g. SQLite), but the currently supported database is @@ -138,13 +254,14 @@ username and password of your postgres database. Make sure that the user specified in database.yml exists, and has full permissions on these databases. As they need the ability to turn off constraints whilst running the tests they also need to be a superuser. -(See http://dev.rubyonrails.org/ticket/9981) - +If you don't want your database user to be a superuser, you can add a line +`disable_constraints: false` to the test config in database.yml, as seen in database.yml-example + You can create a `foi` user from the command line, thus: # su - postgres $ createuser -s -P foi - + And you can create a database thus: $ createdb -T template0 -E SQL_ASCII -O foi foi_production @@ -157,24 +274,25 @@ data that may not be valid UTF (for example, data originating from various broken email clients that's not 8-bit clean), it's safer to be able to store *anything*, than reject data at runtime. -# Configure email +## Configure email You will need to set up an email server (MTA) to send and receive emails. Full configuration for an MTA is beyond the scope of this document, though we describe an example configuration for Exim in `INSTALL-exim4.md`. -Note that in development mode, mail is handled by default by mailcatcher -so that you can see the mails in a browser - see http://mailcatcher.me/ -for more details. +Note that in development mode, mail is handled by default by mailcatcher +so that you can see the mails in a browser - see http://mailcatcher.me/ +for more details. Start mailcatcher by running `bundle exec mailcatcher` +in your application directory. -## Minimal +### Minimal If you just want to get the tests to pass, you will at a minimum need to allow sending emails via a `sendmail` command (a requirement met, for example, with `sudo apt-get install exim4`). -## Detailed +### Detailed When an authority receives an email, the email's `reply-to` field is a magic address which is parsed and consumed by the Rails app. @@ -207,7 +325,7 @@ A well-configured installation of this code will separately have had Exim make a backup copy of the email in a separate mailbox, just in case. -# Set up configs +## Set up configs Copy `config/general.yml-example` to `config/general.yml` and edit to your taste. @@ -226,11 +344,11 @@ performance management system. By default, monitoring is switched off by the `agent_enabled: false` setting. See https://github.com/newrelic/rpm for instructions on switching on local and remote performance analysis. -# Deployment +## Deployment In the 'alaveteli' directory, run: - ./script/rails-post-deploy + script/rails-post-deploy (This will need execute privs so `chmod 755` if necessary.) This sets up directory structures, creates logs, installs/updates themes, runs @@ -245,16 +363,16 @@ If you want some dummy data to play with, you can try loading the fixtures that the test suite uses into your development database. You can do this with: - ./script/load-sample-data + script/load-sample-data Next we need to create the index for the search engine (Xapian): - ./script/rebuild-xapian-index + script/rebuild-xapian-index If this fails, the site should still mostly run, but it's a core component so you should really try to get this working. -# Run the Tests +## Run the Tests Make sure everything looks OK: @@ -266,7 +384,7 @@ workaround). You might be able to move on to the next step, depending on how serious they are, but ideally you should try to find out what's gone wrong. -## glibc bug workaround +### glibc bug workaround There's a [bug in glibc](http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=637239) @@ -277,11 +395,11 @@ it's not as of version `2.11.3-2`. Until it's fixed (e.g. `libc6 2.13-26` does work), you can get the tests to pass by setting `export LD_PRELOAD=/lib/libuuid.so.1`. -# Run the Server +## Run the Server Run the following to get the server running: - ./script/server --environment=development + bundle exec rails server --environment=development By default the server listens on all interfaces. You can restrict it to the localhost interface by adding ` --binding=127.0.0.1` @@ -289,7 +407,7 @@ localhost interface by adding ` --binding=127.0.0.1` The server should have told you the URL to access in your browser to see the site in action. -# Administrator privileges +## Administrator privileges The administrative interface is at the URL `/admin`. @@ -301,7 +419,7 @@ There is an emergency user account which can be accessed via `/admin?emergency=1`, using the credentials `ADMIN_USERNAME` and `ADMIN_PASSWORD`, which are set in `general.yml`. To bootstrap the first `super` level accounts, you will need to log in as the emergency -user. +user. You can disable the emergency user account by setting `DISABLE_EMERGENCY_USER` to `true` in `general.yml`. Users with the superuser role also have extra privileges in the website frontend, such as being able to categorise any request, being @@ -312,11 +430,11 @@ in the front end. It is possible completely to override the administrator authentication by setting `SKIP_ADMIN_AUTH` to `true` in `general.yml`. -# Cron jobs and init scripts +## Cron jobs and init scripts -`config/crontab.ugly` contains the cronjobs run on WhatDoTheyKnow. +`config/crontab-example` contains the cronjobs run on WhatDoTheyKnow. It's in a strange templating format they use in mySociety. mySociety -render the "ugly" file to reference absolute paths, and then drop it +render the example file to reference absolute paths, and then drop it in `/etc/cron.d/` on the server. The `ugly` format uses simple variable substitution. A variable looks @@ -328,7 +446,7 @@ like `!!(*= $this *)!!`. The variables are: port with a path to the directory where your Alaveteli software installation lives, e.g. `/var/www/` * `vhost_dir`: the entire path to the directory where the software is - served from. -- you should replace this with a path to the + served from. -- you should replace this with a path to the directory where your Alaveteli software installation lives, e.g. `/var/www/` * `vcspath`: the name of the alaveteli checkout, e.g. `alaveteli`. @@ -337,8 +455,20 @@ like `!!(*= $this *)!!`. The variables are: * `user`: the user that the software runs as * `site`: a string to identify your alaveteli instance -There is a dumb python script at `script/make-crontab` which you can -edit and run to do some basic substitution for you. +There is a rake task that will help to rewrite this file into +one that is useful to you, which can be invoked with: + + bundle exec rake config_files:convert_crontab \ + DEPLOY_USER=deploy \ + VHOST_DIR=/dir/above/alaveteli \ + VCSPATH=alaveteli \ + SITE=alaveteli \ + CRONTAB=config/crontab-example > crontab + +You should change the `DEPLOY_USER`, `VHOST_DIR`, `VCSPATH` and +`SITE` environment variables to match your server and +installation. You should also edit the resulting `crontab` file +to customize the `MAILTO` variable. One of the cron jobs refers to a script at `/etc/init.d/foi-alert-tracks`. This is an init script, a copy of @@ -361,14 +491,14 @@ discussion of where to find this program, and how you might replace it. This [one line script](https://gist.github.com/3741194) can install this program system-wide. -# Set up production web server +## Set up production web server It is not recommended to run the website using the default Rails web server. There are various recommendations here: http://rubyonrails.org/deploy -We usually use Passenger / mod_rails. The file at `conf/httpd.conf` -contains the WhatDoTheyKnow settings. At a minimum, you should +We usually use Passenger / mod_rails. The file at `conf/httpd.conf-example` +gives you an example config file for WhatDoTheyKnow. At a minimum, you should include the following in an Apache configuration file: PassengerResolveSymlinksInDocumentRoot on @@ -378,11 +508,39 @@ Under all but light loads, it is strongly recommended to run the server behind an http accelerator like Varnish. A sample varnish VCL is supplied in `../conf/varnish-alaveteli.vcl`. +It's strongly recommended that you run the site over SSL. (Set FORCE_SSL to true in +config/general.yml). For this you will need an SSL certificate for your domain and you will +need to configure an SSL terminator to sit in front of Varnish. If you're already using Apache +as a web server you could simply use Apache as the SSL terminator. A minimal configuration +would look something like this: + +<VirtualHost *:443> + ServerName www.yourdomain + + ProxyRequests Off + ProxyPreserveHost On + ProxyPass / http://localhost:80/ + ProxyPassReverse / http://localhost:80/ + RequestHeader set X-Forwarded-Proto 'https' + + SSLEngine on + SSLProtocol all -SSLv2 + SSLCipherSuite ALL:!ADH:!EXPORT:!SSLv2:RC4+RSA:+HIGH:+MEDIUM + + SSLCertificateFile /etc/apache2/ssl/ssl.crt + SSLCertificateKeyFile /etc/apache2/ssl/ssl.key + SSLCertificateChainFile /etc/apache2/ssl/sub.class2.server.ca.pem + SSLCACertificateFile /etc/apache2/ssl/ca.pem + SetEnvIf User-Agent ".*MSIE.*" nokeepalive ssl-unclean-shutdown +</VirtualHost> + +Notice the line "RequestHeader" that sets the X-Forwarded-Proto header. This is important. This ultimately tells Rails that it's serving a page over https and so it knows to include that in any absolute urls it serves. + Some [production server best practice notes](https://github.com/mysociety/alaveteli/wiki/Production-Server-Best-Practices) are evolving on the wiki. -# Upgrading Alaveteli +## Upgrading Alaveteli The developer team policy is that the master branch in git should always contain the latest stable release. Therefore, in production, @@ -404,23 +562,23 @@ release. Failure to do so means that any new words added to the Alaveteli source code will appear in your website in English by default. If your translations didn't make it to the latest release, you will need to download the updated `app.po` for your locale from -Transifex and save it in the `locales/` folder. +Transifex and save it in the `locale/` folder. You should always run the script `scripts/rails-post-deploy` after each deployment. This runs any database migrations for you, plus various other things that can be automated for deployment. -# Troubleshooting +## Troubleshooting * **Incoming emails aren't appearing in my Alaveteli install** - + First, you need to check that your MTA is delivering relevant incoming emails to the `script/mailin` command. There are various ways of setting your MTA up to do this; we have documented one way of doing it in Exim at `doc/INSTALL-exim4.conf`, including a command you can use to check that the email routing is set up correctly. - + Second, you need to test that the mailin script itself is working correctly, by running it from the command line, First, find a valid "To" address for a request in your system. You can do this @@ -431,7 +589,7 @@ various other things that can be automated for deployment. Loading development environment (Rails 2.3.14) >> InfoRequest.find_by_url_title("why_do_you_have_such_a_fancy_dog").incoming_email => "request-101-50929748@localhost" - + Now take the source of a valid email (there are some sample emails in `spec/fixtures/files/`); edit the `To:` header to match this address; and then pipe it through the mailin script. A non-zero exit code @@ -447,7 +605,7 @@ various other things that can be automated for deployment. `CONTACT_EMAIL` (from your `general.yml` file). A common problem is for the user that the MTA runs as not to have write access to `files/raw_emails/`. - + * **Various tests fail with "*Your PostgreSQL connection does not support unescape_bytea. Try upgrading to pg 0.9.0 or later.*"** @@ -470,29 +628,16 @@ various other things that can be automated for deployment. Normally, the encoding should just work, but under some circumstances it appears that `elinks` ignores the parameters passed to it from Alaveteli. - + To force `elinks` always to treat input as UTF8, add the following to `/etc/elinks/elinks.conf`: - + set document.codepage.assume = "utf-8" set document.codepage.force_assumed = 1 - You should also check that your locale is set up correctly. See + You should also check that your locale is set up correctly. See [https://github.com/mysociety/alaveteli/issues/128#issuecomment-1814845](this issue followup) for further discussion. - -* **I'm getting lots of `SourceIndex.new(hash) is deprecated` errors when running the tests** - - The latest versions of rubygems contain a large number of noisy - deprecation warnings that you can't turn off individually. Rails - 2.x isn't under active development so isn't going to get fixed (in - the sense of using a non-deprecated API). So the only vaguely - sensible way to avoid this noisy output is to downgrade rubygems. - - For example, you might do this by uninstalling your - system-packaged rubygems, and then installing the latest rubygems - from source, and finally executing `sudo gem update --system - 1.6.2`. * **I'm seeing `rake: command not found` when running the post install script |