--- layout: page title: Manual installation --- # Manual Installation

The following instructions describe the step-by-step process for installing Alaveteli. You don't necessarily need to do it this way: it's usually easier to use the installation script or the Amazon EC2 AMI.

Note that there are [other ways to install Alaveteli]({{ site.baseurl }}docs/installing/).
## Configure the Operating System ### Target operating system These instructions assume a 64-bit version of Debian 6 (Wheezy), Debian 7 (Squeeze) or Ubuntu 12.04 LTS (Precise). Debian is the best supported deployment platform. We also have instructions for [installing on MacOS]({{ site.baseurl }}docs/installing/macos/). ### Set the locale **Debian Wheezy or Squeeze** Follow the [Debian guide](https://wiki.debian.org/Locale#Standard) for configuring the locale of the operating system. Generate the locales you wish to make available. When the interactive screen asks you to pick a default locale, choose "None", as the SSH session will provide the locale required. dpkg-reconfigure locales Start a new SSH session to use your SSH locale. ### Update the OS Update the Operating System with the latest packages apt-get update -y apt-get upgrade -y `sudo` is not installed on Debian by default. Install it along with `vim` (a useful text editor) and `git` (the version control tool we'll use to get a copy of the Alaveteli code). apt-get install -y sudo vim git-core ### Prepare to install system dependencies using OS packages These are packages that the software depends on: third-party software used to parse documents, host the site, and so on. There are also packages that contain headers necessary to compile some of the gem dependencies in the next step. #### Using other repositories to get more recent packages Add the following repositories to `/etc/apt/sources.list`: **Debian Squeeze** cat > /etc/apt/sources.list.d/debian-extra.list <> /etc/apt/preferences < /etc/apt/sources.list.d/debian-extra.list < /etc/apt/sources.list.d/ubuntu-extra.list <> /etc/apt/preferences < /etc/apt/sources.list.d/mysociety-debian.list <> /etc/apt/preferences < /tmp/update-template.sql sudo -u postgres psql -f /tmp/update-template.sql rm /tmp/update-template.sql Then create the databases: sudo -u postgres createdb -T template_utf8 -O foi alaveteli_production sudo -u postgres createdb -T template_utf8 -O foi alaveteli_test sudo -u postgres createdb -T template_utf8 -O foi alaveteli_development ## Configure email You will need to set up an email server – or Mail Transfer Agent (MTA) – to send and receive emails. Full configuration for an MTA is beyond the scope of this document -- see the guide for [configuring the Exim4 or Postfix MTAs]({{ site.baseurl }}docs/installing/email/). Note that in development mode mail is handled by [`mailcatcher`](http://mailcatcher.me/) by default so that you can see the mails in a browser. Start mailcatcher by running `bundle exec mailcatcher` in the application directory. ## Configure Alaveteli Alaveteli has three main configuration files: - `config/database.yml`: Configures Alaveteli to communicate with the database - `config/general.yml`: The general Alaveteli application settings - `config/newrelic.yml`: Configuration for the [NewRelic](http://newrelic.com) monitoring service Copy the configuration files and update their permissions: cp /var/www/alaveteli/config/database.yml-example /var/www/alaveteli/config/database.yml cp /var/www/alaveteli/config/general.yml-example /var/www/alaveteli/config/general.yml cp /var/www/alaveteli/config/newrelic.yml-example /var/www/alaveteli/config/newrelic.yml chown alaveteli:alaveteli /var/www/alaveteli/config/{database,general,newrelic}.yml chmod 640 /var/www/alaveteli/config/{database,general,newrelic}.yml ### database.yml Now you need to set up the database config file so that the application can connect to the postgres database. Edit each section to point to the relevant local postgresql database. Example `development` section of `config/database.yml`: development: adapter: postgresql template: template_utf8 database: alaveteli_development username: foi password: secure-password-here host: localhost port: 5432 Make sure that the user specified in `database.yml` exists, and has full permissions on these databases. As the user needs the ability to turn off constraints whilst running the tests they also need to be a superuser. If you don't want your database user to be a superuser, you can add this line to the `test` section in `database.yml` (as seen in `config/database.yml-example`): constraint_disabling: false ### general.yml We have a full [guide to Alaveteli configuration]({{ site.baseurl }}docs/customising/config/) which covers all the settings in `config/general.yml`. The default settings for frontpage examples are designed to work with the dummy data shipped with Alaveteli; once you have real data, you should certainly edit these. The default theme is the "Alaveteli" theme. When you run `rails-post-deploy` (see below), that theme gets installed automatically. ### newrelic.yml This file contains configuration information for the New Relic performance management system. By default, monitoring is switched off by the `agent_enabled: false` setting. See New Relic's [remote performance analysis](https://github.com/newrelic/rpm) instructions for switching it on for both local and remote analysis. ## Deployment You should run the `rails-post-deploy` script after each new software upgrade: sudo -u alaveteli RAILS_ENV=production \ /var/www/alaveteli/script/rails-post-deploy This sets up installs Ruby dependencies, installs/updates themes, runs database migrations, updates shared directories and runs other tasks that need to be run after a software update. That the first time you run this script can take a *long* time, as it must compile native dependencies for `xapian-full`. Precompile the static assets: sudo -u alaveteli \ bash -c 'RAILS_ENV=production cd /var/www/alaveteli && \ bundle exec rake assets:precompile' Create the index for the search engine (Xapian): sudo -u alaveteli RAILS_ENV=production \ /var/www/alaveteli/script/rebuild-xapian-index If this fails, the site should still mostly run, but it's a core component so you should really try to get this working.
Note that we set RAILS_ENV=production. Use RAILS_ENV=development if you are installing Alaveteli to make changes to the code.
## Configure the Application Server Alaveteli can run under many popular application servers. mySociety recommends the use of [Phusion Passenger](https://www.phusionpassenger.com) (AKA mod_rails) or [thin](http://code.macournoyer.com/thin). ### Using Phusion Passenger Passenger is the recommended application server as it is well proven in production environments. It is implemented as an Apache mod, so it cannot be run independently. apt-get install -y libapache2-mod-passenger See later in the guide for configuring the Apache web server with Passenger. ### Using Thin Thin is a lighter-weight application server which can be run independently of a web server. Run the following to get the server running: cd /var/www/alaveteli bundle exec thin \ --environment=production \ --user=alaveteli \ --group=alaveteli \ start By default the server listens on all interfaces. You can restrict it to the localhost interface by adding `--binding=127.0.0.1` The server should have told you the URL to access in your browser to see the site in action. You can daemonize the process by starting it with the `--daemonize` option. ## Cron jobs and init scripts The crontab and init scripts use the `.ugly` file format, which is a strange templating format used by mySociety. The `ugly` format uses simple variable substitution. A variable looks like `!!(*= $this *)!!`. ### Generate crontab `config/crontab-example` contains the cron jobs that run on WhatDoTheyKnow. mySociety render the example file to reference absolute paths, and then drop it in `/etc/cron.d/` on the server. **Template Variables:** * `vhost_dir`: the full path to the directory where alaveteli is checked out. e.g. If your checkout is at `/var/www/alaveteli` then set this to `/var/www` * `vcspath`: the name of the directory that contains the alaveteli code. e.g. `alaveteli` * `user`: the user that the software runs as * `site`: a string to identify your alaveteli instance * `mailto`: The email address that cron output will be sent to There is a rake task that will help to rewrite this file into one that is useful to you. Change the variables to suit your installation. bundle exec rake config_files:convert_crontab \ DEPLOY_USER=deploy \ VHOST_DIR=/var/www \ VCSPATH=alaveteli \ SITE=alaveteli \ MAILTO=cron-alaveteli@example.org \ CRONTAB=config/crontab-example > /etc/cron.d/alaveteli ### Generate alert daemon One of the cron jobs refers to a script at `/etc/init.d/foi-alert-tracks`. This is an init script, which can be generated from the `config/alert-tracks-debian.ugly` template. **Template Variables:** * `vhost_dir`: the full path to the directory where alaveteli is checked out. e.g. If your checkout is at `/var/www/alaveteli` then set this to `/var/www` * `user`: the user that the software runs as There is a rake task that will help to rewrite this file into one that is useful to you. Change the variables to suit your installation. bundle exec rake config_files:convert_init_script \ DEPLOY_USER=deploy \ VHOST_DIR=/var/www \ SCRIPT_FILE=config/alert-tracks-debian.ugly > /etc/init.d/foi-alert-tracks ### Generate varnish purge daemon `config/purge-varnish-debian.ugly` is a similar init script, which is optional and not required if you choose not to run your site behind Varnish (see below). **Template Variables:** * `daemon_name`: The name of the daemon. Set this to `purge-varnish`. * `vhost_dir`: the full path to the directory where alaveteli is checked out. e.g. If your checkout is at `/var/www/alaveteli` then set this to `/var/www` * `user`: the user that the software runs as This template does not yet have a rake task to generate it. ### Init script permissions Either tweak the file permissions to make the scripts executable by your deploy user, or add the following line to your sudoers file to allow these to be run by your deploy user (named `deploy` in this case). deploy ALL = NOPASSWD: /etc/init.d/foi-alert-tracks, /etc/init.d/foi-purge-varnish There is also an example config for stopping and starting the Alaveteli app server as a service in `config/sysvinit.example`. This example assumes you're using Thin as an application server, so will need tweaking for Passenger or any other app server. You can install this by copying it to `/etc/init.d/alaveteli` and setting the `SITE_HOME` variable to the path where Alaveteli is running, and the `USER` variable to the Unix user that will be running Alaveteli. Once that's done, you can restart Alaveteli with `/etc/init.d/alaveteli restart`. ## Set up production web server It is not recommended to run the website using the default Rails web server. There are various recommendations here: http://rubyonrails.org/deploy We usually use Passenger / mod_rails. The file at `conf/httpd.conf-example` gives you an example config file for WhatDoTheyKnow. At a minimum, you should include the following in an Apache configuration file: PassengerResolveSymlinksInDocumentRoot on PassengerMaxPoolSize 6 # Recommend setting this to 3 or less on servers with 512MB RAM Under all but light loads, it is strongly recommended to run the server behind an http accelerator like Varnish. A sample varnish VCL is supplied in `conf/varnish-alaveteli.vcl`. It's strongly recommended that you run the site over SSL. (Set FORCE_SSL to true in config/general.yml). For this you will need an SSL certificate for your domain and you will need to configure an SSL terminator to sit in front of Varnish. If you're already using Apache as a web server you could simply use Apache as the SSL terminator. A minimal configuration would look something like this: ServerName www.yourdomain ProxyRequests Off ProxyPreserveHost On ProxyPass / http://localhost:80/ ProxyPassReverse / http://localhost:80/ RequestHeader set X-Forwarded-Proto 'https' SSLEngine on SSLProtocol all -SSLv2 SSLCipherSuite ALL:!ADH:!EXPORT:!SSLv2:RC4+RSA:+HIGH:+MEDIUM SSLCertificateFile /etc/apache2/ssl/ssl.crt SSLCertificateKeyFile /etc/apache2/ssl/ssl.key SSLCertificateChainFile /etc/apache2/ssl/sub.class2.server.ca.pem SSLCACertificateFile /etc/apache2/ssl/ca.pem SetEnvIf User-Agent ".*MSIE.*" nokeepalive ssl-unclean-shutdown Notice the line `RequestHeader` that sets the `X-Forwarded-Proto` header. This is important. This ultimately tells Rails that it's serving a page over https and so it knows to include that in any absolute urls it serves. We have some [production server best practice notes]({{ site.baseurl}}docs/running/server/). ## What next? Check out the [next steps]({{ site.baseurl }}docs/installing/next_steps/). ## Troubleshooting * **Run the Tests** Make sure everything looks OK: bundle exec rake spec If there are failures here, something has gone wrong with the preceding steps (see the next section for a common problem and workaround). You might be able to move on to the next step, depending on how serious they are, but ideally you should try to find out what's gone wrong. * **glibc bug workaround** There's a [bug in glibc](http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=637239) which causes Xapian to segfault when running the tests. Although the bug report linked to claims it's fixed in the current Debian stable, it's not as of version `2.11.3-2`. Until it's fixed (e.g. `libc6 2.13-26` does work), you can get the tests to pass by setting `export LD_PRELOAD=/lib/libuuid.so.1`. * **Incoming emails aren't appearing in my Alaveteli install** First, you need to check that your MTA is delivering relevant incoming emails to the `script/mailin` command. There are various ways of setting your MTA up to do this; we have documented one way of doing it [in Exim]({{ site.baseurl }}docs/installing/email/#example-setup-on-exim4), including [a command you can use]({{ site.baseurl }}docs/installing/email/#troubleshooting-exim) to check that the email routing is set up correctly. We've also documented one way of setting up [Postfix]({{ site.baseurl }}docs/installing/email/#example-setup-on-postfix), with a similar [debugging command]({{ site.baseurl }}docs/installing/email/#troubleshooting-postfix). Second, you need to test that the mailin script itself is working correctly, by running it from the command line, First, find a valid "To" address for a request in your system. You can do this through your site's admin interface, or from the command line, like so: $ ./script/console Loading development environment (Rails 2.3.14) >> InfoRequest.find_by_url_title("why_do_you_have_such_a_fancy_dog").incoming_email => "request-101-50929748@localhost" Now take the source of a valid email (there are some sample emails in `spec/fixtures/files/`); edit the `To:` header to match this address; and then pipe it through the mailin script. A non-zero exit code means there was a problem. For example: $ cp spec/fixtures/files/incoming-request-plain.email /tmp/ $ perl -pi -e 's/^To:.*/To: /' /tmp/incoming-request-plain.email $ ./script/mailin < /tmp/incoming-request-plain.email $ echo $? 75 The `mailin` script emails the details of any errors to `CONTACT_EMAIL` (from your `general.yml` file). A common problem is for the user that the MTA runs as not to have write access to `files/raw_emails/`. * **Various tests fail with "*Your PostgreSQL connection does not support unescape_bytea. Try upgrading to pg 0.9.0 or later.*"** You have an old version of `pg`, the ruby postgres driver. In Ubuntu, for example, this is provided by the package `libdbd-pg-ruby`. Try upgrading your system's `pg` installation, or installing the pg gem with `gem install pg` * **Some of the tests relating to mail are failing, with messages like "*when using TMail should load an email with funny MIME settings' FAILED*"** This sounds like the tests are running using the `production` environment, rather than the `test` environment, for some reason. * **Non-ASCII characters are being displayed as asterisks in my incoming messages** We rely on `elinks` to convert HTML email to plain text. Normally, the encoding should just work, but under some circumstances it appears that `elinks` ignores the parameters passed to it from Alaveteli. To force `elinks` always to treat input as UTF8, add the following to `/etc/elinks/elinks.conf`: set document.codepage.assume = "utf-8" set document.codepage.force_assumed = 1 You should also check that your locale is set up correctly. See [this issue followup](https://github.com/mysociety/alaveteli/issues/128#issuecomment-1814845) for further discussion. * **I'm seeing `rake: command not found` when running the post install script** The script uses `rake`. It may be that the binaries installed by bundler are not put in the system `PATH`; therefore, in order to run `rake` (needed for deployments), you may need to do something like: ln -s /usr/lib/ruby/gems/1.8/bin/rake /usr/local/bin/