diff options
author | Martin Wright <martin@mynameismartin.com> | 2015-04-07 14:46:11 +0100 |
---|---|---|
committer | Martin Wright <martin@mynameismartin.com> | 2015-04-07 14:46:11 +0100 |
commit | 270606788ba97b9c509183a06cdefe3b83f07464 (patch) | |
tree | 3304ea68b1e5cfecc2518aa6effa643175e12fdd /docs/installing | |
parent | 970396737e03ec2336f0ecc34d07047d83ff2629 (diff) | |
parent | 6cc59ede7bca866f7e56c771cc12ca1afa9406ef (diff) |
Merge branch 'gh-pages' of ssh://git.mysociety.org/data/git/public/alaveteli into gh-pages
Diffstat (limited to 'docs/installing')
-rw-r--r-- | docs/installing/ami.md | 174 | ||||
-rw-r--r-- | docs/installing/deploy.md | 61 | ||||
-rw-r--r-- | docs/installing/email.md | 605 | ||||
-rw-r--r-- | docs/installing/index.md | 24 | ||||
-rw-r--r-- | docs/installing/macos.md | 40 | ||||
-rw-r--r-- | docs/installing/manual_install.md | 924 | ||||
-rw-r--r-- | docs/installing/next_steps.md | 181 | ||||
-rw-r--r-- | docs/installing/script.md | 24 | ||||
-rw-r--r-- | docs/installing/vagrant.md | 95 |
9 files changed, 1661 insertions, 467 deletions
diff --git a/docs/installing/ami.md b/docs/installing/ami.md index 1f7195761..974c55d0c 100644 --- a/docs/installing/ami.md +++ b/docs/installing/ami.md @@ -1,59 +1,161 @@ --- layout: page -title: Installing the easy way +title: Installation from AMI --- # Installation on Amazon EC2 <p class="lead"> - We've made an Amazon Machine Image (AMI) so you can quickly deploy on Amazon EC2. This is handy if you just want to evaluate Alaveteli, for example. + We've made an Amazon Machine Image (AMI) so you can quickly deploy on Amazon + EC2. This is handy if you just want to evaluate Alaveteli, for example. </p> Note that there are [other ways to install Alaveteli]({{ site.baseurl }}docs/installing/). ## Installing from our AMI -To help people try out Alaveteli, we have created an AMI (Amazon Machine Image) -with a basic installation of Alaveteli, which you can use to create a running -server on an Amazon EC2 instance. This creates an instance that runs in -development mode, so we wouldn't recommend you use it for a production system -without changing the configuration. - -Unfortunately, Alaveteli will not run properly on a free Micro -instance due to the low amount of memory available on those -instances; you will need to use at least a Small instance, which -Amazon will charge for. - -The AMI can be found in the EU West (Ireland) region, with the ID ami-23519f54 -and name “Basic Alaveteli installation 2014-06-12”. You can launch an instance -based on that AMI with [this -link](https://console.aws.amazon.com/ec2/home?region=eu-west-1#launchAmi=ami-23519f54). - -When you create an EC2 instance based on that AMI, make sure that you choose -Security Groups that allows at least inbound HTTP, HTTPS, SSH and, if you want -to test incoming mail as well, SMTP. - -When your EC2 instance is launched, you will be able to log in as the `ubuntu` -user. This user can `sudo` freely to run commands as root. However, the code is -actually owned by (and runs as) the `alaveteli` user. After creating the -instance, you may want to edit a configuration file to customize the site's -configuration. That configuration file is -`/var/www/alaveteli/alaveteli/config/general.yml`, which can be edited with: +To help you try out Alaveteli, we have created an AMI with a basic installation +of Alaveteli, which you can use to create a running server on an Amazon EC2 +instance. This creates an instance that runs as a +<a href="{{ site.baseurl }}docs/glossary/#development" class="glossary__link">development site</a>. +If you want to use this for a +<a href="{{ site.baseurl }}docs/glossary/#production" class="glossary__link">production site</a>, +you must +[change the configuration]({{ site.baseurl }}docs/customising/config/#staging_site). + +<div class="attention-box"> + <p> + <strong>What's in the AMI?</strong> + The AMI gives you exactly the same thing as the + <a href="{{ site.baseurl}}docs/installing/script/">installation script</a> + does. You get an Alaveteli website powered by Rails running the Thin + application server under nginx, using a postgreSQL database. All this + running on Amazon's EC2 servers, ready to be + <a href="{{ site.baseurl }}docs/customising/">configured and customised</a>. + </p> +</div> + +Amazon instances are graded by size. Unfortunately, the *Micro* instance does +not have enough memory for Alaveteli to run -- and that's the only size +available on Amazon's free usage tier. You need to use a *Small* instance or +larger, which Amazon will charge you for. + +### Using Amazon web services + +To do this, you'll need: + + * an account with Amazon + * a SSL key pair (the Amazon web service screens guide you through this) + +If you don't have these already, you'll need to create them. See Amazon's +introduction on +[running a Virtual Server on AWS](http://docs.aws.amazon.com/gettingstarted/latest/awsgsg-intro/gsg-aws-virtual-server.html). + +### Launch the instance + +Once you're logged in to Amazon's service, and navigated to the **EC2 +Management Console**, you can launch the instance. If you prefer to do this +manually, you can find the AMI in the "EU West (Ireland)" region, with the ID +`ami-455ac032` and name “Basic Alaveteli installation 2014-12-05”. +Alternatively, use this link: + +<p class="action-buttons"> + <a href="https://console.aws.amazon.com/ec2/home?region=eu-west-1#launchAmi=ami-455ac032" class="button">launch + instance with Alaveteli installation AMI</a> +</p> + +When the instance launches, the first thing you need to choose is the instance +*type*. Remember that the *Micro* type does not have enough memory to run +Alaveteli, so you must choose at least *Small* or *Medium* -- note that these +are not available on Amazon's free usage tier. + +When the instance is created, the Amazon interface presents you with a lot of +choices about its configuration. You can generally accept the defaults for +everything, except the Security Groups. It's safe to click on **Review and +Launch** right away (rather than manually configuring all the instance details) +because you still get an opportunity to configure the security groups. Click on +**Edit Security Groups** on the summary page before you hit the big **Launch** +button. + +You must choose Security Groups that allow at least inbound HTTP, HTTPS, SSH +and, if you want to test incoming mail as well, SMTP. Amazon's settings here +let you specify the IP address(es) from which your instance will accept +requests. It's good practice to restrict these (if in doubt, choose a *Source* +of "My IP" for them all -- except incoming HTTP: for that, simpy to set +*Source* to "Anywhere"). You can change any of these settings later if you need +to. + +### Log into the server (shell) + +You need access to the server's command line shell to control and configure +your Alaveteli site. + +To access the server, use `ssh` and the `.pem` file from your SSL key pair. +Change the `.pem` file and instance ID to match your own in this command, which +connects to your server and logs you in as the user called `ubuntu`. Issue this +command from your own machine, to log in to the server: + + ssh -i path-to/your-key-pair.pem ubuntu@instance-id.eu-west-1.compute.amazonaws.com + +You won't be asked for a password, because the `.pem` file you supply with the +`-i` option contains the authorisation that matches the one at the other end, +on the server. You will be logged into the shell on your new Alaveteli server, +and can issue Unix commands to it. + +### Smoke test: start Alavateli + +You must configure your Alavateli site, but if you just want to see that you've +got your instance running OK, you *can* fire it up right away. Ideally, you +should skip this step and go straight to the configuration... but we know most +people like to see something in their browser first. ;-) + +On the command line shell, as the `ubuntu` user, start Alaveteli by doing: + + sudo service alaveteli start + +Find the "public DNS" URL of your EC2 instance from the AWS console, and look +at it in a browser. It will be of the form +`http://your-ec2-hostname.eu-west-1.compute.amazonaws.com`. You'll see your +Alaveteli site there. + +Your site isn't configured yet, so *this is insecure* (for example, you haven't +set your own passwords for access to the administration yet), so once you've +seen this running, bring the Alaveteli site down with: + + sudo service alaveteli stop + + +### Shell users: `ubuntu` and `alaveteli` + +When you log into your instance's command line shell, you must do so as the +`ubuntu` user. This user can `sudo` freely to run commands as root. However, +the code is actually owned by (and runs as) the `alaveteli` user. + +You will need to +[customise the site's configuration]({{ site.baseurl }}docs/customising/config/). +Do this by logging into your EC2 server and editing the `general.yml` +configuration file. + +The configuration file you need to edit is +`/var/www/alaveteli/alaveteli/config/general.yml`. For example, use the `nano` +editor (as the `alaveteli` user) like this: ubuntu@ip-10-58-191-98:~$ sudo su - alaveteli alaveteli@ip-10-58-191-98:~$ cd alaveteli alaveteli@ip-10-58-191-98:~/alaveteli$ nano config/general.yml -Then you should restart the Thin webserver with: +After making changes to that file, you'll need to start the application +server (use `restart` rather than `start` if it's already running): alaveteli@ip-10-58-191-98:~/alaveteli$ logout - ubuntu@ip-10-58-191-98:~$ sudo /etc/init.d/alaveteli restart + ubuntu@ip-10-58-191-98:~$ sudo service alaveteli start + +Your site will be running at the public URL again, which is of the form +`http://your-ec2-hostname.eu-west-1.compute.amazonaws.com`. + +If you have any problems or questions, please ask on the [Alaveteli developer mailing list](https://groups.google.com/forum/#!forum/alaveteli-dev) or [report an issue](https://github.com/mysociety/alaveteli/issues?state=open). -If you find the hostname of your EC2 instance from the AWS console, you should -then be able to see the site at -`http://your-ec2-hostname.eu-west-1.compute.amazonaws.com` -If you have any problems or questions, please ask on the [Alaveteli Google -Group](https://groups.google.com/forum/#!forum/alaveteli-dev) or [report an -issue](https://github.com/mysociety/alaveteli/issues?state=open). +##What next? +Check out the [next steps]({{ site.baseurl }}docs/installing/next_steps/). diff --git a/docs/installing/deploy.md b/docs/installing/deploy.md index 4bbc91a9d..74c7a8560 100644 --- a/docs/installing/deploy.md +++ b/docs/installing/deploy.md @@ -8,7 +8,8 @@ title: Deploying <p class="lead"> Although you can install Alaveteli and just change it when you need it, we recommend you adopt a way of <strong>deploying</strong> it automatically, - especially on your <a href="{{ site.baseurl }}docs/glossary/#production">production server</a>. + especially on your + <a href="{{ site.baseurl }}docs/glossary/#production" class="glossary__link">production server</a>. Alaveteli provides a deployment mechanism using Capistrano. </p> @@ -27,13 +28,13 @@ changes or copying files by hand, so your site will be down for the shortest possible time. We **strongly recommend** you use the deployment mechanism for your -<a href="{{ site.baseurl }}docs/glossary/#production">production server</a> and, if -you're running one, your -<a href="{{ site.baseurl }}docs/glossary/#staging">staging server</a> too. +<a href="{{ site.baseurl }}docs/glossary/#production" class="glossary__link">production server</a> +and, if you're running one, your +<a href="{{ site.baseurl }}docs/glossary/#staging" class="glossary__link">staging server</a> too. ## Capistrano -<a href="{{site.baseurl}}docs/glossary/#capistrano" class="glossary">Capistrano</a> +<a href="{{site.baseurl}}docs/glossary/#capistrano" class="glossary__link">Capistrano</a> is included as part of Alaveteli as a standard deployment system. The basic principle of Capistrano is that you execute `cap [do-something]` @@ -68,7 +69,11 @@ and thereafter you'll be able to deploy very easily (see [usage, below](#usage)) First, on the server: * [install Alaveteli]({{ site.baseurl }}docs/installing/) -* then move the Alaveteli app to a temporary place on the server, like your home +* give the Unix user that runs Alaveteli the ability to ssh to your server. Either give them a password or, preferably, set up ssh keys for them so they can ssh from your local machine to the server: + * to give them a password (if they don't already have one) - `sudo passwd [UNIX-USER]`. Store this password securely on your local machine e.g in a password manager + * to set up ssh keys for them, follow the instructions in the [capistrano documentation](http://capistranorb.com/documentation/getting-started/authentication-and-authorisation/). There's no need to set up ssh keys to the git repository as it is public. +* make sure the Unix user that runs Alaveteli has write permissions on the parent directory of your Alaveteli app +* move the Alaveteli app to a temporary place on the server, like your home directory (temporarily, your site will be missing, until the deployment puts new files in place) @@ -82,20 +87,28 @@ Next, on your local machine: need some of the files available locally even though you might not be running Alaveteli on this machine) * copy the example file `config/deploy.yml.example` to `config/deploy.yml` -* now customise the deployment settings in that file: edit `config/deploy.yml` - appropriately -- for example, edit the name of the server. Also, change - `deploy_to` to be the path where Alaveteli is currently installed on the - server -- if you used the installation script, this will be - `/var/www/alaveteli/alaveteli`. +* now customise the deployment settings in that file: edit + `config/deploy.yml` appropriately -- for example, edit the name of the + server. Also, change `deploy_to` to be the path where Alaveteli is + currently installed on the server -- if you used the installation + script , this will be `/var/www/[HOST or alaveteli]/alaveteli`. Set + `daemon_name` to the name you used in setting up the [application + daemon]({{ site.baseurl }}docs/installing/manual_install/#generate-application-daemon). The + default should be `alaveteli`. * `cd` into the Alaveteli repo you checked out (otherwise the `cap` commands you're about to execute won't work) * still on your local machine, run `cap -S stage=staging deploy:setup` to setup capistrano on the server -* again on your local machine, run `cap -S stage=staging deploy:update_code` to get a code checkout on the server + +If you get an error `SSH::AuthenticationFailed`, and are not prompted for the password of the deployment user, you may have run into [a bug](http://stackoverflow.com/questions/21560297/capistrano-sshauthenticationfailed-not-prompting-for-password) in the net-ssh gem version 2.8.0. Try installing version 2.7.0 instead: + + gem uninstall net-ssh + + gem install net-ssh -v 2.7.0 Back on the server: * copy the following config files from the temporary copy of Alaveteli you made at - the begining (perhaps in your home directory) to the `shared` directory that + the beginning (perhaps in your home directory) to the `shared` directory that Capistrano just created on the server: * `general.yml` * `database.yml` @@ -110,17 +123,23 @@ Back on the server: `shared` directory created by Capistrano on the server: * `cache/` * `files/` + * `lib/acts_as_xapian/xapiandbs` (copy this to straight into `shared` so it becomes `shared/xapiandbs`) + * `log/` Now, back on your local machine: * make sure you're still in the Alaveteli repo (if not, `cd` back into it) +* run `cap -S stage=staging deploy:update_code` to get a code checkout on the server. * create a deployment directory on the server by running *one* of these commands: - * `cap deploy` if you're deploying a <a href="{{site.baseurl}}docs/glossary/#staging" class="glossary">staging site</a>, or... - * `cap -S stage=production deploy` for <a href="{{site.baseurl}}docs/glossary/#production" class="glossary">production</a> + * `cap deploy` if you're deploying a <a href="{{site.baseurl}}docs/glossary/#staging" class="glossary__link">staging site</a>, or... + * `cap -S stage=production deploy` for <a href="{{site.baseurl}}docs/glossary/#production" class="glossary__link">production</a> + +Back on the server: + * update the webserver config (either apache or nginx) to add the `current` element to the path where it is serving Alaveteli from. If you installed using the installation script, this will be replacing `/var/www/alaveteli/alaveteli/` with - `/var/www/alaveteli/alaveteli/current` in `etc/nginx/sites-available/default`. + `/var/www/alaveteli/alaveteli/current` in `/etc/nginx/sites-available/default`. * edit the server crontab so that the paths in the cron jobs also include the `current` element. If you used the installation script the crontab will be in `etc/cron.d/alaveteli`. @@ -130,7 +149,13 @@ Now, back on your local machine: `argv=/var/www/alaveteli/alaveteli/script/mailin` with `argv=/var/www/alaveteli/alaveteli/current/script/mailin`. If you're using Exim as your MTA, edit `etc/exim4/conf.d/04_alaveteli_options` - to update the `ALAVETELI_HOME` variable to the new Alaveteli path. + to update the `ALAVETELI_HOME` variable to the new Alaveteli path. Restart the MTA after you've made these changes. + +* You will also need to update the path to Alaveteli in your [init scripts]({{site.baseurl}}docs/installing/manual_install/#cron-jobs-and-init-scripts). + You should have a script for running the alert tracks + (`/etc/init.d/foi-alert-tracks`), and possibly scripts for purging the + varnish cache (`/etc/init.d/foi-purge-varnish`), and restarting the + app server (`/etc/init.d/alaveteli`). Phew, you're done! @@ -146,7 +171,7 @@ for the config that you've set up). Ensure you've got a `config/deploy.yml` file with the correct settings for your site. If there are other people in your team who need to deploy, you'll need to share it with them too -- it might be a good idea to keep the latest -version in a [Gist](http://gist.github.com/). +version in a private [Gist](http://gist.github.com/). * to deploy to staging, just run `cap deploy` * to deploy to production, run `cap -S stage=production deploy` diff --git a/docs/installing/email.md b/docs/installing/email.md index 98dff0087..44476cefa 100644 --- a/docs/installing/email.md +++ b/docs/installing/email.md @@ -11,6 +11,77 @@ title: Installing MTA here for both postfix and exim4, two of the most popular MTAs. </p> +## How Alaveteli handles email + +### Request mail + +When someone makes a Freedom of Information request to an authority through +Alaveteli, the application sends an email containing the request to the authority. + +The email's `reply-to` address is a special one so that any replies to it +can be automatically directed back to Alaveteli, and so that Alaveteli +can tell which request the reply needs to be shown with. This requires +some configuration of the MTA on the server that is running Alaveteli, +so that it will pipe all emails to these special addresses to Alaveteli +to handle, via its `script/mailin` script. The special addresses are of +the form: + + <foi+request-3-691c8388@example.com> + +Parts of this address are controlled with options in +`config/general.yml`: + + INCOMING_EMAIL_PREFIX = 'foi+' + INCOMING_EMAIL_DOMAIN = 'example.com' + +If there is some error inside Rails while processing an email, an exit code `75` is returned to the MTA by the `script/mailin` script. Postfix and Exim (and maybe others) take this as a signal for the MTA to try again later. Additionally, a stacktrace is emailed to `CONTACT_EMAIL`. + +[Production]({{ site.baseurl }}/docs/glossary/#production) installs of Alaveteli should make a backup copy of emails sent to the special addresses. You can configure your chosen MTA to backup these in a separate mailbox. + +### Transactional mail + +Alaveteli also sends emails to users about their requests – letting them know when someone has replied to them, or prompting them to take further action. + +Configure the address that these messages are sent from in the [`CONTACT_EMAIL`]({{site.baseurl}}docs/customising/config/#contact_email) option in `config/general.yml`: + + CONTACT_EMAIL = 'team@example.com' + +The address in [`CONTACT_EMAIL`]({{ site.baseurl }}docs/customising/config/#contact_email) is also visible in various places on the site so that users can get in touch with the team that runs the site. + +You must configure your MTA to deliver mail sent to these addresses to the administrators of your site so that they can respond to it. + +### Tracks mail + +Users subscribed to updates from the site – known as `tracks` – receive emails when there is something new of interest to them on the site. + +Configure the address that these messages are sent from in the [`TRACK_SENDER_EMAIL`]({{site.baseurl}}docs/customising/config/#track_sender_email) option in `config/general.yml`: + + TRACK_SENDER_EMAIL = 'track@example.com' + +### Automatic bounce handling (optional) + +As [`CONTACT_EMAIL`]({{ site.baseurl }}docs/customising/config/#contact_email) and [`TRACK_SENDER_EMAIL`]({{site.baseurl}}docs/customising/config/#track_sender_email) appear in the `From:` header of emails sent from Alaveteli, they sometimes receive reply emails, including <a href="{{ site.baseurl }}docs/glossary/#bounce-message">bounce messages</a> and ‘out of office’ notifications. + +Alaveteli provides a script (`script/handle-mail-replies`) that handles bounce messages and ‘out of office’ notifications and forwards genuine mails to your administrators. + +It also prevents further track emails being sent to a user email address that appears to have a permanent delivery problem. + +To make use of automatic bounce-message handling, set [`TRACK_SENDER_EMAIL`]({{ site.baseurl }}docs/customising/config/#track_sender_email) and [`CONTACT_EMAIL`]({{ site.baseurl }}docs/customising/config/#contact_email) to an address that you will filter through `script/handle-mail-replies`. Messages that are not bounces or out-of-office autoreplies will be forwarded to [`FORWARD_NONBOUNCE_RESPONSES_TO`]({{ site.baseurl }}docs/customising/config/#forward_nonbounce_responses_to), which you should set to a mail alias that points at your list of site administrators. + +See the MTA-specific instructions for how to do this for [exim]({{ site.baseurl }}docs/installing/email#filter-incoming-messages-to-admin-addresses) and [postfix]({{ site.baseurl }}docs/installing/email#filter-incoming-messages-to-site-admin-addresses). + +_Note:_ Bounce handling is not applied to [request emails]({{ site.baseurl }}docs/installing/email#request-mail). Bounce messages from authorities get added to the request page so that the user can see what has happened. Users can ask site admins for help redelivering the request if necessary. + + +--- + +<div class="attention-box"> + <ul> + <li>Commands in this guide will require root privileges</li> + <li>Commands are intended to be run via the terminal or over ssh</li> + </ul> +</div> + Make sure you follow the correct instructions for the specific MTA you're using: * [postfix](#example-setup-on-postfix) @@ -19,54 +90,199 @@ Make sure you follow the correct instructions for the specific MTA you're using: ## Example setup on postfix This section shows an example of how to set up your MTA if you're using -**postfix** (running on Ubuntu). See the example for +**postfix**. See the example for [exim4](#example-setup-on-exim4) if you're using that instead of postfix. -### Instructions +### Install postfix + + # Install debconf so we can configure non-interactively + apt-get -qq install -y debconf >/dev/null + + # Set the default configuration 'Internet Site' + echo postfix postfix/main_mailer_type select 'Internet Site' | debconf-set-selections -For example, with: + # Set your hostname (change example.com to your hostname) + echo postfix postfix/mail_name string "example.com" | debconf-set-selections - ALAVETELI_HOME=/path/to/alaveteli/software - ALAVETELI_USER=www-data + # Install postfix + DEBIAN_FRONTEND=noninteractive apt-get -qq -y install postfix >/dev/null -In `/etc/postfix/master.cf`: +### Configure postfix + +#### Pipe incoming mail for requests into Alaveteli + +If the Unix user that is going to +run your site is `alaveteli`, and the directory where Alaveteli is installed is +`/var/www/alaveteli`, create the pipe that will receive request mail: + + cat >> /etc/postfix/master.cf <<EOF alaveteli unix - n n - 50 pipe - flags=R user=ALAVETELI_USER argv=ALAVETELI_HOME/script/mailin + flags=R user=alaveteli argv=/var/www/alaveteli/script/mailin + EOF + +The Unix user should have write permissions on the directory where Alaveteli is installed. -The user ALAVETELI_USER should have write permissions on ALAVETELI_HOME. +Configure postfix to accept messages for local delivery where +recipients are: -In `/etc/postfix/main.cf`: + - defined by a regular expression in `/etc/postfix/transports` + - local UNIX accounts + - local aliases specified as regular expressions in `/etc/postfix/recipients` - virtual_alias_maps = regexp:/etc/postfix/regexp +<!-- Comment to enable markdown to render code fence under list --> -And, assuming you set -[`INCOMING_EMAIL_PREFIX`]({{ site.baseurl }}docs/customising/config/#incoming_email_prefix) -in `config/general` to "foi+", create `/etc/postfix/regexp` with the following -content: + cat >> /etc/postfix/main.cf <<EOF + transport_maps = regexp:/etc/postfix/transports + local_recipient_maps = proxy:unix:passwd.byname regexp:/etc/postfix/recipients + EOF - /^foi.*/ alaveteli +In `/etc/postfix/main.cf` update the `mydestination` line (which determines what domains this machine will deliver locally). Add your domain, not `example.com`, to the beginning of the list: -You should also configure postfix to discard any messages sent to the -[`BLACKHOLE_PREFIX`]({{ site.baseurl }}docs/customising/config/#blackhole_prefix) -address, whose default value is `do-not-reply-to-this-address`. For example, add the -following to `/etc/aliases`: + mydestination = example.com, localhost.localdomain, localhost +<div class="attention-box"> +This guide assumes you have set <a href="{{ site.baseurl }}docs/customising/config/#incoming_email_prefix"><code>INCOMING_EMAIL_PREFIX</code></a> to <code>foi+</code> in <code>config/general.yml</code> +</div> + +Pipe all incoming mail where the `To:` address starts with `foi+` to the `alaveteli` pipe (`/var/www/alaveteli/script/mailin`, as specified in `/etc/postfix/master.cf` at the start of this section): + + cat > /etc/postfix/transports <<EOF + /^foi.*/ alaveteli + EOF + +#### Backup request mail + +You can copy all incoming mail to Alaveteli to a backup account to a separate mailbox, just in case. + +Create a UNIX user `backupfoi` + + adduser --quiet --disabled-password \ + --gecos "Alaveteli Mail Backup" backupfoi + +Add the following line to `/etc/postfix/main.cf` + + recipient_bcc_maps = regexp:/etc/postfix/recipient_bcc + +Configure mail sent to an `foi+` prefixed address to be sent to the backup user: + + cat > /etc/postfix/recipient_bcc <<EOF + /^foi.*/ backupfoi + EOF + + +#### Define the valid recipients for your domain + +Create `/etc/postfix/recipients` with the following command: + + cat > /etc/postfix/recipients <<EOF + /^foi.*/ this-is-ignored + /^postmaster@/ this-is-ignored + /^user-support@/ this-is-ignored + /^team@/ this-is-ignored + EOF + +The left-hand column of this file specifies regular expressions that +define addresses that mail will be accepted for. The values on the +right-hand side are ignored by postfix. Here we allow postfix to accept +mails to special Alaveteli addresses, and `postmaster@example.com`, +`user-support@example.com` and `team@example.com`. + +The `@example.com` domain is set in the `mydestination` as above. This should be set to your actual domain. + +#### Set up contact email recipient groups + +To set up recipient groups for the `postmaster@`, `team@` and `user-support@` email addresses at your domain, add alias records for them in `/etc/aliases`: + + cat >> /etc/aliases <<EOF + team: user@example.com, otheruser@example.com + user-support: team + EOF + +#### Discard unwanted incoming email + +Configure postfix to discard any messages sent to the [`BLACKHOLE_PREFIX`]({{ site.baseurl }}docs/customising/config/#blackhole_prefix) address, whose default value is `do-not-reply-to-this-address`: + + cat >> /etc/aliases <<EOF # We use this for envelope from for some messages where # we don't care about delivery - do-not-reply-to-this-address: :blackhole: + do-not-reply-to-this-address: /dev/null + EOF + +If you have set [`BLACKHOLE_PREFIX`]({{ site.baseurl }}docs/customising/config/#blackhole_prefix) address, replace `do-not-reply-to-this-address` with the address you have configured. + +#### Filter incoming messages to site admin addresses + +You can make use of Alaveteli's [automatic bounce handling]({{site.baseurl}}docs/installing/email/#automatic-bounce-handling-optional) to filter bounces sent to [`TRACK_SENDER_EMAIL`]({{site.baseurl}}docs/customising/config/#track_sender_email) +and [`CONTACT_EMAIL`]({{site.baseurl}}docs/customising/config/#contact_email). + + +<div class="attention-box"> +This guide assumes you have set the following in <code>config/general.yml</code>: + + <ul> + <li><a href="{{site.baseurl}}docs/customising/config/#contact_email">CONTACT_EMAIL</a>: <code>user-support@example.com</code></li> + <li><a href="{{site.baseurl}}docs/customising/config/#track_sender_email">TRACK_SENDER_EMAIL</a>: <code>user-support@example.com</code></li> + <li><a href="{{site.baseurl}}docs/customising/config/#forward_nonbounce_responses_to">FORWARD_NONBOUNCE_RESPONSES_TO</a>: <code>team@example.com</code></li> + </ul> + +Change the examples below to the addresses you have configured. +</div> + +Create a new pipe to handle replies: + + cat >> /etc/postfix/master.cf <<EOF + alaveteli_replies unix - n n - 50 pipe + flags=R user=alaveteli argv=/var/www/alaveteli/script/handle-mail-replies + EOF + +_Note:_ Replace `/var/www/alaveteli` with the correct path to alaveteli if required. + +Pipe mail sent to `user-support@example.com` to the `alaveteli_replies` pipe: + + cat >> /etc/postfix/transports <<EOF + /^user-support@*/ alaveteli_replies + EOF -### Logging +Finally, edit `/etc/aliases` to remove `user-support`: -For the postfix logs to be succesfully read by the script `load-mail-server-logs`, they need -to be log rotated with a date in the filename. Since that will create a lot of rotated log -files (one for each day), it's good to have them in their own directory. For example (on Ubuntu), -in `/etc/rsyslog.d/50-default.conf` set: + team: user@example.com, otheruser@example.com + +#### Logging + +For the postfix logs to be successfully read by +`script/load-mail-server-logs`, they need to be log rotated with a date in the +filename. Since that will create a lot of rotated log files (one for +each day), it's good to have them in their own directory. + +You'll also need to tell Alaveteli where the log files are stored and that they're in postfix +format. Update +[`MTA_LOG_PATH`]({{ site.baseurl }}docs/customising/config/#mta_log_path) and +[`MTA_LOG_TYPE`]({{ site.baseurl }}docs/customising/config/#mta_log_type) in `config/general.yml`: + + MTA_LOG_PATH: '/var/log/mail/mail.log-*' + MTA_LOG_TYPE: "postfix" + +Configure postfix to log to its own directory: + +##### Debian + +In `/etc/rsyslog.conf`, set: mail.* -/var/log/mail/mail.log -And also edit `/etc/logrotate.d/rsyslog`: +##### Ubuntu + +In `/etc/rsyslog.d/50-default.conf` set: + + mail.* -/var/log/mail/mail.log + +##### Configure logrotate + +Configure logrotate to rotate the log files in the required format: + + cat >> /etc/logrotate.d/rsyslog <<EOF /var/log/mail/mail.log { rotate 30 @@ -81,65 +297,136 @@ And also edit `/etc/logrotate.d/rsyslog`: reload rsyslog >/dev/null 2>&1 || true endscript } + EOF -You'll also need to tell Alaveteli where the log files are stored and that they're in postfix -format. Update -[`MTA_LOG_PATH`]({{ site.baseurl }}docs/customising/config/#mta_log_path) and -[`MTA_LOG_TYPE`]({{ site.baseurl }}docs/customising/config/#mta_log_type) in `config/general.yml` with: +#### Making the changes live - MTA_LOG_PATH: '/var/log/mail/mail.log-*' - MTA_LOG_TYPE: "postfix" +As the root user, make all these changes live with the following commands: -### Troubleshooting (postfix) + service rsyslog restart + + newaliases + postmap /etc/postfix/transports + postmap /etc/postfix/recipients + postmap /etc/postfix/recipient_bcc + postfix reload + +#### Troubleshooting (postfix) To test mail delivery, run: - $ /usr/sbin/sendmail -bv foi+requrest-1234@localhost + $ /usr/sbin/sendmail -bv foi+request-1234@example.com + +Make sure to replace `example.com` with your domain. This command tells +you if sending the emails to `foi\+.*example.com` and the backup account +is working (it doesn't actually send any mail). If it is working, you +should receive a delivery report email, with text like: + + <foi+request-1234@example.com>: delivery via alaveteli: + delivers to command: /var/www/alaveteli/script/mailin + <backupfoi@local.machine.name>: delivery via local: delivers to mailbox + +You can also test the other aliases you have set up for your domain in +this section to check that they will deliver mail as you expect. For +example, you can test bounce message routing in the same way - the text +of this delivery report mail should read something like: + + <user-support@example.com>: delivery via alaveteli_replies: delivers to command: /var/www/alaveteli/script/handle-mail-replies + + +Note that you may need to install the `mailutils` package to read the +delivery report email using the `mail` command on a new server: + + apt-get install mailutils + +If emails are not being received by your Alaveteli install, we have some +more troubleshooting tips for incoming mail in [general email troubleshooting]({{ site.baseurl }}docs/installing/email#general-email-troubleshooting). -This tells you if sending the emails to `foi\+.*localhost` is working. ## Example setup on exim4 This section shows an example of how to set up your MTA if you're using -**exim4** (running on Ubuntu). See the example for +**exim4**. See the example for [postfix](#example-setup-on-postfix) if you're using that instead of exim4. -### Instructions +### Install exim4 + +Install exim4: + + apt-get install exim4 -We suggest you add the following to your exim configuration. -In `/etc/exim4/conf.d/main/04_alaveteli_options`, set: +### Configure exim4 - ALAVETELI_HOME=/path/to/alaveteli/software - ALAVETELI_USER=www-data +#### Set up exim to receive mail from other servers + +Edit `/etc/exim4/update-exim4.conf.conf`. Set the following settings (use your hostname, not `example.com`): + + dc_eximconfig_configtype='internet' + dc_other_hostnames='example.com' + dc_local_interfaces='0.0.0.0 ; ::1' + dc_use_split_config='true' + +This final line tells exim to use the files in `/etc/exim4/conf.d` to configure itself. + +#### Define general variables and logging settings + +Create `/etc/exim4/conf.d/main/04_alaveteli_options` with the command: + + cat > /etc/exim4/conf.d/main/04_alaveteli_options <<'EOF' + ALAVETELI_HOME=/var/www/alaveteli + ALAVETELI_USER=alaveteli log_file_path=/var/log/exim4/exim-%slog-%D MAIN_LOG_SELECTOR==+all -retry_defer extract_addresses_remove_arguments=false + EOF + +This sets up `ALAVETELI_HOME` and `ALAVETELI_USER` for use in other config files, and sets up logging. + +- **`ALAVETELI_HOME`:** set to the directory where Alaveteli is installed. +- **`ALAVETELI_USER`:** should be the Unix user that is going to run your site. They should have write permissions on `ALAVETELI_HOME`. +- **`log_file_path`:** The name and location of the log files created by Exim must match what the `load-mail-server-logs` script expects +- **`MAIN_LOG_SELECTOR`:** The `check-recent-requests-sent` scripts expects the logs to contain the `from=<...>` envelope information, so we make the logs more verbose +- **`extract_addresses_remove_arguments`:** setting to `false` gets exim to treat the `-t` command line option that the `mail` gem uses when specifying delivery addresses on the command line as specifying that the addresses should be added, not removed. See [this `mail` issue](https://github.com/mikel/mail/issues/70) for more details. + +<div class="attention-box"> +Note: If you are editing an existing exim config rather than creating a new one, check the <code>untrusted_set_sender</code> option in <code>/etc/exim4/conf.d/main/02_exim4-config_options</code>. By default, untrusted users in exim are only allowed to set an empty envelope sender address, to declare that a message should never generate any bounces. <code>untrusted_set_sender</code> can be set to a list of address patterns, meaning that untrusted users are allowed to set envelope sender addresses that match any of the patterns in the list. If a pattern list is specified, you will need also to add <code>ALAVETELI_USER</code> to the <code>MAIN_TRUSTED_USERS</code> list in order to allow them to set the return path on outgoing mail. This option is also in <code>/etc/exim4/conf.d/main/02_exim4-config_options</code> in a split config. Look for the line that begins with <code>MAIN_TRUSTED_USERS</code> - something like: -The user ALAVETELI_USER should have write permissions on ALAVETELI_HOME. + <pre><code>MAIN_TRUSTED_USERS = uucp</code></pre> -The name and location of the log files created by Exim must match what the -`load-mail-server-logs` script expects, which is why you must provide the -`log_file_path` setting. +and add the alaveteli user: -The `check-recent-requests-sent` scripts expects the logs to contain the -`from=<...>` envelope information, so we make the logs more verbose with -`log_selector`. The ALAVETELI_USER may need to also need to be added to the -`trusted_users` list in your Exim config in order to set the return path on -outgoing mail, depending on your setup. + <pre><code>MAIN_TRUSTED_USERS = uucp : alaveteli</code></pre> -In `/etc/exim4/conf.d/router/04_alaveteli`: + If <code>untrusted_set_sender</code> is set to <code>*</code>, that means that untrusted users can set envelope sender addresses without restriction, so there's no need to add <code>ALAVETELI_USER</code> to the <code>MAIN_TRUSTED_USERS</code> list. +</div> +#### Pipe incoming mail for requests from Exim to Alaveteli + +In this section, we'll add config to pipe incoming mail for special +Alaveteli addresses into Alaveteli, and also send them to a local backup +mailbox. + +Create the `backupfoi` UNIX user + + adduser --quiet --disabled-password \ + --gecos "Alaveteli Mail Backup" backupfoi + +Specify an exim `router` for special Alaveteli addresses, which will route messages into Alaveteli using a local pipe transport: + + cat > /etc/exim4/conf.d/router/04_alaveteli <<'EOF' alaveteli_request: debug_print = "R: alaveteli for $local_part@$domain" driver = redirect data = ${lookup{$local_part}wildlsearch{ALAVETELI_HOME/config/aliases}} pipe_transport = alaveteli_mailin_transport + EOF -In `/etc/exim4/conf.d/transport/04_alaveteli`: +Create `/etc/exim4/conf.d/transport/04_alaveteli`, which sets the properties of the pipe `transport` that will deliver mail to Alaveteli: + cat > /etc/exim4/conf.d/transport/04_alaveteli <<'EOF' alaveteli_mailin_transport: driver = pipe command = $address_pipe ${lc:$local_part} @@ -147,67 +434,197 @@ In `/etc/exim4/conf.d/transport/04_alaveteli`: home_directory = ALAVETELI_HOME user = ALAVETELI_USER group = ALAVETELI_USER + EOF + + +<div class="attention-box"> + This guide assumes you have set <a href="/docs/customising/config/#incoming_email_prefix"><code>INCOMING_EMAIL_PREFIX</code></a> to <code>foi+</code> in <code>config/general.yml</code> +</div> + +Create the `config/aliases` file that the `alaveteli_request` exim `router` sources. This pipes mail from the special address to `script/mailin` and the `backupfoi` user. + + cat > /var/www/alaveteli/config/aliases <<'EOF' + ^foi\\+.*: "|/var/www/alaveteli/script/mailin", backupfoi + EOF -And, assuming you set -[`INCOMING_EMAIL_PREFIX`]({{ site.baseurl }}docs/customising/config/#incoming_email_prefix) -in your config at `config/general.yml` to "foi+", create `config/aliases` with the following -content: +_Note:_ Replace `/var/www/alaveteli` with the correct path to alaveteli if required. - ^foi\\+.*: |/path/to/alaveteli/software/script/mailin +#### Set up your contact email recipient groups -You should also configure exim to discard any messages sent to the -[`BLACKHOLE_PREFIX`]({{ site.baseurl }}docs/customising/config/#blackhole_prefix) -address, whose default value is -`do-not-reply-to-this-address`. For example, add the following to -`config/aliases`: +To set up recipient groups for the `team@` and `user-support@` email addresses at your domain, add alias records for them in `/var/www/alaveteli/config/aliases` - # We use this for envelope from for some messages where we don't care about delivery + cat >> /var/www/alaveteli/config/aliases <<EOF + team: user@example.com, otheruser@example.com + user-support: team + EOF + +#### Discard unwanted incoming email + +Configure exim to discard any messages sent to the [`BLACKHOLE_PREFIX`]({{ site.baseurl }}docs/customising/config/#blackhole_prefix) address, whose default value is `do-not-reply-to-this-address` + + cat >> /var/www/alaveteli/config/aliases <<EOF + # We use this for envelope from for some messages where + # we don't care about delivery do-not-reply-to-this-address: :blackhole: + EOF + +_Note:_ Replace `/var/www/alaveteli` with the correct path to alaveteli if required. + +#### Filter incoming messages to admin addresses + +You can make use of Alaveteli's [automatic bounce handling]({{site.baseurl}}docs/installing/email/#automatic-bounce-handling-optional) to filter bounces sent to [`TRACK_SENDER_EMAIL`]({{site.baseurl}}docs/customising/config/#track_sender_email) +and [`CONTACT_EMAIL`]({{site.baseurl}}docs/customising/config/#contact_email). + +<div class="attention-box"> +This guide assumes you have set the following in <code>config/general.yml</code>: -If you want to make use of the automatic bounce-message handling, then set the -[`TRACK_SENDER_EMAIL`]({{ site.baseurl }}docs/customising/config/#track_sender_email) -address to be filtered through -`script/handle-mail-replies`. Messages that are not bounces or -out-of-office autoreplies will be forwarded to -[`FORWARD_NONBOUNCE_RESPONSES_TO`]({{ site.baseurl }}docs/customising/config/#forward_nonbounce_responses_to). -For example, in WhatDoTheyKnow the -configuration looks like this: + <ul> + <li><a href="{{site.baseurl}}docs/customising/config/#contact_email">CONTACT_EMAIL</a>: <code>user-support@example.com</code></li> + <li><a href="{{site.baseurl}}docs/customising/config/#track_sender_email">TRACK_SENDER_EMAIL</a>: <code>user-support@example.com</code></li> + <li><a href="{{site.baseurl}}docs/customising/config/#forward_nonbounce_responses_to">FORWARD_NONBOUNCE_RESPONSES_TO</a>: <code>team@example.com</code></li> + </ul> - raw_team: [a list of people on the team] - team: |/path/to/alaveteli/software/script/handle-mail-replies +Change the examples below to the addresses you have configured. +</div> -with `FORWARD_NONBOUNCE_RESPONSES_TO`: 'raw_team@whatdotheyknow.com'` +Change the `user-support` line in `/var/www/alaveteli/config/aliases`: -Finally, make sure you have `dc_use_split_config='true'` in -`/etc/exim4/update-exim4.conf.conf`, and execute the command -`update-exim4.conf`. + user-support: |/var/www/alaveteli/script/handle-mail-replies + +#### Logging + +You’ll need to tell Alaveteli where the log files are stored and that they’re in exim format. Update [`MTA_LOG_PATH`]({{ site.baseurl }}docs/customising/config/#mta_log_path) and [`MTA_LOG_TYPE`]({{ site.baseurl }}docs/customising/config/#mta_log_type) in `config/general.yml`: + + MTA_LOG_PATH: '/var/log/exim4/exim-mainlog-*' + MTA_LOG_TYPE: 'exim' + + +#### Making the changes live in exim + +Finally, execute the commands: + + update-exim4.conf + service exim4 restart Note that if the file `/etc/exim4/exim4.conf` exists then `update-exim4.conf` will silently do nothing. Some distributions include this file. If -yours does, you will need to rename it before running `update-exim4.conf`. +yours does, you will need to remove or rename it before running `update-exim4.conf`. -(You may also want to set `dc_eximconfig_configtype='internet'`, -`dc_local_interfaces='0.0.0.0 ; ::1'`, and -`dc_other_hostnames='<your-host-name>'`). -### Troubleshooting (exim) +#### Troubleshooting (exim) -To test mail delivery, run: +To test mail delivery, as a privileged user run: - exim -bt foi+request-1234@localhost + exim4 -bt foi+request-1234@example.com -This should tell you which routers are being processed. You should +replacing `example.com` with your domain name. This should tell you which routers are being processed. You should see something like: - $ exim -bt foi+request-1234@localhost - R: alaveteli pipe for snafflerequest-234@localhost - snafflerequest-234@localhost -> |/home/alaveteli/alaveteli/script/mailin - transport = alaveteli_mailin_transport + $ exim4 -bt foi+request-1234@example.com + R: alaveteli for foi+request-1234@example.com + foi+request-1234@example.com -> |/var/www/alaveteli/script/mailin + transport = alaveteli_mailin_transport + R: alaveteli for backupfoi@your.machine.name + R: system_aliases for backupfoi@your.machine.name + R: userforward for backupfoi@your.machine.name + R: procmail for backupfoi@your.machine.name + R: maildrop for backupfoi@your.machine.name + R: lowuid_aliases for backupfoi@your.machine.name (UID 1001) + R: local_user for backupfoi@your.machine.name + backupfoi@your.machine.name + <-- foi+request-1234@example.com + router = local_user, transport = mail_spool This tells you that the routing part (making emails to -`foi\+.*@localhost` be forwarded to Alaveteli's `mailin` script) is -working. +`foi\+.*@example.com` be forwarded to Alaveteli's `mailin` script, and +also sent to the local backup account) is working. You can test bounce +message routing in the same way: + + exim4 -bt user-support@example.com + R: alaveteli for user-support@example.com + user-support@example.com -> |/var/www/alaveteli/script/handle-mail-replies + transport = alaveteli_mailin_transport + +If emails are not being received by your Alaveteli install, we have some +more troubleshooting tips for incoming mail in the next section. There is also a +great [Exim +Cheatsheet](http://bradthemad.org/tech/notes/exim_cheatsheet.php) online +that you may find useful. + +## General Email Troubleshooting + +First, you need to check that your MTA is delivering relevant +incoming emails to the `script/mailin` command. There are various +ways of setting your MTA up to do this; we have documented +one way of doing it +[in Exim]({{ site.baseurl }}docs/installing/email/#example-setup-on-exim4), including [a command you can use]({{ site.baseurl }}docs/installing/email/#troubleshooting-exim) to check that the email +routing is set up correctly. We've also documented one way of setting up [Postfix]({{ site.baseurl }}docs/installing/email/#example-setup-on-postfix), with a similar [debugging command]({{ site.baseurl }}docs/installing/email/#troubleshooting-postfix). + +Second, you need to test that the mailin script itself is working +correctly, by running it from the command line, First, find a +valid "To" address for a request in your system. You can do this +through your site's admin interface, or from the command line, +like so: + + $ ./script/console + Loading development environment (Rails 2.3.14) + >> InfoRequest.find_by_url_title("why_do_you_have_such_a_fancy_dog").incoming_email + => "request-101-50929748@localhost" + +Now take the source of a valid email (there are some sample emails in +`spec/fixtures/files/`); edit the `To:` header to match this address; +and then pipe it through the mailin script. A non-zero exit code +means there was a problem. For example: + + $ cp spec/fixtures/files/incoming-request-plain.email /tmp/ + $ perl -pi -e 's/^To:.*/To: <request-101-50929748@localhost>/' /tmp/incoming-request-plain.email + $ ./script/mailin < /tmp/incoming-request-plain.email + $ echo $? + 75 + +The `mailin` script emails the details of any errors to +`CONTACT_EMAIL` (from your `general.yml` file). A common problem is +for the user that the MTA runs as not to have write access to +`files/raw_emails/`. + +If everything seems fine locally, you should also check from another +computer connected to the Internet that the DNS for your chosen +domain indicates that your Alaveteli server is handling mail, and +that your server is receiving mail on port 25. The following +command is a query to ask which server is handling the mail for +the domain `example.com`, which receives the answer `mail.example.com`. + + $ host -t mx example.com + example.com mail is handled by 5 mail.example.com. + +This next command tries to connect to port 25, the standard SMTP +port, on `mail.example.com`, and is refused. + + $ telnet mail.example.com 25 + Trying 10.10.10.30... + telnet: connect to address 10.10.10.30: Connection refused + +The transcript below shows a successful connection where the server +accepts mail for delivery (the commands you would type are prefixed +by a `$`): + + $ telnet 10.10.10.30 25 + Trying 10.10.10.30... + Connected to 10.10.10.30. + Escape character is '^]'. + 220 mail.example.com ESMTP Exim 4.80 Tue, 12 Aug 2014 11:10:39 +0000 + $ HELO X + 250 mail.example.com Hello X [10.10.10.1] + $ MAIL FROM: <test@local.domain> + 250 OK + $ RCPT TO:<foi+request-1234@example.com> + 250 Accepted + $ DATA + 354 Enter message, ending with "." on a line by itself + $ Subject: Test + $ + $ This is a test mail. + $ . + 250 OK id=1XHA03-0001Vx-Qn + QUIT -There is a great -[Exim Cheatsheet](http://bradthemad.org/tech/notes/exim_cheatsheet.php) -online that you may find useful. diff --git a/docs/installing/index.md b/docs/installing/index.md index c276c3d08..c04aaa3ca 100644 --- a/docs/installing/index.md +++ b/docs/installing/index.md @@ -6,17 +6,19 @@ title: Installing # Installing Alaveteli <p class="lead"> - Although you can install Alaveteli and just change it when you need it, we - recommend you adopt a way of <strong>deploying</strong> it automatically. - This has several advantages, especially for your - <a href="{{ site.baseurl }}docs/glossary/#production">production server</a>. + There are a number of ways to install Alaveteli. + We've made an Amazon Machine Image (AMI) so you can quickly deploy on + Amazon EC2 (handy if you just want to evaluate it, for example). + If you prefer to use your own server, there's an installation script + which does most of the work for you, or you can follow the manual + installation instructions. </p> ## Before you start This is important: you need to decide if you are installing Alaveteli for -[development]({{ site.baseurl }}docs/glossary/#development) or -[production]({{ site.baseurl }}docs/glossary/#production). +<a href="{{ site.baseurl }}docs/glossary/#development" class="glossary__link">development</a> or +<a href="{{ site.baseurl }}docs/glossary/#production" class="glossary__link">production</a>. A **development** site is one where you're going to change, customise, and perhaps experiment while you get it up and running. You should always do this @@ -30,9 +32,10 @@ messages switched off. It's important to be able to deploy changes to a production site quickly and efficiently, so we recommend you consider using a [deployment mechanism]({{ site.baseurl }}docs/installing/deploy/) too. -Ideally, you should also have a [staging site]({{ site.baseurl }}docs/glossary/#staging), +Ideally, you should also have a +<a href="{{ site.baseurl }}docs/glossary/#staging" class="glossary__link">staging site</a>, which is used solely to test new code in an identical environment to your -production site but before it goes live. +production site before it goes live. If you're in doubt, you're probably running a development site. Get it up and running, play with it, customise it, and -- later -- you can install it as a @@ -47,6 +50,7 @@ those servers, because Capistrano takes care of that for you. ## Installing the core code +* [Install into a Vagrant virtual development environment]({{ site.baseurl }}docs/installing/vagrant/) -- a good choice for development, and playing around with the site. * [Install on Amazon EC2]({{ site.baseurl }}docs/installing/ami/) using our AMI * [Use the installation script]({{ site.baseurl }}docs/installing/script/) which does the full installation on your own server * [Manual installation]({{ site.baseurl }}docs/installing/manual_install/) -- step-by-step instructions @@ -56,7 +60,7 @@ If you're setting up a development server on MacOS X, we've also got ## Other installation information -Alaveteli needs to be able to send and receive email, so you need to setup your -MTA (Mail Transfer Agent) appropriately. +Alaveteli needs to be able to send and receive email. If you're installing manually, you need to [setup your +MTA (Mail Transfer Agent) appropriately]({{ site.baseurl }}docs/installing/email/). The other install methods will do this for you. * [Installing the MTA]({{ site.baseurl }}docs/installing/email/) diff --git a/docs/installing/macos.md b/docs/installing/macos.md index c46bdf4ba..2c08be0e5 100644 --- a/docs/installing/macos.md +++ b/docs/installing/macos.md @@ -71,7 +71,7 @@ Read `rvm notes` and `rvm requirements` carefully for further instructions. Then The `mahoro` and `pg` gems require special installation commands. Rubygems must be downgraded to 1.6.2 to avoid deprecation warnings when running tests. rvm 1.8.7 - gem update --system 1.6.2 + gem update --system 2.1.11 gem install mahoro -- --with-ldflags="-L/usr/local/Cellar/libmagic/5.09/lib" --with-cppflags="-I/usr/local/Cellar/libmagic/5.09/include" env ARCHFLAGS="-arch x86_64" gem install pg @@ -88,32 +88,36 @@ The following is mostly from [the manual installation process]({{ site.baseurl}} ### Configure database -Creates Alaveteli databases and an `foi` user with password `foi`. +Create a database for your Mac user as homebrew doesn't create one by default: - echo "CREATE DATABASE foi_development encoding = 'UTF8'; - CREATE DATABASE foi_test encoding = 'UTF8'; - CREATE USER foi WITH CREATEUSER; - ALTER USER foi WITH PASSWORD 'foi'; - ALTER USER foi WITH CREATEDB; - GRANT ALL PRIVILEGES ON DATABASE foi_development TO foi; - GRANT ALL PRIVILEGES ON DATABASE foi_test TO foi; - ALTER DATABASE foi_development OWNER TO foi; - ALTER DATABASE foi_test OWNER TO foi;" | psql -h localhost template1 + createdb -### Clone Alaveteli +Create a `foi` user from the command line, like this: + + createuser -s -P foi + +_Note:_ After running this command you will be prompted to set a +password for the user. Don't leave it blank if you are new to +PostgreSQL, or it could be difficult to set later for you. + +We'll create a template for our Alaveteli databases: + + createdb -T template0 -E UTF-8 template_utf8 + echo "update pg_database set datistemplate=true where datname='template_utf8';" | psql -We don't want to vendor Rails, as it causes problems locally. +Then create the databases: + + createdb -T template_utf8 -O foi alaveteli_production + createdb -T template_utf8 -O foi alaveteli_test + createdb -T template_utf8 -O foi alaveteli_development + +### Clone Alaveteli git clone https://github.com/mysociety/alaveteli.git cd alaveteli git submodule init - - sed -i~ 's/\\[submodule "vendor\/rails"\\]//' .git/config - - sed -i~ 's/url = git:\/\/github.com\/rails\/rails.git//' .git/config git submodule update -**Note:** Due to Markdown bugs, the first `sed` command above does not display properly if it appears in blockquote. ### Configure Alaveteli diff --git a/docs/installing/manual_install.md b/docs/installing/manual_install.md index 777d95139..9cad6b5b9 100644 --- a/docs/installing/manual_install.md +++ b/docs/installing/manual_install.md @@ -17,91 +17,183 @@ title: Manual installation Note that there are [other ways to install Alaveteli]({{ site.baseurl }}docs/installing/). -## Target operating system +<div class="attention-box"> + <ul> + <li>Commands in this guide will require root privileges</li> + <li>Commands are intended to be run via the terminal or over ssh</li> + </ul> +</div> -These instructions assume Debian Squeeze (64-bit) or Ubuntu 12.04 LTS -(precise). Debian Squeeze is the best supported deployment platform. We also +## Configure the Operating System + +### Target operating system + +These instructions assume a 64-bit version of Debian 6 (Wheezy), Debian 7 (Squeeze) +or Ubuntu 12.04 LTS (Precise). Debian is the best supported deployment platform. We also have instructions for [installing on MacOS]({{ site.baseurl }}docs/installing/macos/). -Commands are intended to be run via the terminal or over ssh. +### Set the locale +**Debian Wheezy or Squeeze** -## Get Alaveteli +Follow the [Debian guide](https://wiki.debian.org/Locale#Standard) for configuring the locale of the operating system. + +Generate the locales you wish to make available. When the interactive screen asks you to pick a default locale, choose "None", as the SSH session will provide the locale required. -To start with, you may need to install git, e.g. with `sudo apt-get install -git-core` + dpkg-reconfigure locales + +Start a new SSH session to use your SSH locale. + +**Ubuntu Precise** -Next, get hold of the Alaveteli source code from github: +Unset the default locale, as the SSH session should provide the locale required. - git clone https://github.com/mysociety/alaveteli.git - cd alaveteli + update-locale LC_ALL= -This will get the rails-3-develop branch, which has the latest (possibly buggy) -code. If you don't want to add or try new features, swap to the master branch -(which always contains the latest stable release): +Start a new SSH session to use your SSH locale. - git checkout master +### Update the OS -## Install mySociety libraries +Update the Operating System with the latest packages -Next, install mySociety's common ruby libraries. To fetch the contents of the -submodules, run: + apt-get update -y + apt-get upgrade -y - git submodule update --init +`sudo` is not installed on Debian by default. Install it along with `git` (the version control tool we'll use to get a copy of the Alaveteli code). -## Install system dependencies + apt-get install -y sudo git-core + +### Prepare to install system dependencies using OS packages These are packages that the software depends on: third-party software used to parse documents, host the site, and so on. There are also packages that contain headers necessary to compile some of the gem dependencies in the next step. +#### Using other repositories to get more recent packages + Add the following repositories to `/etc/apt/sources.list`: **Debian Squeeze** - cat > /etc/apt/sources.list.d/debian-backports.list <<EOF + cat > /etc/apt/sources.list.d/debian-extra.list <<EOF + # Debian mirror to use, including contrib and non-free: + deb http://the.earth.li/debian/ squeeze main contrib non-free + deb-src http://the.earth.li/debian/ squeeze main contrib non-free + + # Security Updates: + deb http://security.debian.org/ squeeze/updates main non-free + deb-src http://security.debian.org/ squeeze/updates main non-free + + # Debian Backports deb http://backports.debian.org/debian-backports squeeze-backports main contrib non-free + deb-src http://backports.debian.org/debian-backports squeeze-backports main contrib non-free + + # Wheezy + deb http://ftp.uk.debian.org/debian wheezy main contrib non-free + EOF + +The squeeze-backports repository is providing a more recent version of rubygems, and the wheezy repository is providing bundler. You should configure package-pinning to reduce the priority of the wheezy repository so other packages aren't pulled from it. + + cat >> /etc/apt/preferences <<EOF + + Package: bundler + Pin: release n=wheezy + Pin-Priority: 990 + + Package: * + Pin: release n=wheezy + Pin-Priority: 50 EOF -The repositories above let you install `wkhtmltopdf-static` and `bundler` using -`apt`. +**Debian Wheezy** + + cat > /etc/apt/sources.list.d/debian-extra.list <<EOF + # Debian mirror to use, including contrib and non-free: + deb http://the.earth.li/debian/ wheezy main contrib non-free + deb-src http://the.earth.li/debian/ wheezy main contrib non-free + + # Security Updates: + deb http://security.debian.org/ wheezy/updates main non-free + deb-src http://security.debian.org/ wheezy/updates main non-free + EOF **Ubuntu Precise** cat > /etc/apt/sources.list.d/ubuntu-extra.list <<EOF - deb http://eu-west-1.ec2.archive.ubuntu.com/ubuntu/ precise multiverse - deb-src http://eu-west-1.ec2.archive.ubuntu.com/ubuntu/ precise multiverse - deb http://eu-west-1.ec2.archive.ubuntu.com/ubuntu/ precise-updates multiverse - deb-src http://eu-west-1.ec2.archive.ubuntu.com/ubuntu/ precise-updates multiverse + deb http://de.archive.ubuntu.com/ubuntu/ precise multiverse + deb-src http://de.archive.ubuntu.com/ubuntu/ precise multiverse + deb http://de.archive.ubuntu.com/ubuntu/ precise-updates multiverse + deb-src http://de.archive.ubuntu.com/ubuntu/ precise-updates multiverse + deb http://de.archive.ubuntu.com/ubuntu/ trusty universe + deb-src http://de.archive.ubuntu.com/ubuntu/ trusty universe EOF -The repositories above let you install `wkhtmltopdf-static` using `apt`. -`bundler` will have to be installed manually on Ubuntu Precise. +The trusty repo is used here to get a more recent version of bundler. You should configure package-pinning to reduce the priority of the trusty repository so other packages aren't pulled from it. + + cat >> /etc/apt/preferences <<EOF -### Packages customised by mySociety + Package: ruby-bundler + Pin: release n=trusty + Pin-Priority: 990 -If you're using Debian, you should add the mySociety Debian archive to your + Package: * + Pin: release n=trusty + Pin-Priority: 50 + EOF + + +#### Packages customised by mySociety + +If you're using Debian or Ubuntu, you should add the mySociety Debian archive to your apt sources. Note that mySociety packages are currently only built for 64-bit Debian. +**Debian Squeeze, Wheezy or Ubuntu Precise** + cat > /etc/apt/sources.list.d/mysociety-debian.list <<EOF deb http://debian.mysociety.org squeeze main EOF +The repository above lets you install `wkhtmltopdf-static` and `pdftk` (for squeeze) using `apt`. + Add the GPG key from the [mySociety Debian Package Repository](http://debian.mysociety.org/). - wget -O - https://debian.mysociety.org/debian.mysociety.org.gpg.key | sudo apt-key add - + wget -O - https://debian.mysociety.org/debian.mysociety.org.gpg.key | apt-key add - + +**Ubuntu Precise only** + + cat > /etc/apt/sources.list.d/mysociety-launchpad.list <<EOF + deb http://ppa.launchpad.net/mysociety/alaveteli/ubuntu precise main + deb-src http://ppa.launchpad.net/mysociety/alaveteli/ubuntu precise main + EOF + +The repository above lets you install a recent version of `pdftk` using `apt`. + +Add the GPG key from the +[mySociety Alaveteli Ubuntu Package Repository](https://launchpad.net/~mysociety/+archive/ubuntu/alaveteli). + + apt-get install -y python-software-properties + add-apt-repository -y ppa:mysociety/alaveteli + +**Debian Wheezy or Ubuntu Precise** -You should also configure package-pinning to reduce the priority of this -repository. +You should also configure package-pinning to reduce the priority of the +mysociety Debian repository - we only want to pull wkhtmltopdf-static +from mysociety. + + cat >> /etc/apt/preferences <<EOF - cat > /etc/apt/preferences <<EOF Package: * Pin: origin debian.mysociety.org Pin-Priority: 50 EOF -If you're using some other platform, you can optionally install these +**Debian Squeeze** + +No special package pinning is required. + +#### Other platforms +If you're using some other linux platform, you can optionally install these dependencies manually, as follows: 1. If you would like users to be able to get pretty PDFs as part of the @@ -115,37 +207,76 @@ everything will still work, but users will get ugly, plain text versions of their requests when they download them. 2. Version 1.44 of `pdftk` contains a bug which makes it loop forever in -certain edge conditions. Until it's incorporated into an official release, you -can either hope you don't encounter the bug (it ties up a rails process until -you kill it), patch it yourself, or use the Debian package -compiled by mySociety (see link in [issue -305](https://github.com/mysociety/alaveteli/issues/305)) +certain edge conditions. This is fixed in the standard 1.44.7 package which is available in wheezy (Debian) and raring (Ubuntu). + +If you can't get an official release for your OS with the fix, you can +either hope you don't encounter the bug (it ties up a rails process +until you kill it), patch it yourself, or use the +[Debian](http://debian.mysociety.org/dists/squeeze/main/binary-amd64/) +or +[Ubuntu](https://launchpad.net/~mysociety/+archive/ubuntu/alaveteli/+packages) +packages compiled by mySociety. -### Install the dependencies +#### Refresh sources Refresh the sources after adding the extra repositories: - sudo apt-get update + apt-get -y update + +### Create Alaveteli User + +Create a new linux user to run the Alaveteli application. + + adduser --quiet --disabled-password --gecos "Alaveteli" alaveteli + +## Get Alaveteli + +Create the target directory and clone the Alaveteli source code in to this directory: + + mkdir -p /var/www/alaveteli + chown alaveteli:alaveteli /var/www + chown alaveteli:alaveteli /var/www/alaveteli + cd /home/alaveteli + sudo -u alaveteli git clone --recursive \ + --branch master \ + https://github.com/mysociety/alaveteli.git /var/www/alaveteli + +This clones the master branch which always contains the latest stable release. If you want to try out the latest (possibly buggy) code you can switch to the `rails-3-develop` branch. + + pushd /var/www/alaveteli + sudo -u alaveteli git checkout rails-3-develop + sudo -u alaveteli git submodule update + popd + +The `--recursive` option installs mySociety's common libraries which are required to run Alaveteli. + +## Install the dependencies Now install the packages relevant to your system: + # Debian Wheezy + apt-get -y install $(cat /var/www/alaveteli/config/packages.debian-wheezy) + # Debian Squeeze - sudo apt-get install $(cat config/packages.debian-squeeze) + apt-get -y install $(cat /var/www/alaveteli/config/packages.debian-squeeze) # Ubuntu Precise - sudo apt-get install $(cat config/packages.ubuntu-precise) + apt-get -y install $(cat /var/www/alaveteli/config/packages.ubuntu-precise) Some of the files also have a version number listed in config/packages - check that you have appropriate versions installed. Some also list "`|`" and offer a choice of packages. -## Install Ruby dependencies +<div class="attention-box"> + +<strong>Note:</strong> To install Alaveteli's Ruby dependencies, you need to install bundler. In +Debian and Ubuntu, this is provided as a package (installed as part of the +package install process above). For other OSes, you could also install it as a gem: + + <pre><code> gem install bundler --no-rdoc --no-ri</code></pre> -To install Alaveteli's Ruby dependencies, you need to install bundler. In -Debian, this is provided as a package (installed as part of the package install -process above). You could also install it as a gem: +</div> - sudo gem install bundler ## Configure Database @@ -153,348 +284,573 @@ There has been a little work done in trying to make the code work with other databases (e.g., SQLite), but the currently supported database is PostgreSQL ("postgres"). -If you don't have postgres installed: - - $ sudo apt-get install postgresql postgresql-client - Create a `foi` user from the command line, like this: - # sudo -u postgres createuser -s -P foi + sudo -u postgres createuser -s -P foi _Note:_ Leaving the password blank will cause great confusion if you're new to PostgreSQL. +We'll create a template for our Alaveteli databases: + + sudo -u postgres createdb -T template0 -E UTF-8 template_utf8 + echo "update pg_database set datistemplate=true where datname='template_utf8';" > /tmp/update-template.sql + sudo -u postgres psql -f /tmp/update-template.sql + rm /tmp/update-template.sql + Then create the databases: - # sudo -u postgres createdb -T template0 -E SQL_ASCII -O foi foi_production - # sudo -u postgres createdb -T template0 -E SQL_ASCII -O foi foi_test - # sudo -u postgres createdb -T template0 -E SQL_ASCII -O foi foi_development + sudo -u postgres createdb -T template_utf8 -O foi alaveteli_production + sudo -u postgres createdb -T template_utf8 -O foi alaveteli_test + sudo -u postgres createdb -T template_utf8 -O foi alaveteli_development + +## Configure email + +You will need to set up an email server – or Mail Transfer Agent (MTA) – to +send and receive emails. + +Full configuration for an MTA is beyond the scope of this document -- see the guide for [configuring the Exim4 or Postfix MTAs]({{ site.baseurl }}docs/installing/email/). + +Note that in development mode mail is handled by [`mailcatcher`](http://mailcatcher.me/) by default so +that you can see the mails in a browser. Start mailcatcher by running `bundle exec mailcatcher` in the application directory. + +## Configure Alaveteli + +Alaveteli has three main configuration files: + + - `config/database.yml`: Configures Alaveteli to communicate with the database + - `config/general.yml`: The general Alaveteli application settings + - `config/newrelic.yml`: Configuration for the [NewRelic](http://newrelic.com) monitoring service + +Copy the configuration files and update their permissions: -We create using the ``SQL_ASCII`` encoding, because in postgres this is means -"no encoding"; and because we handle and store all kinds of data that may not -be valid UTF (for example, data originating from various broken email clients -that's not 8-bit clean), it's safer to be able to store *anything*, than reject -data at runtime. + cp /var/www/alaveteli/config/database.yml-example /var/www/alaveteli/config/database.yml + cp /var/www/alaveteli/config/general.yml-example /var/www/alaveteli/config/general.yml + cp /var/www/alaveteli/config/newrelic.yml-example /var/www/alaveteli/config/newrelic.yml + chown alaveteli:alaveteli /var/www/alaveteli/config/{database,general,newrelic}.yml + chmod 640 /var/www/alaveteli/config/{database,general,newrelic}.yml -Now you need to set up the database config file to contain the name, username -and password of your postgres database. +### database.yml -* Copy `database.yml-example` to `database.yml` in `alaveteli/config` -* Edit it to point to your local postgresql database in the development - and test sections. +Now you need to set up the database config file so that the application can +connect to the postgres database. + +Edit each section to point to the relevant local postgresql database. Example `development` section of `config/database.yml`: development: adapter: postgresql - database: foi_development + template: template_utf8 + database: alaveteli_development username: foi password: secure-password-here host: localhost port: 5432 Make sure that the user specified in `database.yml` exists, and has full -permissions on these databases. As they need the ability to turn off -constraints whilst running the tests they also need to be a superuser +permissions on these databases. +As the user needs the ability to turn off constraints whilst running the tests +they also need to be a superuser (clarification: a <em>Postgres</em> superuser, +not an Alaveteli +<a href="{{ site.baseurl }}docs/glossary/#super" class="glossary__link">superuser</a>). If you don't want your database user to be a superuser, you can add this line -to the test config in `database.yml` (as seen in `database.yml-example`) +to the `test` section in `database.yml` (as seen in `config/database.yml-example`): constraint_disabling: false -## Configure email +### general.yml -You will need to set up an email server (MTA) to send and receive emails. Full -configuration for an MTA is beyond the scope of this document -- see this -[example config for Exim4]({{ site.baseurl }}docs/installing/email/). +We have a full [guide to Alaveteli configuration]({{ site.baseurl }}docs/customising/config/) which covers all the settings in `config/general.yml`. -Note that in development mode mail is handled by mailcatcher by default so -that you can see the mails in a browser - see [http://mailcatcher.me/](http://mailcatcher.me/) for more -details. Start mailcatcher by running `bundle exec mailcatcher` in your -application directory. +_Note:_ If you are setting up Alaveteli to run in production, set the [`STAGING_SITE`]({{ site.baseurl }}docs/customising/config/#staging_site) variable to `0` in `/var/www/alaveteli/config/general.yml` now. -### Minimal + STAGING_SITE: 0 -If you just want to get the tests to pass, you will at a minimum need to allow -sending emails via a `sendmail` command (a requirement met, for example, with -`sudo apt-get install exim4`). +The default settings for frontpage examples are designed to work with +the dummy data shipped with Alaveteli; once you have real data, you should +certainly edit these. -### Detailed +The default theme is the ["Alaveteli" theme](https://github.com/mysociety/alavetelitheme). When you run `rails-post-deploy` (see below), that theme gets installed automatically. -When an authority receives an email, the email's `reply-to` field is a magic -address which is parsed and consumed by the Rails app. +### newrelic.yml -To receive such email in a production setup, you will need to configure your -MTA to pipe incoming emails to the Alaveteli script `script/mailin`. Therefore, -you will need to configure your MTA to accept emails to magic addresses, and to -pipe such emails to this script. +This file contains configuration information for the New Relic performance +management system. By default, monitoring is switched off by the +`agent_enabled: false` setting. See New Relic's [remote performance analysis](https://github.com/newrelic/rpm) instructions for switching it on +for both local and remote analysis. -Magic email addresses are of the form: +## Deployment - <foi+request-3-691c8388@example.com> +You should run the `rails-post-deploy` script after each new software upgrade: -The respective parts of this address are controlled with options in -`config/general.yml`, thus: + sudo -u alaveteli RAILS_ENV=production \ + /var/www/alaveteli/script/rails-post-deploy - INCOMING_EMAIL_PREFIX = 'foi+' - INCOMING_EMAIL_DOMAIN = 'example.com' +This installs Ruby dependencies, installs/updates themes, runs database +migrations, updates shared directories and runs other tasks that need to be run +after a software update, like precompiling static assets for a production install. -When you set up your MTA, if there is some error inside Rails, the -email is returned with an exit code 75, which for Exim at least means the MTA -will try again later. Additionally, a stacktrace is emailed to `CONTACT_EMAIL`. +That the first time you run this script can take a *long* time, as it must +compile native dependencies for `xapian-full`. -See [this example]({{ site.baseurl }}docs/installing/email/) for a possible configuration for Exim (>=1.9). +Create the index for the search engine (Xapian): -A well-configured installation of this code will have had Exim make -a backup copy of the email in a separate mailbox, just in case. + sudo -u alaveteli RAILS_ENV=production \ + /var/www/alaveteli/script/rebuild-xapian-index -## Set up configs +If this fails, the site should still mostly run, but it's a core component so +you should really try to get this working. -Copy `config/general.yml-example` to `config/general.yml` and edit to your -taste. +<div class="attention-box"> + Note that we set <code>RAILS_ENV=production</code>. Use + <code>RAILS_ENV=development</code> if you are installing Alaveteli to make + changes to the code. +</div> -Note that the default settings for frontpage examples are designed to work with -the dummy data shipped with Alaveteli; once you have real data, you should -certainly edit these. +## Configure the Application Server -The default theme is the "Alaveteli" theme. When you run `rails-post-deploy` -(see below), that theme gets installed automatically. +Alaveteli can run under many popular application servers. mySociety recommends +the use of [Phusion Passenger](https://www.phusionpassenger.com) (AKA +mod_rails) or [thin](http://code.macournoyer.com/thin). -Finally, copy `config/newrelic.yml-example` to `config/newrelic.yml`. This file -contains configuration information for the New Relic performance management -system. By default, monitoring is switched off by the `agent_enabled: false` -setting. See New Relic's [remote performance analysis](https://github.com/newrelic/rpm) instructions for switching it on -for both local and remote analysis. +### Using Phusion Passenger +Passenger is the recommended application server as it is well proven in +production environments. It is implemented as an Apache mod, so it cannot be +run independently. -## Deployment + apt-get install -y libapache2-mod-passenger -In the `alaveteli` directory, run: +See later in the guide for configuring the Apache web server with Passenger. - script/rails-post-deploy +### Using Thin -(This will need execute privs so `chmod 755` if necessary.) This sets up -directory structures, creates logs, installs/updates themes, runs database -migrations, etc. You should run it after each new software update. +Thin is a lighter-weight application server which can be run independently of +a web server. Thin will be installed in the application bundle and used to run Alaveteli by default. -One of the things the script does is install dependencies (using `bundle -install`). Note that the first time you run it, part of the `bundle install` -that compiles `xapian-full` takes a *long* time! +Run the following to get the server running: -If you want some dummy data to play with, you can try loading the fixtures that -the test suite uses into your development database. You can do this with: + cd /var/www/alaveteli + bundle exec thin \ + --environment=production \ + --user=alaveteli \ + --group=alaveteli \ + start - script/load-sample-data +By default the server listens on all interfaces. You can restrict it to the +localhost interface by adding `--address=127.0.0.1` -Next, create the index for the search engine (Xapian): +The server should have told you the URL to access in your browser to see the +site in action. - script/rebuild-xapian-index +You can daemonize the process by starting it with the `--daemonize` option. -If this fails, the site should still mostly run, but it's a core component so -you should really try to get this working. +Later in this guide we'll actually create a SysVinit daemon to run the application, so stop any thin processes you've started here. -## Run the Tests +## Cron jobs and Daemons -Make sure everything looks OK: +The crontab and init scripts use the `.ugly` file format, which is a strange +templating format used by mySociety. - bundle exec rake spec +The `ugly` format uses simple variable substitution. A variable looks like +`!!(*= $this *)!!`. -If there are failures here, something has gone wrong with the preceding steps -(see the next section for a common problem and workaround). You might be able -to move on to the next step, depending on how serious they are, but ideally you -should try to find out what's gone wrong. +### Generate crontab -### glibc bug workaround +`config/crontab-example` contains the cron jobs that run on +Alaveteli. Rewrite the example file to replace the variables, +and then drop it in `/etc/cron.d/` on the server. -There's a [bug in -glibc](http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=637239) which causes -Xapian to segfault when running the tests. Although the bug report linked to -claims it's fixed in the current Debian stable, it's not as of version -`2.11.3-2`. +**Template Variables:** -Until it's fixed (e.g. `libc6 2.13-26` does work), you can get the tests to -pass by setting `export LD_PRELOAD=/lib/libuuid.so.1`. +* `vhost_dir`: the full path to the directory where alaveteli is checked out. + e.g. If your checkout is at `/var/www/alaveteli` then set this to `/var/www` +* `vcspath`: the name of the directory that contains the alaveteli code. + e.g. `alaveteli` +* `user`: the user that the software runs as +* `site`: a string to identify your alaveteli instance +* `mailto`: The email address or local account that cron output will be sent to - setting an email address depends on your MTA having been configured for remote delivery. -## Run the Server +There is a rake task that will help to rewrite this file into one that is +useful to you. This example sends cron output to the local `alaveteli` user. Change the variables to suit your installation. -Run the following to get the server running: + pushd /var/www/alaveteli + bundle exec rake config_files:convert_crontab \ + DEPLOY_USER=alaveteli \ + VHOST_DIR=/var/www \ + VCSPATH=alaveteli \ + SITE=alaveteli \ + MAILTO=alaveteli \ + CRONTAB=/var/www/alaveteli/config/crontab-example > /etc/cron.d/alaveteli + popd - bundle exec rails server --environment=development + chown root:alaveteli /etc/cron.d/alaveteli + chmod 754 /etc/cron.d/alaveteli -By default the server listens on all interfaces. You can restrict it to the -localhost interface by adding `--binding=127.0.0.1` +### Generate application daemon -The server should have told you the URL to access in your browser to see the -site in action. +Generate a daemon based on the application server you installed. This allows you +to use the native `service` command to stop, start and restart the application. -## Administrator privileges +#### Passenger -The administrative interface is at the URL `/admin`. +**Template Variables:** -Only users with the `super` admin level can access the admin interface. Users -create their own accounts in the usual way, and then administrators can give -them `super` privileges. +* `vhost_dir`: the full path to the directory where alaveteli is checked out. + e.g. If your checkout is at `/var/www/alaveteli` then set this to `/var/www` +* `vcspath`: the name of the directory that contains the alaveteli code. + e.g. `alaveteli` +* `site`: a string to identify your alaveteli instance +* `user`: the user that the software runs as -There is an emergency user account which can be accessed via -`/admin?emergency=1`, using the credentials `ADMIN_USERNAME` and -`ADMIN_PASSWORD`, which are set in `general.yml`. To bootstrap the -first `super` level accounts, you will need to log in as the emergency -user. You can disable the emergency user account by setting `DISABLE_EMERGENCY_USER` to `true` in `general.yml`. +There is a rake task that will help to rewrite this file into one that is +useful to you. Change the variables to suit your installation. -Users with the superuser role also have extra privileges in the website -frontend, such as being able to categorise any request, being able to view -items that have been hidden from the search, and being presented with "admin" -links next to individual requests and comments in the front end. + pushd /var/www/alaveteli + bundle exec rake config_files:convert_init_script \ + DEPLOY_USER=alaveteli \ + VHOST_DIR=/var/www \ + VCSPATH=alaveteli \ + SITE=alaveteli \ + SCRIPT_FILE=/var/www/alaveteli/config/sysvinit-passenger.ugly > /etc/init.d/alaveteli + popd -It is possible completely to override the administrator authentication by -setting `SKIP_ADMIN_AUTH` to `true` in `general.yml`. + chown root:alaveteli /etc/init.d/alaveteli + chmod 754 /etc/init.d/alaveteli -## Cron jobs and init scripts +Start the application: -`config/crontab-example` contains the cronjobs run on WhatDoTheyKnow. It's in a -strange templating format they use in mySociety. mySociety render the example -file to reference absolute paths, and then drop it in `/etc/cron.d/` on the -server. + service alaveteli start -The `ugly` format uses simple variable substitution. A variable looks like -`!!(*= $this *)!!`. The variables are: - -* `vhost`: part of the path to the directory where the software is - served from. In the mySociety files, it usually comes as - `/data/vhost/!!(*= $vhost *)!!` -- you should replace that whole - port with a path to the directory where your Alaveteli software - installation lives, e.g. `/var/www/` -* `vhost_dir`: the entire path to the directory where the software is - served from. -- you should replace this with a path to the - directory where your Alaveteli software installation lives, - e.g. `/var/www/` -* `vcspath`: the name of the alaveteli checkout, e.g. `alaveteli`. - Thus, `/data/vhost/!!(*= $vhost *)!!/!!(*= $vcspath *)!!` might be - replaced with `/var/www/alaveteli` in your cron tab +#### Thin + +**Template Variables:** + +* `vhost_dir`: the full path to the directory where alaveteli is checked out. + e.g. If your checkout is at `/var/www/alaveteli` then set this to `/var/www` +* `vcspath`: the name of the directory that contains the alaveteli code. + e.g. `alaveteli` +* `site`: a string to identify your alaveteli instance * `user`: the user that the software runs as + +There is a rake task that will help to rewrite this file into one that is +useful to you. Change the variables to suit your installation. + + pushd /var/www/alaveteli + bundle exec rake config_files:convert_init_script \ + DEPLOY_USER=alaveteli \ + VHOST_DIR=/var/www \ + VCSPATH=alaveteli \ + SITE=alaveteli \ + SCRIPT_FILE=/var/www/alaveteli/config/sysvinit-thin.ugly > /etc/init.d/alaveteli + popd + + chown root:alaveteli /etc/init.d/alaveteli + chmod 754 /etc/init.d/alaveteli + +Start the application: + + service alaveteli start + +### Generate alert daemon + +One of the cron jobs refers to a script at `/etc/init.d/alaveteli-alert-tracks`. This +is an init script, which can be generated from the +`config/alert-tracks-debian.ugly` template. This script sends out emails to users subscribed to updates from the site – known as [`tracks`]({{ site.baseurl }}docs/installing/email/#tracks-mail) – when there is something new matching their interests. + +**Template Variables:** + +* `daemon_name`: The name of the daemon. This is set by the rake task. +* `vhost_dir`: the full path to the directory where alaveteli is checked out. + e.g. If your checkout is at `/var/www/alaveteli` then set this to `/var/www` +* `vcspath`: the name of the directory that contains the alaveteli code. + e.g. `alaveteli` * `site`: a string to identify your alaveteli instance +* `user`: the user that the software runs as There is a rake task that will help to rewrite this file into one that is -useful to you, which can be invoked with: +useful to you. Change the variables to suit your installation. - bundle exec rake config_files:convert_crontab \ - DEPLOY_USER=deploy \ - VHOST_DIR=/dir/above/alaveteli \ - VCSPATH=alaveteli \ - SITE=alaveteli \ - CRONTAB=config/crontab-example > crontab - -You should change the `DEPLOY_USER`, `VHOST_DIR`, `VCSPATH` and `SITE` -environment variables to match your server and installation. You should also -edit the resulting `crontab` file to customize the `MAILTO` variable. - -One of the cron jobs refers to a script at `/etc/init.d/foi-alert-tracks`. This -is an init script, a copy of which lives in `config/alert-tracks-debian.ugly`. -As with the cron jobs above, replace the variables (and/or bits near the -variables) with paths to your software. You can use the rake task `rake -config_files:convert_init_script` to do this. + pushd /var/www/alaveteli + bundle exec rake RAILS_ENV=production config_files:convert_init_script \ + DEPLOY_USER=alaveteli \ + VHOST_DIR=/var/www \ + VCSPATH=alaveteli \ + SITE=alaveteli \ + SCRIPT_FILE=/var/www/alaveteli/config/alert-tracks-debian.ugly > /etc/init.d/alaveteli-alert-tracks + popd + + chown root:alaveteli /etc/init.d/alaveteli-alert-tracks + chmod 754 /etc/init.d/alaveteli-alert-tracks + +Start the alert tracks daemon: + + service alaveteli-alert-tracks start + +### Generate varnish purge daemon `config/purge-varnish-debian.ugly` is a similar init script, which is optional -and not required if you choose not to run your site behind Varnish (see below). -Either tweak the file permissions to make the scripts executable by your deploy -user, or add the following line to your sudoers file to allow these to be run -by your deploy user (named `deploy` in this case): +and not required if you choose not to run your site behind Varnish (see below). It notifies Varnish of cached pages that need to be purged from Varnish's cache. It will not run if Varnish is not installed. - deploy ALL = NOPASSWD: /etc/init.d/foi-alert-tracks, /etc/init.d/foi-purge-varnish +**Template Variables:** -The cron jobs refer to a program `run-with-lockfile`. See [this -issue](https://github.com/mysociety/alaveteli/issues/112) for a discussion of -where to find this program, and how you might replace it. This [one line -script](https://gist.github.com/3741194) can install this program system-wide. +* `daemon_name`: The name of the daemon. This is set by the rake task. +* `vhost_dir`: the full path to the directory where alaveteli is checked out. + e.g. If your checkout is at `/var/www/alaveteli` then set this to `/var/www` +* `vcspath`: the name of the directory that contains the alaveteli code. + e.g. `alaveteli` +* `site`: a string to identify your alaveteli instance +* `user`: the user that the software runs as -## Set up production web server +There is a rake task that will help to rewrite this file into one that is +useful to you. Change the variables to suit your installation. -It is not recommended to run the website using the default Rails web server. -There are various recommendations here: http://rubyonrails.org/deploy + pushd /var/www/alaveteli + bundle exec rake RAILS_ENV=production config_files:convert_init_script \ + DEPLOY_USER=alaveteli \ + VHOST_DIR=/var/www \ + VCSPATH=alaveteli \ + SITE=alaveteli \ + SCRIPT_FILE=/var/www/alaveteli/config/purge-varnish-debian.ugly > /etc/init.d/alaveteli-purge-varnish + popd -We usually use Passenger / mod_rails. The file at `conf/httpd.conf-example` -gives you an example config file for WhatDoTheyKnow. At a minimum, you should -include the following in an Apache configuration file: + chown root:alaveteli /etc/init.d/alaveteli-purge-varnish + chmod 754 /etc/init.d/alaveteli-purge-varnish - PassengerResolveSymlinksInDocumentRoot on - PassengerMaxPoolSize 6 # Recommend setting this to 3 or less on servers with 512MB RAM +Start the alert tracks daemon: -Under all but light loads, it is strongly recommended to run the server behind -an http accelerator like Varnish. A sample varnish VCL is supplied in -`conf/varnish-alaveteli.vcl`. + service alaveteli-purge-varnish start + + +## Configure the web server + +In almost all scenarios, we recommend running the Alaveteli Rails application +behind a web server. This allows the web server to serve static content without +going through the Rails stack, which improves performance. + +We recommend two main combinations of application and web server: + +- Apache & Passenger +- Nginx & Thin + +There are ways to run Passenger with Nginx, and indeed Thin with Apache, but +that's out of scope for this guide. If you want to do something that isn't +documented here, get in touch on [alaveteli-dev](https://groups.google.com/forum/#!forum/alaveteli-dev) and we'll +be more than happy to help you get set up. + +You should have already installed an application server if you have followed +this guide, so pick the appropriate web server to configure. + +### Apache (with Passenger) + +Install Apache with the Suexec wrapper: + + apt-get install -y apache2 + apt-get install -y apache2-suexec + +Enable the required modules + + a2enmod actions + a2enmod expires + a2enmod headers + a2enmod passenger + a2enmod proxy + a2enmod proxy_http + a2enmod rewrite + a2enmod suexec + +Create a directory for optional Alaveteli configuration + + mkdir -p /etc/apache2/vhost.d/alaveteli + +Copy the example VirtualHost configuration file. You will need to change all +occurrences of `www.example.com` to your URL + + cp /var/www/alaveteli/config/httpd.conf-example \ + /etc/apache2/sites-available/alaveteli + +Disable the default site and enable the `alaveteli` VirtualHost + + a2dissite default + a2ensite alaveteli + +Check the configuration and fix any issues + + apachectl configtest + +Restart apache to load the new Alaveteli config + + service apache2 graceful + +It's strongly recommended that you run the site over SSL. (Set `FORCE_SSL` to +true in `config/general.yml`). For this you will need an SSL certificate for your domain. + +Enable the SSL apache mod + + a2enmod ssl + +Copy the SSL configuration – again changing `www.example.com` to your domain – +and enable the VirtualHost + + cp /var/www/alaveteli/config/httpd-ssl.conf.example \ + /etc/apache2/sites-available/alaveteli_https + a2ensite alaveteli_https -It's strongly recommended that you run the site over SSL. (Set FORCE_SSL to -true in config/general.yml). For this you will need an SSL certificate for your -domain and you will need to configure an SSL terminator to sit in front of -Varnish. If you're already using Apache as a web server you could simply use -Apache as the SSL terminator. A minimal configuration would look something like -this: +Force HTTPS requests from the HTTP VirtualHost - <VirtualHost *:443> - ServerName www.yourdomain + cp /var/www/alaveteli/config/httpd-force-ssl.conf.example \ + /etc/apache2/vhost.d/alaveteli/force-ssl.conf - ProxyRequests Off - ProxyPreserveHost On - ProxyPass / http://localhost:80/ - ProxyPassReverse / http://localhost:80/ - RequestHeader set X-Forwarded-Proto 'https' +If you are testing Alaveteli or setting up an internal staging site, generate +self-signed SSL certificates. **Do not use self-signed certificates for a +production server**. Replace `www.example.com` with your domain name. - SSLEngine on - SSLProtocol all -SSLv2 - SSLCipherSuite ALL:!ADH:!EXPORT:!SSLv2:RC4+RSA:+HIGH:+MEDIUM + openssl genrsa -out /etc/ssl/private/www.example.com.key 2048 + chmod 640 /etc/ssl/private/www.example.com.key - SSLCertificateFile /etc/apache2/ssl/ssl.crt - SSLCertificateKeyFile /etc/apache2/ssl/ssl.key - SSLCertificateChainFile /etc/apache2/ssl/sub.class2.server.ca.pem - SSLCACertificateFile /etc/apache2/ssl/ca.pem - SetEnvIf User-Agent ".*MSIE.*" nokeepalive ssl-unclean-shutdown + openssl req -new -x509 \ + -key /etc/ssl/private/www.example.com.key \ + -out /etc/ssl/certs/www.example.com.cert \ + -days 3650 \ + -subj /CN=www.example.com + chmod 640 /etc/ssl/certs/www.example.com.cert - </VirtualHost> +Check the configuration and fix any issues -Notice the line `RequestHeader` that sets the `X-Forwarded-Proto` header. This -is important. This ultimately tells Rails that it's serving a page over https -and so it knows to include that in any absolute urls it serves. + apachectl configtest + +Restart apache to load the new Alaveteli config. This will also restart +Passenger (the application server). + + service apache2 graceful + +### Nginx (with Thin) + +Install nginx + + apt-get install -y nginx + +#### Running over SSL + +It's strongly recommended that you run the site over SSL. (Set `FORCE_SSL` to +true in `config/general.yml`). For this you will need an SSL certificate for your domain. + +Copy the SSL configuration – changing `www.example.com` to your domain – +and enable the `alaveteli_https` server, disabling the default site. + + cp /var/www/alaveteli/config/nginx-ssl.conf.example \ + /etc/nginx/sites-available/alaveteli_https + rm /etc/nginx/sites-enabled/default + ln -s /etc/nginx/sites-available/alaveteli_https \ + /etc/nginx/sites-enabled/alaveteli_https + +<div class="attention-box"> + <strong>Note:</strong> For historical reasons, <code>nginx-ssl.conf.example</code> has the path to Alaveteli set as <code>/var/www/alaveteli/alaveteli</code> – you will need to manually change this to <code>/var/www/alaveteli</code>, or to the root of your Alaveteli install +</div> + +If you are testing Alaveteli or setting up an internal staging site, generate +self-signed SSL certificates. **Do not use self-signed certificates for a +production server**. Replace `www.example.com` with your domain name. + + openssl genrsa -out /etc/ssl/private/www.example.com.key 2048 + chmod 640 /etc/ssl/private/www.example.com.key + + openssl req -new -x509 \ + -key /etc/ssl/private/www.example.com.key \ + -out /etc/ssl/certs/www.example.com.cert \ + -days 3650 \ + -subj /CN=www.example.com + chmod 640 /etc/ssl/certs/www.example.com.cert + +Check the configuration and fix any issues + + service nginx configtest + +Reload the new nginx configuration and restart the application + + service nginx reload + service alaveteli restart + +#### Running without SSL + +Set `FORCE_SSL` to +false in `config/general.yml`. Copy the example nginx config + + cp /var/www/alaveteli/config/nginx.conf.example \ + /etc/nginx/sites-available/alaveteli + +<div class="attention-box"> + <strong>Note:</strong> For historical reasons, <code>nginx.conf.example</code> has the path to Alaveteli set as <code>/var/www/alaveteli/alaveteli</code> – you will need to manually change this to <code>/var/www/alaveteli</code>, or to the root of your Alaveteli install +</div> + +Disable the default site and enable the `alaveteli` server + + rm /etc/nginx/sites-enabled/default + ln -s /etc/nginx/sites-available/alaveteli \ + /etc/nginx/sites-enabled/alaveteli + +Check the configuration and fix any issues + + service nginx configtest + +Start the rails application with thin (if you haven't already). + + service alaveteli start + +Reload the nginx configuration + + service nginx reload + + +--- + +## Add varnish as an HTTP accelerator + +Under all but light loads, it is strongly recommended to run the server behind +an http accelerator like Varnish. A sample varnish VCL is supplied in +`conf/varnish-alaveteli.vcl`. + +If you are using SSL you will need to configure an SSL terminator to sit in +front of Varnish. If you're already using Apache as a web server you could +simply use Apache as the SSL terminator. We have some [production server best practice notes]({{ site.baseurl}}docs/running/server/). +## What next? + +Check out the [next steps]({{ site.baseurl }}docs/installing/next_steps/). + ## Troubleshooting +* **Run the Tests** + + Make sure everything looks OK. As the alaveteli user, run: + + bundle exec rake spec + + If there are failures here, something has gone wrong with the preceding + steps (see the next section for a common problem and workaround). You might + be able to move on to the [next steps]({{ site.baseurl }}docs/installing/next_steps/), depending on how serious they are, but + ideally you should try to find out what's gone wrong. + + +<div class="attention-box"> + <strong>Note:</strong> If you have setup your install of Alaveteli for production, you will need to temporarily remove the file <code>config/rails_env.rb</code>, which is used to force the rails environment to production, and edit your <code>.bundle/config</code> file to remove the <code>BUNDLE_WITHOUT</code> line that excludes development dependencies. After you have done this, as the alaveteli user, run <code>bundle install</code>. You will also need to make alaveteli the owner of <code>/var/www/alaveteli/log/development.log</code>, and run the database migrations. + + <pre><code>chown alaveteli:alaveteli /var/www/alaveteli/log/development.log +sudo -u alaveteli bundle exec rake db:migrate</code></pre> + +You should then be able to run the tests. Don't forget to restore <code>config/rails_env.rb</code> when you're done. You will probably see some errors from cron jobs in the meantime, as they'll be running in development mode. + +</div> + + * **Incoming emails aren't appearing in my Alaveteli install** - First, you need to check that your MTA is delivering relevant - incoming emails to the `script/mailin` command. There are various - ways of setting your MTA up to do this; we have documented - [one way of doing it]({{ site.baseurl }}docs/installing/email/#troubleshooting-exim) - in Exim, including a command you can use to check that the email - routing is set up correctly. - - Second, you need to test that the mailin script itself is working - correctly, by running it from the command line, First, find a - valid "To" address for a request in your system. You can do this - through your site's admin interface, or from the command line, - like so: - - $ ./script/console - Loading development environment (Rails 2.3.14) - >> InfoRequest.find_by_url_title("why_do_you_have_such_a_fancy_dog").incoming_email - => "request-101-50929748@localhost" - - Now take the source of a valid email (there are some sample emails in - `spec/fixtures/files/`); edit the `To:` header to match this address; - and then pipe it through the mailin script. A non-zero exit code - means there was a problem. For example: - - $ cp spec/fixtures/files/incoming-request-plain.email /tmp/ - $ perl -pi -e 's/^To:.*/To: <request-101-50929748@localhost>/' /tmp/incoming-request-plain.email - $ ./script/mailin < /tmp/incoming-request-plain.email - $ echo $? - 75 - - The `mailin` script emails the details of any errors to - `CONTACT_EMAIL` (from your `general.yml` file). A common problem is - for the user that the MTA runs as not to have write access to - `files/raw_emails/`. + See the [general email troubleshooting guide]({{ site.baseurl }}docs/installing/email#general-email-troubleshooting). * **Various tests fail with "*Your PostgreSQL connection does not support unescape_bytea. Try upgrading to pg 0.9.0 or later.*"** diff --git a/docs/installing/next_steps.md b/docs/installing/next_steps.md new file mode 100644 index 000000000..2b7acbd45 --- /dev/null +++ b/docs/installing/next_steps.md @@ -0,0 +1,181 @@ +--- +layout: page +title: Next Steps +--- +# Next Steps + +<p class="lead"> + OK, you've installed a copy of Alaveteli, and can see the site in a browser. What next? +</p> + + * [Create a superuser admin account](#create-a-superuser-admin-account) + * [Load sample data](#load-sample-data) + * [Test out the request process](#test-out-the-request-process) + * [Import Public Authorities](#import-public-authorities) + * [Set the amount of time [...] to respond to requests](#set-the-amount-of-time-authorities-will-be-given-to-respond-to-requests) + * [Add some public holidays](#add-some-public-holidays) + * [Start thinking about customising Alaveteli](#start-thinking-about-customising-alaveteli) + + +## Create a superuser admin account + +Alaveteli ships with an +<a href="{{site.baseurl}}docs/glossary/#emergency" class="glossary__link">emergency user</a> +that has access to the admin. So when you've just created a new site, you +should sign up to create your own account, then log into admin as the emergency +user to promote your new account to be an administrator with +<a href="{{ site.baseurl }}docs/glossary/#super" class="glossary__link">super</a> +privilege. + +As soon as that's done, disable the emergency user, because you don't need to +use it any more: you've superseded it with your new admin account. + +Alaveteli ships with sample data that includes a dummy admin user called "Joe +Admin". If the sample data has been loaded into the database (this will depend on +how you installed), you must revoke Joe's administrator status too, because you +will be using your own admin account instead. + +### Step-by-step: + +First, in the browser: + +* Go to `/profile/sign_in` and create a user by signing up. +* Check your email and confirm your account. +* Go to `/admin?emergency=1`, log in with the username and password you specified in + [`ADMIN_USERNAME`]({{site.baseurl}}docs/customising/config/#admin_username) + and [`ADMIN_PASSWORD`]({{site.baseurl}}docs/customising/config/#admin_password). + You can find these settings in `config/general.yml`. +* You're now on the Alaveteli admin page. +* Click on **Users** (in the navigation menu across the top of the page), and + click on your name in the list of users. On *that* page, click **Edit**. +* Change your *Admin level* to "super" and click **Save**. +* From now on, when you are logged into your Alavateli site, you'll have access + to the admin (at `/admin`). Furthermore, you'll see links to admin pages off + the main site (which don't appear for regular users). + +If your installation has loaded the sample data, there will be a dummy user in +your database called "Joe Admin" who has admin status too. You should remove +this status so there's no risk of it being used to access your site admin. You +can either do this while you're still logged in as the emergency user... or +else, later, logged in as yourself: + +* Go to `/admin/users` or click on **Users** in the navigation menu on any + admin page. +* Find "Joe Admin" in the list of users, and click on the name to see the + user details. On *that* page, click **Edit**. +* Change the *Admin level* from "super" to "none" and click **Save**. +* Joe Admin no longer has admin status. + +Now that your account is a superuser admin, you don't need to allow the +emergency user access to the admin. On the command line shell, edit +`/var/www/alaveteli/alaveteli/config/general.yml`: + +* It's important that you change the emergency user's password (and, ideally, + the username too) from the values Alavateli ships with, because they are + public and hence insecure. In `general.yml`, change + [`ADMIN_PASSWORD`]({{site.baseurl}}docs/customising/config/#admin_password) + (and maybe [`ADMIN_USERNAME`]({{site.baseurl}}docs/customising/config/#admin_username) + too) to new, unique values. +* Additionally, you can totally disable the emergency user. Under normal + operation you don't need it, because from now on you'll be using the admin + user you've just created. + Set [`DISABLE_EMERGENCY_USER`]({{site.baseurl}}docs/customising/config/#disable_emergency_user) + to `true`. +* To apply these changes restart the service as a user with root privileges: + `sudo service alaveteli restart` + +You can use the same process (logged in as your admin account) to add or remove +superuser admin status to any users that are subsequently added to your site. +If you accidentally remove admin privilege from all accounts (try not to do +this, though!), you can enable the emergency user by editing the `general.yml` +file and restarting Alaveteli. + +## Load sample data + +If you want some dummy data to play with, you can try loading the fixtures that +the test suite uses into your development database. As the `alaveteli` user, do: + + script/load-sample-data + +If the sample data has already been loaded into the database, this command won't +do anything, but will instead <abbr +title='PG::Error: ERROR: permission denied: "RI_ConstraintTrigger_XXXXXX" is a system trigger'>fail +with an error</abbr>. + +If you have added the sample data, update the Xapian search index afterwards: + + script/update-xapian-index + +Remember that the sample data includes a user with admin access to your site. +You should revoke that status so it cannot be used to access your site -- +follow the steps described in the previous section. + +## Test out the request process + +* Create a new public authority in the admin interface -- give it a name like + "Test authority". Set the request email to an address that you will receive. + +* From the main interface of the site, make a request to the new authority. + +* You should receive the request email -- try replying to it. Your response + email should appear in Alaveteli. Not working? Take a look at our + [troubleshooting tips]({{ site.baseurl}}docs/installing/manual_install/#troubleshooting). + If that doesn't sort it out, [get in touch]({{ site.baseurl}}community/) on + the [developer mailing list](https://groups.google.com/forum/#!forum/alaveteli-dev) or [IRC](http://www.irc.mysociety.org/) for help. + +## Import Public Authorities + +Alaveteli can import a list of public authorities and their contact email addresses from a CSV file. + +Follow the instructions for +[uploading public authority data]({{ site.baseurl }}docs/running/admin_manual/#creating-changing-and-uploading-public-authority-data). + +## Set the amount of time authorities will be given to respond to requests + +In most countries that have a Freedom of Information law, authorities +have a certain number of days in order to respond to requests. Alaveteli +helps requesters by reminding them when their request is overdue for a +response according to the law. You can set the number of days an +authority is given to respond to a request in the +[`REPLY_LATE_AFTER_DAYS`]({{site.baseurl}}docs/customising/config/#reply_late_after_days), +[`REPLY_VERY_LATE_AFTER_DAYS`]({{site.baseurl}}docs/customising/config/#reply_very_late_after_days) +and +[`SPECIAL_REPLY_VERY_LATE_AFTER_DAYS`]({{site.baseurl}}docs/customising/config/#special_reply_very_late_after_days) +options in `config/general.yml`. Most laws specify that the days are +either working days, or calendar days. You can set this using the +[`WORKING_OR_CALENDAR_DAYS`]({{site.baseurl}}docs/customising/config/#working_or_calendar_days) +option in `config/general.yml`. + +## Add some public holidays + +<div class="attention-box info"> +Interface introduced in Alaveteli version 0.21 +</div> + +Alaveteli calculates the due dates of requests taking account of the +<a href="{{ site.baseurl }}docs/glossary/#holiday" class="glossary__link">public holidays</a> +you enter into the admin interface. + +If you have set the +[`WORKING_OR_CALENDAR_DAYS`]({{site.baseurl}}docs/customising/config/#working_or_calendar_days) +setting for Alaveteli to `working`, the date when a response to a +request is officially overdue will be calculated in days that are not +weekends or public holidays. + +If you have set +[`WORKING_OR_CALENDAR_DAYS`]({{site.baseurl}}docs/customising/config/#working_or_calendar_days) +to `calendar`, the date will be calculated in calendar days, but if the +due date falls on a public holiday or weekend day, then the due date is +considered to be the next week day that isn't a holiday. + +To add public holidays, go to the +<a href="{{ site.baseurl }}docs/glossary/#admin" class="glossary__link">admin interface</a> +and click on **Holidays**. From here you can either add each day of holiday by +hand, using the **New Holiday** button, or you can create multiple holidays at +once using the **Create holidays from suggestions or iCalendar feed** button. + +## Start thinking about customising Alaveteli + +Check out [our guide]({{ site.baseurl}}docs/customising/). + + diff --git a/docs/installing/script.md b/docs/installing/script.md index b8b678a0a..72fbd6438 100644 --- a/docs/installing/script.md +++ b/docs/installing/script.md @@ -1,6 +1,6 @@ --- layout: page -title: Installing the easy way +title: Installation script --- # Installation script @@ -16,7 +16,7 @@ Note that there are [other ways to install Alaveteli]({{ site.baseurl }}docs/ins If you have a clean installation of Debian squeeze 64-bit or Ubuntu precise, you can use an install script in our commonlib repository to set up a working instance of Alaveteli. This is not suitable for production (it runs in development mode, -for example) but should set up a functional installation of the site. +for example) but should set up a functional installation of the site, which can send and receive email. **Warning: only use this script on a newly installed server – it will make significant changes to your server’s setup, including modifying your nginx @@ -25,7 +25,7 @@ etc.** To download the script, run the following command: - curl -O https://raw.github.com/mysociety/commonlib/master/bin/install-site.sh + curl -O https://raw.githubusercontent.com/mysociety/commonlib/master/bin/install-site.sh If you run this script with `sh install-site.sh`, you'll see its usage message: @@ -56,10 +56,20 @@ could download the script, make it executable and then invoke it with: sudo ./install-site.sh --default alaveteli alaveteli +If you have any problems or questions, please ask on the [Alaveteli Google + Group](https://groups.google.com/forum/#!forum/alaveteli-dev) or [report an + issue](https://github.com/mysociety/alaveteli/issues?state=open). + +## What the install script does + When the script has finished, you should have a working copy of the website, -accessible via the hostname you supplied to the script. +accessible via the hostname you supplied to the script. So, for this example, you could access the site in a browser at `http://alaveteli.10.10.10.30.xip.io`. The site runs using the thin application server, and the nginx webserver. By default, Alaveteli will be installed into `/var/www/[HOST]` on the server. + +The server will also be configured to accept replies to information request emails (as long as the MX record for the domain is pointing at the server). Incoming mail handling is set up using Postfix as the MTA. + +##What next? + +Check out the [next steps]({{ site.baseurl }}docs/installing/next_steps/). + -If you have any problems or questions, please ask on the [Alaveteli Google -Group](https://groups.google.com/forum/#!forum/alaveteli-dev) or [report an -issue](https://github.com/mysociety/alaveteli/issues?state=open). diff --git a/docs/installing/vagrant.md b/docs/installing/vagrant.md new file mode 100644 index 000000000..a0b058da5 --- /dev/null +++ b/docs/installing/vagrant.md @@ -0,0 +1,95 @@ +--- +layout: page +title: Vagrant +--- +# Alaveteli using Vagrant + +<p class="lead"> + <a href="https://www.vagrantup.com">Vagrant</a> provides an easy method to set + up virtual development environments We bundle an example Vagrantfile in the + repository, which runs the + <a href="{{ site.baseurl}}docs/installing/script/">install script</a> for you. +</p> + +Note that this is just one of [several ways to install Alaveteli]({{ site.baseurl }}docs/installing/). + +The included steps will use vagrant to create a development environment +where you can run the test suite and the development server, and make +changes to the codebase. + +The basic process is to create a base virtual machine (VM), and then +provision it with the software packages and setup needed. The supplied +scripts will create you a Vagrant VM based on the server edition of +Ubuntu 12.04 LTS that contains everything you need to work on Alaveteli. + +1. Get a copy of Alaveteli from + <a href="{{ site.baseurl }}docs/glossary/#git" class="glossary__link">GitHub</a>: + + # on your machine + $ git clone git@github.com:mysociety/alaveteli.git + $ cd alaveteli + $ git submodule update --init + +2. Create the Vagrant VM. This will provision the system and can take some time + — sometimes as long as 20 minutes. + + $ vagrant --no-color up + +3. You should now be able to log in to the Vagrant guest OS with `ssh` and run + the test suite: + + $ vagrant ssh + + # You are now in a terminal on the virtual machine + $ cd /home/vagrant/alaveteli + $ bundle exec rake spec + + +4. Run the rails server: + + # in the virtual machine terminal + bundle exec rails server + +You can now visit the application in your browser (on the same machine that is +running Vagrant) at `http://10.10.10.30:3000`. + +If you need to stop the server, simply press **Ctl-C** within that shell. + +It's also possible to stop the server from a different terminal shell in the +Vagrant VM. Log in, find the process ID for the Alaveteli server (in the example +below, this is `1234`), and issue the `kill` command: + + $ vagrant ssh + + # now in a terminal on the virtual machine + $ cat /home/vagrant/alaveteli/tmp/pids/server.pid + 1234 + $ kill -2 1234 + +Alternatively, you can shut down the whole VM without deleting it with the +command <code>vagrant halt</code> +on the host command line. To start it up again, go to step 2, above — it +won't take so long this time, because the files are already in place. +See [the Vagrant documentation](https://docs.vagrantup.com/v2/) +for full instructions on using Vagrant. + +## What next? + +Check out the [next steps]({{ site.baseurl }}docs/installing/next_steps/). + +## Customizing the Vagrant instance + +The Vagrantfile allows customisation of some aspects of the virtual machine. See the customization options in the file [`Vagrantfile`](https://github.com/mysociety/alaveteli/blob/master/Vagrantfile#L30) at the top level of the Alaveteli repository. + +The options can be set either by prefixing the vagrant command, or by +exporting to the environment. + + # Prefixing the command + $ ALAVETELI_VAGRANT_MEMORY=2048 vagrant up + + # Exporting to the environment + $ export ALAVETELI_VAGRANT_MEMORY=2048 + $ vagrant up + +Both have the same effect, but exporting will retain the variable for the duration of your shell session. + |